Member of Technical Staff | Knowledge Platform

Added
2 minutes ago
Type
Full time
Salary
Salary not provided

Related skills

javascript sql python kubernetes dbt

πŸ“‹ Description

  • Build and run the acquisition layer: crawlers, scrapers and browser RPA that collect data from
  • Keep collection reliable as sources change: detect layout and access changes early, track coverage
  • Run event-driven ingestion on Kafka and Pub/Sub, including ordering, replays, dead-letter queues
  • Build batch and incremental pipelines in Spark, dbt and SQL that turn raw captures into clean
  • Own the lakehouse on Parquet and Iceberg: schema evolution, partitioning, compaction, and snapshots
  • Set data quality and lineage standards: contracts, tests, and traceability from a prediction back

🎯 Requirements

  • 5+ years of relevant engineering experience at tech and product-driven companies.
  • Required: a production background in at least one of web scraping, large-scale crawlers, or browser
  • Required: a data engineering background with large volumes of data, using Spark, dbt and SQL, on
  • Experience with messaging and streaming systems in production: Kafka and/or Pub/Sub.
  • Production code in Python and JavaScript, from pipelines and crawlers to browser automation.
  • Hands-on Kubernetes experience, including packaging with Helm.

🎁 Benefits

  • Work in a frontier AI lab building relational foundation models from the ground up.
  • Collaborate with leading enterprises in LatAm, including largest banks and marketplaces.
  • Be part of a team of builders.
  • Accelerated by NVIDIA, AWS, and Google.
Get alerts for new remote data jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest β€” finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs β†’