Engineer III - Data Engineer

📋 Description

• Design, build, and run batch and scheduled data pipelines (Airflow, Dagster, Prefect) and lead
• Own and extend data collectors and engines, including web/API ingestion, large-scale web scraping
• Build scalable distributed processing architectures (Spark on EMR, Databricks, Dataproc) and
• Improve entity matching and caching services (Redis, MemoryDB, DynamoDB) to optimize accuracy
• Develop and maintain high-performance Python data access layers and APIs for downstream ML and
• Own data quality initiatives, including validation frameworks, retention policies, backfills

🎯 Requirements

• 6+ years of software or data engineering experience, including 3+ years of experience building and
• Strong Python development skills with a commitment to writing clean, tested code (using pytest
• Deep experience with workflow orchestration (Airflow DAG design, operators, scheduling, and failure
• Hands-on experience with distributed compute (Spark) and cloud data ecosystems (AWS EMR/Athena/S3
• Solid SQL and data-modeling skills, alongside practical experience with NoSQL databases, caching
• Comfort deploying and managing containerized workloads on Kubernetes using Docker and Helm

🎁 Benefits

• Diversity, Equity, Inclusion and Belonging (DEIB) initiatives
• Reasonable accommodation for individuals with disabilities during the application or interview
• Disclosure of use of AI or automated tools in the recruitment process