Added
18 days ago
Type
Full time
Salary
Salary not provided

Related skills

java sql python scala airflow

📋 Description

  • Creating and supporting batch, incremental, and real-time ETL pipelines
  • Standardizing ingestion, validation, and cleaning processes across clients
  • Automating data validation to increase data quality
  • Managing and evolving schemas in all parts of our pipeline
  • Monitoring, tuning, and optimizing Spark jobs
  • Becoming a domain-expert in connected vehicle data

🎯 Requirements

  • 3+ years as a data engineer
  • Experience as a tech lead or mentor
  • Proficiency in Python/Scala/Java/C++ and SQL
  • 3+ years of experience with Spark or equivalent technologies
  • 2+ years of experience with a workflow scheduler (Airflow, Prefect, Argo, etc)
  • 2+ years of experience with distributed file-systems (HDFS, S3, etc)
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs →