Software Engineer, Data Ingestion

📋 Description

• Debug and unblock failing ingestion pipelines, stalled jobs and processing queues
• Investigate corrupt, malformed or unexpected data from a variety of sources
• Optimise Spark jobs and batch-processing pipelines for throughput, reliability and compute
• Design and implement more resilient pipelines so individual bad data segments do not block wider
• Improve monitoring, retries and failure recovery across multi-step ingestion workflows
• Help prioritise and unblock important datasets for downstream annotation, data science and

🎯 Requirements

• Strong production experience with Apache Spark and Python
• Experience building, operating or debugging large-scale ingestion, ETL or distributed
• Comfortable diagnosing production failures involving corrupt data, stalled queues, retries and
• Understanding of how to optimise high-throughput systems for reliability, performance and cost
• Ability to learn an existing system quickly, work independently and deliver practical improvements

🎁 Benefits

• Salaries benchmarked against the market annually
• Meaningful equity, sharing in the ownership and long term success of Wayve
• Relocation support and visa sponsorship where applicable
• Hybrid working, core hours and the chance to work hands on in vehicle workshops and labs
• Learning and development budgets with support for training, conferences and growth
• Comprehensive benefits including health insurance, dental, enhanced maternity and paternity leave