SWE, Data Ingestion

📋 Description

• Debug and resolve failing or blocked ingestion pipelines
• Investigate issues caused by corrupt, malformed or unexpected data
• Design and implement more resilient pipelines so individual bad data segments do not block wider
• Improve how we handle varied data formats from partners, suppliers and third-party sources
• Support orchestration across multi-step ingestion workflows, including dependencies, retries and
• Optimise Spark jobs and data-processing pipelines for throughput, compute efficiency and reliability

🎯 Requirements

• Strong production experience with Apache Spark
• Strong Python engineering experience
• Experience building, debugging or operating large-scale data-ingestion, ETL or data-processing
• Experience with distributed data-processing systems
• Ability to optimise jobs for throughput, compute efficiency and reliability
• Experience debugging production pipeline failures

🎁 Benefits

• Competitive equity package
• Hybrid working policy
• Inclusive and diverse work environment