Senior Software Engineer, Data

📋 Description

• Build and operate large-scale data transfer pipelines for video, LiDAR and sensor data from global
• Contribute to the technical design of the Data Transfer Hub, balancing UX, reliability, throughput
• Develop parallelised, distributed data transfer pipelines using Flyte for workflow orchestration
• Help implement and evolve a globally distributed hub-and-spoke data transfer model to share sensor
• Write infrastructure-as-code with Terraform and pipeline logic in Python
• Build reliable operational foundations including retry strategies, backfills, data integrity

🎯 Requirements

• Experience building and operating large-scale data transfer or data processing pipelines, multi-TB
• Strong software engineering skills
• Experience delivering and maintaining production systems
• Experience with workflow orchestration for parallelised jobs; Flyte preferred, but Airflow, Prefect
• Experience building observable data systems with retry, backfills, data integrity checks and
• Strong cloud platform skills, with Azure preferred; AWS or GCP a plus

🎁 Benefits

• Hybrid working policy with time in the London office and time working from home
• Collaborative, fast-paced engineering culture
• Opportunity to impact data infrastructure used by partners and customers worldwide