Master Data Developer (Databricks)

📋 Description

• Build, optimize, and maintain scalable data pipelines and ETL/ELT processes
• Implement data architectures including data lakes, warehouses, and integration layers
• Implement data quality checks, validation processes, and monitoring systems
• Develop and maintain integrations between data sources and analytics platforms
• Collaborate with Data & Analytics Managers and business stakeholders
• Monitor and optimize data pipeline performance, query efficiency, and storage costs

🎯 Requirements

• Hands-on Databricks migration with the databricks/migrate tool
• Notebook and job migration including DAG pipelines and DLT pipelines
• Storage path refactoring from DBFS to S3-backed locations
• Data migration from Delta to Iceberg format on S3
• Streaming experience with Auto Loader and DLT checkpoint re-bootstrap
• Validation techniques such as row counts and checksums

🎁 Benefits

• Health and dental insurance
• Meal and food allowance
• Childcare assistance
• Extended paternity leave
• Partnership with gyms via Wellhub
• Profit sharing and results participation