Senior Data Engineer - Databricks

📋 Description

• Own Databricks production pipelines for Sugar Predict, with monitoring, alerts, and incident response.
• Maintain SLA metrics and platform health visibility for stakeholders.
• Migrate legacy ETL/ELT to Databricks; build automation for reliable data delivery.
• Build high-performance Databricks pipelines for ERP/CRM data across Azure and AWS.
• Enforce data security RBAC, secrets management, and compliance for enterprise data.
• Collaborate with ML engineers, product teams, and EA to keep data fast and clean.

🎯 Requirements

• 4+ years in data engineering.
• 2+ years on Databricks or Spark (Azure/AWS).
• Proficient in PySpark, SQL, Python; production pipelines under SLA.
• Strong PostgreSQL experience in production data pipelines.
• Operate Databricks across Azure and AWS; cost governance and cluster mgmt.
• Delta Lake: schema evolution, ACID, incremental/streaming.

🎁 Benefits

• Excellent healthcare package for you and your family
• 401(k) match
• Unlimited Paid Time Off
• Paid Parental Leave
• Online Legal Services (Rocket Lawyer)
• Financial Planning Services (Origin)