Lead Data Engineer, Databricks

Added
19 hours ago
Location
Type
Full time
Salary
Salary not provided

Related skills

azure aws sql python databricks

📋 Description

  • Lead data engineering activities across customer engagements, from initial design and
  • Build, maintain, and optimize batch and streaming data pipelines using PySpark, SQL, and Delta Lake
  • Implement Medallion architecture across cloud environments and establish consistent data layer
  • Take ownership of platform performance and cost efficiency by benchmarking Spark jobs, tuning SQL
  • Implement data quality and validation processes as code, including Lakeflow expectations and
  • Establish and maintain engineering standards, ensuring the reliability, scalability, and quality of

🎯 Requirements

  • At least 7 years of professional data engineering experience, including experience leading projects
  • Strong proficiency in Python, SQL, Apache Spark, and Delta Lake.
  • Hands-on production experience building both batch and streaming data pipelines at enterprise scale.
  • Practical experience with at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud
  • Experience with data modeling, Git, and CI/CD practices for data engineering workloads.
  • Demonstrated ability to optimize Spark performance, improve pipeline efficiency, and manage

🎁 Benefits

  • Unlimited paid time off.
  • Medical, dental, and vision coverage.
  • Retirement savings plan, including 401(k) and IRA options.
  • Basic and voluntary life insurance.
  • Accidental Death and Dismemberment (AD&D) coverage.
  • Maternity and paternity leave.
Get alerts for new remote data jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs →