Added
1 hour ago
Location
Type
Full time
Salary
Salary not provided

Related skills

s3 python databricks scala apache spark

📋 Description

  • Design, build, and maintain scalable ETL/ELT pipelines with Databricks.
  • Ingest batch and streaming data from databases, APIs, and events.
  • Ensure fault-tolerant, efficient, and cost-optimized pipelines on AWS.
  • Develop data lake/lakehouse architectures using AWS S3, Delta Lake, and Databricks.
  • Implement medallion architecture (Bronze/Silver/Gold).
  • Optimize storage formats and partitioning (Parquet, Delta).

🎯 Requirements

  • Databricks: Hands-on experience.
  • Proficiency in Apache Spark (PySpark/Scala).
  • Strong Spark SQL and performance optimization.
  • Delta Lake and lakehouse architectures.
  • Python for data engineering; clean, testable code.
  • SQL for data transformation; Git version control.
  • Soft skills: problem-solving; independent work; collaboration.
  • Nice to have: Unity Catalog, BI analytics, cost optimization, Spark knowledge.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs →