Senior Data Engineer - Azure Databricks

📋 Description

• Design, build, and maintain scalable data pipelines using Azure Databricks.
• Ingest data from APIs, databases, files, and streaming systems.
• Develop data transformation logic with PySpark and Spark SQL.
• Ensure data quality, reliability, and performance across pipelines.
• Collaborate with data scientists, BI teams, and stakeholders on data needs.
• Implement security, governance, and compliance within Azure.

🎯 Requirements

• 5+ years of hands-on Data Engineer experience
• Strong Azure Databricks experience
• PySpark, Spark SQL, and Python
• Experience with ADLS Gen2
• Data warehousing and data modeling concepts
• Experience building batch and/or streaming pipelines

🎁 Benefits

• Experience with Delta Lake
• Streaming tech knowledge (Kafka, Event Hubs, Spark Streaming)
• CI/CD pipelines using Azure DevOps
• Agile/Scrum environments
• Cloud cost optimization strategies