Master Data Developer (Databricks)

Added
6 hours ago
Type
Full time
Salary
Salary not provided

Related skills

aws etl s3 databricks pyspark
JobCopilot logo
Meet JobCopilot: Your Personal Al Job Hunter
Automatically Apply to Your Dream Jobs While You Sleep
Try it now โ†’

๐Ÿ“‹ Description

  • Build, optimize, and maintain scalable data pipelines and ETL/ELT processes.
  • Implement data architectures including data lakes, warehouses, and integration layers.
  • Implement data quality checks, validation, and monitoring systems.
  • Develop and maintain integrations between data sources and analytics platforms.
  • Collaborate with Data & Analytics Managers and business stakeholders.
  • Monitor and optimize pipeline performance, query efficiency, and storage costs.

๐ŸŽฏ Requirements

  • Hands-on Databricks migration using databricks/migrate tool (notebooks, jobs, clusters, secrets
  • Notebook and job migration: notebooks, DAG pipelines, DLT pipelines.
  • Storage path refactoring: /mnt/ DBFS references to S3-backed locations.
  • Data migration: Deltalake to Iceberg format on S3 with incremental sync.
  • Streaming: Auto Loader / DLT checkpoint re-bootstrap, event_id dedup.
  • AWS services: S3, Glue, Secrets Manager, IAM, Athena, LakeFormation, EMR; Languages: Spark SQL

๐ŸŽ Benefits

  • Premium Healthcare
  • Meal voucher
  • Maternity and Parental leaves
  • Mobile services subsidy
  • Sick pay and Life insurance
  • CI&T University and paid vacations
Get alerts for new remote data jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest โ€” finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs โ†’