Senior Data Engineer

📋 Description

• Design, develop, and optimize scalable ETL/ELT data pipelines for batch and real-time processing
• Implement and maintain Medallion Architecture, organizing data into Bronze, Silver, and Gold layers
• Process terabyte-scale datasets using distributed computing techniques, ensuring pipeline
• Manage Databricks environments, including workflows, clusters, Delta Lake, and related lakehouse
• Implement Unity Catalog for metadata management, granular access controls, OAuth-based
• Develop and execute cloud migration strategies for legacy databases, including Oracle, Teradata

🎯 Requirements

• Four to seven years of professional experience in data engineering, big data development, or a
• Advanced hands-on proficiency in Python, SQL, and PySpark, including expert-level SQL query
• Strong practical experience with Apache Spark, Spark Structured Streaming, Delta Live Tables, and
• Demonstrated experience working with terabyte-scale datasets and distributed data processing
• Extensive working knowledge of Databricks, including Workflows, Unity Catalog, Delta Lake, and
• Experience with at least one major cloud platform: Google Cloud Platform (GCP), Amazon Web Services

🎁 Benefits

• Remote working arrangement within India.
• Full-time employment opportunity.
• Opportunities to work with modern data engineering technologies, including Python, Spark
• Exposure to large-scale data processing, cloud migration, automation, and machine learning-related
• Opportunities for professional learning, technical development, and collaboration with colleagues
• A workplace culture emphasizing transparency, diversity, integrity, continuous learning, and career