Data Engineer

📋 Description

• Owner of the core data pipeline, scaling data processing to meet rapid growth
• Evolve data model and data schema based on business and engineering needs
• Implement systems tracking data quality and consistency
• Develop tools supporting self-service data pipeline management (ETL)
• SQL and MapReduce job tuning to improve data processing performance
• Write well-crafted, well-tested, readable, maintainable code

🎯 Requirements

• 4+ years of professional experience in data engineering, ideally with large-scale distributed systems
• Strong experience with Spark
• Experience with Hadoop (or similar) Ecosystem, S3, DynamoDB, MapReduce, Yarn, HDFS, Hive, Spark, Presto, Pig, HBase, Parquet
• Strong skills in a scripting language (Python, Go)
• Good understanding of SQL Engine and able to conduct advanced performance tuning
• Proficient in at least one of the SQL languages (SparkSQL, Trino)