Added
11 days ago
Type
Full time
Salary
Upgrade to Premium to se...

Related skills

etl sql postgres python pyspark

šŸ“‹ Description

  • Design, build, and optimize scalable data pipelines in Apache NiFi to automate ingestion
  • Ensure pipelines meet low-latency and high-throughput requirements for distributed processing.
  • Architect and manage datasets on HDFS to support high-volume, fault-tolerant storage.
  • Develop distributed processing workflows in PySpark and Hive to handle large-scale transformations
  • Implement partitioning, bucketing, and indexing strategies to optimize query performance.
  • Maintain and tune Postgres databases for high availability, integrity, and performance.

šŸŽÆ Requirements

  • Bachelor’s degree in Computer Science, Information Technology, or related field (Master’s
  • Proven experience as a Data Engineer with expertise in HDFS, Apache NiFi, Hive, PySpark, Postgres
  • Strong background in ETL/ELT design, distributed processing, and relational database management.
  • Experience with on-premises big data ecosystems supporting distributed computing.
  • Solid debugging, optimization, and performance tuning skills.
  • Ability to work in agile environments, collaborating with multi-disciplinary teams.
Get alerts for new remote data jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs →