Senior Data Engineer - Web Scraping

📋 Description

• Design, develop, deploy, and maintain web scrapers for diverse sources.
• Use Python and Pandas to clean and transform large datasets.
• Build data pipelines to ingest scraped data into databases/data warehouses.
• Develop and manage scheduled workflows with Apache Airflow.
• Collaborate with analysts and stakeholders to translate requirements into solutions.
• Implement quality checks to ensure data availability and integrity.

🎯 Requirements

• Bachelor’s or Master’s in CS/Engineering or related field.
• 4–6 years in data engineering or related field.
• Strong Python, SQL, and database technologies.
• Advanced Pandas for data cleaning and transformation.
• Web-scraping experience with Selenium, Scrapy, Fiddler, Postman, XPath.
• Experience with Apache Airflow for workflow orchestration.

🎁 Benefits

• Fully remote position with flexibility to work from anywhere.
• Full-time role in a collaborative engineering environment.
• Autonomy, ownership, and opportunity to work on sophisticated data platforms.
• Exposure to complex datasets supporting analytical/investment decisions.
• Collaboration with engineers, product, analysts, and data scientists.
• Entrepreneurial, innovative team with focus on reliability.