Senior Data Engineer - Web Scraping

📋 Description

• Design, develop, deploy, and maintain web scrapers using diverse techniques.
• Use Python and Pandas to clean and prepare large datasets.
• Build data pipelines to ingest scraped data into databases and data warehouses.
• Develop and manage scheduled workflows with Apache Airflow and other tools.
• Collaborate with analysts and stakeholders to translate data requirements into solutions.
• Develop quality-control checks for data availability and integrity.

🎯 Requirements

• Bachelor’s or master’s in Computer Science, Engineering, or related field.
• 4–6 years of experience in data engineering or related field.
• Strong Python programming and SQL/database knowledge.
• Advanced Python Pandas experience for data cleaning and manipulation.
• Solid web-scraping experience with Selenium, Scrapy, Fiddler, Postman, XPath.
• Experience with Apache Airflow for workflow orchestration.

🎁 Benefits

• Fully remote position with flexible work location.
• Full-time opportunity in a collaborative engineering environment.
• Significant autonomy and ownership of technical challenges.
• Opportunity to work on sophisticated web-scraping and data-product initiatives.
• Exposure to complex datasets supporting analytical and investment decisions.
• Collaborative environment with engineers, product, analysts, data scientists, and other tech