Senior Data Infrastructure Engineer (Scraping & Scale)

📋 Description

• Lead the development and maintenance of scalable data collection systems
• Design and manage web scraping infrastructure using residential proxy networks, TLS fingerprinting
• Build fault-tolerant, high-throughput scraping pipelines to collect data from Instagram, TikTok
• Develop distributed queues and workflow orchestration systems using technologies such as Temporal
• Architect storage environments for structured and unstructured data using Parquet, Apache Iceberg
• Implement monitoring, automated alerts, and recovery mechanisms to identify platform changes

🎯 Requirements

• At least 4 years of experience in high-volume web scraping, data engineering, or reverse engineering
• Strong experience handling advanced anti-bot systems, including Cloudflare, Akamai, and PerimeterX
• Advanced proficiency in Python or Go
• Extensive experience with browser automation and scraping frameworks such as Playwright, Puppeteer
• Practical knowledge of distributed systems and task queues, including Temporal, Ray, Kafka, and
• Strong SQL skills and experience with modern analytical data lakes and data warehouses

🎁 Benefits

• Fully remote work arrangement
• Full-time employment