Research Engineer - Web Crawlers

Added
7 hours ago
Type
Full time
Salary
Salary not provided

Related skills

kubernetes machine learning distributed systems data pipelines queues

πŸ“‹ Description

  • Build and operate large-scale, distributed web crawlers
  • Fetch and extract data across billions of pages reliably
  • Address content extraction, deduplication, freshness and recrawl
  • Design pipelines for audio/video/multilingual data
  • Create tooling to request, monitor, explore crawled data

🎯 Requirements

  • Hands-on experience building/scaling web crawlers
  • Distributed systems expertise (Kubernetes, queues, pipelines)
  • Ability to evaluate data quality, coverage, and compliance

🎁 Benefits

  • Innovative culture and impact-focused work
  • Growth opportunities and learning stipends
  • Annual offsite and co-working stipend
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest β€” finding, filtering, and applying while you focus on what matters.

Related Engineering Jobs

See more Engineering jobs β†’