Member of Technical Staff (Web Access Engineer, Crawler)

Added
1 hour ago
Type
Full time
Salary
Salary not provided

Related skills

http rust python go dns

๐Ÿ“‹ Description

  • Own fetching and rendering improvements end to end, from investigation through implementation and
  • Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side
  • Improve page loading, completion detection, and content capture to preserve relevant content and
  • Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself
  • Build tools to reproduce failures, compare crawler output against expected page content, and turn
  • Define metrics for fetching reliability, content completeness, and error classification, and use

๐ŸŽฏ Requirements

  • 5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling
  • Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching
  • Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and
  • Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM
  • Comfort working independently when the path forward is not fully defined.

๐ŸŽ Benefits

  • Experience developing CDN, caching, or web performance systems.
  • Experience operating crawlers, headless browsers, or distributed content-acquisition systems at
  • Contributions to a browser engine or JavaScript runtime.
Get alerts for new remote engineering jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest โ€” finding, filtering, and applying while you focus on what matters.

Related Engineering Jobs

See more Engineering jobs โ†’