ML Model Serving Engineer

Added
9 days ago
Type
Full time
Salary
Upgrade to Premium to se...

Related skills

kubernetes gcp pytorch ml llm

πŸ“‹ Description

  • Turbocharge our serving layer, consisting of a variety of LLM, speech, and vision models.
  • Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate
  • Modify and extend LLM serving frameworks like VLLM and SGLang to take advantage of the latest
  • Work with the training team to identify opportunities to produce faster models without sacrificing
  • Use techniques like in-flight batching, caching, and custom kernels to speed up inference.
  • Find ways to reduce model initialization times without sacrificing quality.

🎯 Requirements

  • Expert in some differentiable array computing framework, preferably PyTorch.
  • Expert in optimizing machine learning models for serving reliably at high throughput, with low
  • Significant systems programming experience; ex. Experience working on high-performance server
  • Significant performance engineering experience; ex. Bottleneck analysis in high-scale server
  • Always up to date on the latest techniques for model serving optimization.

🎁 Benefits

  • 401 (k) max employer match: 3.5% of compensation
  • 100% employer-paid health, vision, and dental benefits for you and your dependents
  • Unlimited PTO and sick time
  • Flexible spending account with employer matching up to $1,650/year (medical FSA)
  • Guardian Employee Assistance Program (EAP)
  • Opportunity to share in the company's success with competitive stock options
Get alerts for new remote engineering jobs
A weekly email with new matches. Unsubscribe anytime.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Engineering Jobs. Just set your preferences and Job Copilot will do the rest β€” finding, filtering, and applying while you focus on what matters.

Related Engineering Jobs

See more Engineering jobs β†’