Scale AIAI Infrastructure Engineer, Model Serving Platform
San Francisco, CA; New York, NY · 全职 · 算法与研究
需要对标的能力
大模型原理 · LLM 基础Prompt Engineering · 提示词工程工具调用 · Function CallingAgent 工程 · Harness / 脚手架AI Infra · 推理部署评测 · Eval 方法论AI 工程设计模式
岗位职责与要求
As a Software Engineer on the ML Infrastructure team, you will design and build platforms for scalable, reliable, and efficient serving of LLMs. Our platform powers cutting-edge research and production systems, supporting both internal and external use cases across various environments.
The ideal candidate combines strong ML fundamentals with deep expertise in backend system design. You’ll work in a highly collaborative environment, bridging research and engineering to deliver seamless experiences to our customers and accelerate innovation across the company.
You will:
· Build and maintain fault-tolerant, high-performance systems for serving LLMs workloads at scale.
· Build an internal platform to empower LLM capability discovery.
· Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
· Conduct architecture and design reviews to uphold best practices in system design and scalability.
· Develop monitoring and observability solutions to ensure system health and performance.
· Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.
Ideally you'd have:
· 4+ years of experience building large-scale, high-performance backend systems.
· Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++).
· Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load balancing, budgets, etc.)
· Experience with LLM capabilities and concepts such as reasoning, tool calling, prompt templates, etc.
· Experience with containers and orchestration tools (e.g., Docker, Kubernetes).
· Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform).
· Proven ability to solve complex problems and work independently in fast-moving environments.
Nice to haves:
· Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference.
$180,000—$225,000 USD
PLEASE NOTE: Our policy requires a 90-day waiting period before reconsidering candidates for the same role. This allows us to ensure a fair and thorough evaluation of all applicants.
About Us:
查看中文参考译文
作为 ML Infrastructure 团队的 Software Engineer，你将设计并构建用于可扩展、可靠且高效地 serving LLMs 的平台。我们的平台为前沿研究和生产系统提供支持，覆盖各种环境下的内部和外部 use cases。
理想的候选人兼具扎实的 ML 基础和深厚的 backend system design 专业知识。你将在一个高度协作的环境中工作，连接 research 和 engineering，为我们的客户交付无缝体验，并加速整个公司的创新。
你将：
· 构建并维护用于大规模 serving LLMs workloads 的容错、高性能系统。
· 构建一个内部平台，以增强 LLM capability discovery。
· 与 researchers 和 engineers 协作，针对生产和研究 use cases 集成并优化 models。
· 开展 architecture 和 design reviews，以维护 system design 和 scalability 方面的最佳实践。
· 开发 monitoring 和 observability 解决方案，以确保 system health 和 performance。
· 在跨职能环境中端到端地领导项目，从 requirements gathering 到 implementation。
理想情况下，你具备：
· 4 年以上构建大规模、高性能 backend systems 的经验。
· 熟练掌握一种或多种编程语言（例如，Python、Go、Rust、C++）。
· 具备 LLM serving 和 routing 基础方面的经验（例如 rate limiting、token streaming、load balancing、budgets 等）。
· 具备 LLM capabilities 和 concepts 方面的经验，例如 reasoning、tool calling、prompt templates 等。
· 具备 containers 和 orchestration tools 方面的经验（例如，Docker、Kubernetes）。
· 熟悉 cloud infrastructure（AWS、GCP）和 infrastructure as code（例如，Terraform）。
· 经过验证的解决复杂问题的能力，并能在快速变化的环境中独立工作。
加分项：
· 具备现代 LLM serving frameworks 方面的经验，例如 vLLM、SGLang、TensorRT-LLM 或 text-generation-inference。
$180,000—$225,000 USD
请注意：我们的政策要求在同一岗位的候选人重新考虑之前须等待 90 天。这使我们能够确保对所有申请人进行公平且全面的评估。
关于我们：
译文仅供辅助阅读，具体要求以原始招聘来源为准。