Scale AIAI Infrastructure Engineer, Serving Platform
London, UK · 全职 · 工程与基础设施
需要对标的能力
大模型原理 · LLM 基础Prompt Engineering · 提示词工程工具调用 · Function CallingAgent 工程 · Harness / 脚手架AI Infra · 推理部署评测 · Eval 方法论AI 工程设计模式
岗位职责与要求
As a Software Engineer on the ML Infrastructure team, you will design and build platforms for scalable, reliable, and efficient serving of LLMs. Our platform powers cutting-edge research and production systems, supporting both internal and external use cases across various environments.
The ideal candidate combines strong ML fundamentals with deep expertise in backend system design. You’ll work in a highly collaborative environment, bridging research and engineering to deliver seamless experiences to our customers and accelerate innovation across the company.
You will:
· Build and maintain fault-tolerant, high-performance systems for serving LLMs and other models at scale.
· Build an internal platform to empower LLM capability discovery.
· Collaborate with researchers and engineers to integrate and optimize models for production and research use cases.
· Conduct architecture and design reviews to uphold best practices in system design and scalability.
· Develop monitoring and observability solutions to ensure system health and performance.
· Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment.
Ideally you'd have:
· 4+ years of experience building large-scale, high-performance backend systems.
· Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++).
· Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load balancing, budgets, etc.)
· Experience with LLM capabilities and concepts such as reasoning, tool calling, prompt templates, etc.
· Experience with containers and orchestration tools (e.g., Docker, Kubernetes).
· Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform).
· Proven ability to solve complex problems and work independently in fast-moving environments.
Nice to haves:
· Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference.
PLEASE NOTE: Our policy requires a 90-day waiting period before reconsidering candidates for the same role. This allows us to ensure a fair and thorough evaluation of all applicants.
About Us:
查看中文参考译文
作为 ML Infrastructure 团队的 Software Engineer，你将设计并构建可扩展、可靠且高效的 LLM serving 平台。我们的平台为前沿研究和生产系统提供支持，覆盖内部和外部使用场景，适用于多种环境。
理想的候选人兼具扎实的 ML 基础与深厚的后端系统设计专长。你将在高度协作的环境中工作，连接研究与工程，为客户交付无缝体验，并加速全公司的创新。
你将：
· 构建并维护容错、高性能的系统，用于大规模 serving LLM 及其他模型。
· 构建内部平台，赋能 LLM 能力发现。
· 与研究人员和工程师协作，针对生产和研究用例集成并优化模型。
· 开展架构和设计评审，确保系统设计和可扩展性的最佳实践。
· 开发监控和可观测性解决方案，确保系统健康与性能。
· 在跨职能环境中端到端主导项目，从需求收集到落地实施。
理想情况下，你具备：
· 4 年以上构建大规模、高性能后端系统的经验。
· 熟练掌握一种或多种编程语言（例如 Python、Go、Rust、C++）。
· 具备 LLM serving 和 routing 基础经验（例如 rate limiting、token streaming、load balancing、budgets 等）。
· 具备 LLM 能力和概念经验，例如 reasoning、tool calling、prompt templates 等。
· 具备容器和编排工具经验（例如 Docker、Kubernetes）。
· 熟悉 cloud infrastructure（AWS、GCP）和 infrastructure as code（例如 Terraform）。
· 在快节奏环境中解决复杂问题并独立工作的能力。
加分项：
· 具备现代 LLM serving 框架经验，例如 vLLM、SGLang、TensorRT-LLM 或 text-generation-inference。
请注意：我们的政策要求对同一岗位的候选人设置 90 天的重新考虑等待期。这有助于我们确保对所有申请者进行公平且充分的评估。
关于我们：
译文仅供辅助阅读，具体要求以原始招聘来源为准。