Thinking Machines Lab · posted · checked
Software Engineer, Inference
Python$300k – $400kOn-site · New York and San Francisco
Who can apply
On-site · New York and San Francisco
Location: New York / San Francisco
ABOUT THINKING MACHINES
The mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.
ABOUT THE ROLE
We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models to real users. Our research and inference teams push the limits of model performance and serving efficiency; this role makes sure those gains reach production safely and stay up — powering Tinker's live, multi-tenant serving and the products built on top of our models.
This is a production-facing systems role at the center of the company. You'll be the bridge between cutting-edge inference techniques and the day-to-day reality of serving real traffic: rollouts, capacity, incidents, and everything that keeps a fast-growing platform online.
WHAT YOU'LL DO
Operate and scale the production inference systems that serve live traffic, including Tinker's multi-tenant serving platform
Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment to production
Build and improve observability, alerting, and capacity planning so the team can detect, diagnose, and resolve production issues quickly
Partner with inference and research teams to productionize new serving techniques without compromising reliability
Lead incident response for production inference issues, driving root cause analysis and durable fixes
Design for graceful degradation, failover, and redundancy so that serving stays resilient as usage grows
Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale