Nvidia · posted · checked
Senior Product Manager – AI Inference Performance
$208k – $328kOn-site · Santa ClaraSenior
Who can apply
On-site · Santa Clara
Location: US, CA, Santa Clara
NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s an outstanding legacy of innovation that’s fueled by great technology—and amazing people. Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.!
NVIDIA is looking for a highly technical Product Manager to own the products that help customers extract the best possible performance from AI models and applications running on NVIDIA hardware. Every inference deployment — from a single-GPU workstation to a multi-thousand-GPU data center — lives or dies on latency, efficiency, and cost per token. Your job is to make NVIDIA the obvious place to run inference by turning deep optimization techniques into products that a broad range of customers can actually adopt. The work spans the entire inference stack — optimization techniques, the frameworks that deliver them; and the benchmarking and operational tooling customers rely on to trust the results. You will translate what our best-performing internal deployments do into capabilities that ship, are detailed, and work for everyone else. The Product Management organization at NVIDIA is a small, high-leverage team driving the company’s Deep Learning and Generative AI strategy. We need a self-starter who can operate with minimal direction, form a point of view from data and customer conversations, and drive it to a shipped result. If that sounds like you, we’d love to talk.!
What You'll Be Doing:
Own the inference performance roadmap. Set direction across the stack: how models are represented, how memory and state are managed, how requests are scheduled and served, and how tokens get generated. The techniques change fast. Judge which ones matter, then decide what we build, what we adopt, and what we retire.
Build platforms, not one-offs. Deliver capabilities that generalize across model families, deployment topologies, and customer sizes. Build for easy adoption, sane defaults, and extensibility.
Agentic and Multi-Turn Workloads: Define the performance strategy for agentic applications, where long-running sessions, tool-call stalls, and unexpected output lengths break the assumptions built into single-turn serving. Drive capabilities around cross-turn cache reuse, request prioritization, and efficient handling of idle time in agent loops.