Nvidia · posted · checked
Senior Software Engineer, Datacenter Automation Platforms
KubernetesLinuxNode.jsPythonRust$184k – $288kOn-site · Santa ClaraSenior
Who can apply
On-site · Santa Clara
Location: US, CA, Santa Clara
Collaborate with NVIDIA professionals responsible for building the platforms that operate AI factories at scale. We drive product roadmaps, architecture, multi-functional alignment, and implementation across multiple open source and internal-facing products. We partner with executive leadership to align platform strategy and drive adoption of innovative approaches to software development across engineering teams. Our team mentors engineering talent and turns successful domain solutions into reusable platform capabilities used across the organization.
What you'll be doing:
Build self-service SDKs, workflow templates, reference implementations, and a catalog of reusable agents, tools, and connectors. Prioritize automation by customer impact, frequency, operational risk, and encouraged reduction in human effort.
Translate operational intent into versioned, testable policies that define triggers, prerequisites, permitted actions, approval requirements, maintenance windows, capacity limits, and recovery or blocking issue paths.
Define common contracts for tools, state, events, and workflow execution. Establish versioning and compatibility standards, guide migration from fragmented implementations to shared services, and select the runtime and durable execution architecture.
Build agents that correlate hardware diagnostics, logs, metrics, topology, configuration, and incident history to investigate failures, propose evidence-backed diagnoses, and produce executable remediation plans. Enforce authorization independently of model output, retain action histories, and support human review, bounded retries, cancellation, and compensating actions.
Automate provisioning validation, node quarantine, workload-aware maintenance, repair, upgrades, and return to service. Integrate with Kubernetes, fleet management systems, and relevant schedulers and hardware interfaces.
Tackle fleet-scale problems including partial failures, duplicate events, stale state, conflicting remediations, noisy telemetry, control-plane bottlenecks, and correlated rack or cluster failures.
Establish shared evaluation, CI/CD, staged rollout, and observability capabilities across teams.
Drive cross-team adoption through build reviews, migration plans, documentation, and hands-on partnership. Align infrastructure, networking, storage, security, and workload teams on shared priorities and contribution standards.