Nvidia · posted · checked
Principal Data and Platform Engineer, DGX Cloud
KafkaPythonSparkSQL$248k – $391kOn-site · Santa ClaraStaff+
Who can apply
On-site · Santa Clara
Location: US, CA, Santa Clara
NVIDIA's DGX Cloud organization is looking for a lead expert in data and platform engineering to guide the building, creation, and management of the DGXC Data Platform. Our group links data across cloud service providers, NVIDIA Cloud Partners, and internal systems. We develop common data functions that assist DGXC teams in publishing, discovering, joining, and using reliable data within operational workflows.
We are seeking a Principal Data & Platform Engineer who can transform complex operational needs into reliable platform services and shared datasets. You will shape the technical approach across ingestion and orchestration, data agreements, quality monitoring, cataloging, and managed access. You will lead delivery and adoption among teams, set production standards, and address technical issues that cross organizational lines.
Your work will reduce repeated integration effort, improve confidence in operational data, and make the platform easier to adopt and operate. Our primary implementation environment is Python, SQL, Databricks, and Spark.
What you'll be doing:
Define the architecture and multiyear technical roadmap, setting system boundaries, shared interfaces, and engineering standards across ingestion, processing, storage, and data serving. Balance immediate operational needs with long term reliability, maintainability, and cost.
Lead delivery across platform engineers, data producers, and consumers. Translate ambiguous requirements into actionable designs and milestones, resolve dependencies, and guide implementation through production adoption.
Build and operate reusable batch and streaming ingestion capabilities for service provider and internal data. Establish supported patterns for APIs, change data capture, streams, database connectors, and files, including orchestration, retries, backfills, and recovery.
Establish shared identifiers, data contracts, consistent modeling, and automated quality and certification capabilities. Partner with source owners to define dataset meaning, keys, supported joins, historical coverage, compatibility, and ownership. Make quality failures actionable and dataset limitations visible to consumers.
Guide cataloging, lineage, and governed access capabilities that help people and AI agents discover datasets, understand their definitions and relationships, and follow approved access procedures.
Make onboarding and operations repeatable through shared libraries, reference implementations, deployment automation, and self service workflows that teams can adopt in their repositories. Measure improvements in onboarding effort, dataset reuse, and manual toil.