Sumo Logic · posted · checked
Senior Data Engineer
AWSPythonSparkSQLOn-site · Uttar PradeshSenior
Who can apply
On-site · Uttar Pradesh
Location: Noida, Uttar Pradesh, India
Sr Data Engineer-Noida(Hybrid)
We are looking for a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines and integrations within our Databricks-based data platform. The role will focus on expanding our data ecosystem by onboarding new data sources, integrating enterprise applications, and delivering reliable, secure, and analytics-ready datasets.
The ideal candidate will have strong hands-on expertise in Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake, along with experience building production-grade data pipelines and enterprise integrations. Working knowledge of AWS infrastructure and cloud services is important for collaborating with Engineering and other platform teams on connectivity, deployments, and evolving infrastructure requirements.
The candidate will report directly to the Senior Manager - Business Insights & Analytics and collaborate with Data Analytics, Engineering, IT/Cloud, Security, and business stakeholders to deliver scalable and reliable data solutions.
Responsibilities:
Design, develop, and optimize scalable data pipelines using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake.
Design and implement a medallion architecture (Bronze, Silver, and Gold layers) to deliver trusted, well-structured, analytics-ready datasets.
Build and maintain integrations with enterprise applications, REST APIs, databases, and other source systems
Developing reusable ingestion frameworks to accelerate onboarding of new data sources.
Build reliable batch and near-real-time ingestion pipelines with incremental loads, CDC, schema evolution and error recovery.
Configure and manage Databricks Workflows and implement ingestion and transformation pipelines using Auto Loader and Lakeflow Spark Declarative Pipelines
Optimize Apache Spark and Delta Lake workloads using execution-plan analysis, efficient join strategies, data-skew handling, partitioning, and SQL query tuning to improve performance and control compute costs.
Implement Unity Catalog governance, including access controls, lineage, auditing, and sensitive-data protection through masking and row-level security.