73 Strings · posted · checked
Senior Data Engineer
AzureKafkaSnowflakeSQLRemote · Canada and IndiaSenior
Who can apply
Remote · Canada and India
Location: Bengaluru, India / Fully remote / Toronto, Canada
OVERVIEW OF 73 STRINGS:
73 Strings is an innovative platform providing comprehensive data extraction, monitoring, and valuation solutions for the private capital industry. The company's AI-powered platform streamlines middle-office processes for alternative investments, enabling seamless data structuring and standardization, monitoring, and fair value estimation aAbout the role
We are hiring a Senior Data Engineer to build and operate the pipelines, integrations and warehouse that support valuation and monitoring.
You will own production pipelines from source capture through transformation, reconciliation and client delivery, and put them under GitHub, automated test and CI/CD. The platform currently captures change data from relational systems, processes it on Azure Databricks, and delivers it to Snowflake, Microsoft SQL Server and Databricks. The capture method may change. Copied client pipelines are being replaced by metadata-driven components, with data contracts, quality rules and lineage.
What you will do
Help redefine the platform’s architecture across ingestion, processing and delivery, so it’s stable, secure and fast enough to support advanced use cases for the business.
Build and operate batch and streaming pipelines from databases, APIs, event streams and semi-structured sources.
Implement change data capture and incremental load, including ordering, deletes, replay and slowly changing dimensions.
Build medallion datasets and dimensional models, and deliver them to Snowflake, Microsoft SQL Server and Databricks.
Apply data contracts, reconciliation and row-level quarantine before publication.
Own the GitHub workflow and CI/CD, including tests, review, environment promotion and deployment as code.
Investigate production data failures, and turn requirements from product, valuation and client-facing teams into operable pipelines.
Requirements
10+ years in data engineering on production systems.