Recorded Future · posted · checked
Senior Data Engineer, Credential Data Pipelines
AWS$129k – $194kOn-site · BostonSenior
Who can apply
On-site · Boston
Location: Boston, MA
With 1,000+ intelligence professionals serving over 1,900 clients worldwide, Recorded Future is the world’s most advanced, and largest, intelligence company!
Digital Risk Protection Engineering is hiring a full-time Engineer to own the credential data pipelines behind Digital Risk Protection: the ingestion path that turns raw stolen-credential data from infostealer logs, breach dumps, and combolists into the exposure records customers rely on. The role pairs that engineering ownership with the daily operational duties that keep the product serving customers, including domain approvals, instance provisioning, and data-correctness triage.
What You'll Do:
Own the domain approval queue, reviewing and approving customer domains within the requester's working day
Provision and backfill customer instances, including during live prospect evaluations
Triage data-correctness escalations from CSMs and account teams, tracing each case from the customer's view back to the raw source log
Build, release, and maintain the infostealer, dump, and combolist ingestion pipelines and the shared libraries the platform builds on
Deliver the HIBP parity backlog and the historical non-HIBP backlog in the reference index
Maintain routing parity on every published dump, applying the corporate-domain versus personal-domain rules
Administer API access for internal consumers such as research analysts
What You'll Bring:
Required
Production experience building and operating ETL or data-ingestion pipelines, including parsing semi-structured and malformed source data
Working knowledge of Elasticsearch and AWS-hosted data services
Show the full posting
Ability to debug end to end, from a customer-facing UI or API symptom to the originating log record