Backend Engineer, Collection Platform

Role overview

Lead the design and operation of backend systems that gather data from websites, APIs, feeds, and external platforms. You’ll build collection services and pipelines that deliver dependable, timely data despite changing interfaces and unreliable sources, while setting technical direction and working with teams close to operational needs.

Responsibilities

- Set the architecture and near-term technical roadmap for data collection and ingestion. - Build and operate scrapers, external integrations, APIs, databases, and data pipelines used across teams. - Design for changing schemas, authentication, rate limits, partial failures, and varying source update schedules. - Implement reliable retries, backfills, deduplication, idempotency, and replay capabilities. - Track data completeness, accuracy, freshness, duplication, and provenance; improve system stability, performance, and scale. - Work with product, platform, and operational partners to prioritize sources and guide engineers through design reviews and mentorship.

Requirements

- At least 8 years of backend engineering experience, including hands-on work operating production collection or ingestion systems such as crawlers, integrations, ETL, or change data capture. - Experience leading delivery of substantial services, libraries, or infrastructure with impact beyond one team. - A record of improving critical systems and maintaining data quality across large, evolving datasets. - Ability to design for upstream failures and changing external interfaces, and to balance reliability, performance, scalability, maintainability, and cost. - Clear technical communication and effective collaboration across engineering and product groups. - U.S. Person status and eligibility to obtain a security clearance are required for access to restricted data systems.

Nice to have

- Experience with distributed task processing, workflow orchestration, or large-scale data pipelines.

Benefits and work setup

- Remote-friendly arrangement with access to coworking spaces. - Health, dental, and vision coverage; retirement plan matching; wellbeing stipends; and paid parental leave. - Flexible paid time off, federal holiday downtime, and a year-end company break. - Additional support includes military reserve duty salary top-up and child or pet care reimbursement during work travel.