Senior Software Engineer, Data Acquisition

Role overview Build and operate the systems that collect, process, and deliver large-scale web data for data products. This senior engineering role spans crawling, backend services, distributed workloads, and data quality, with opportunities to improve existing datasets and support new offerings.

Responsibilities - Improve the architecture, reliability, throughput, and observability of data acquisition and processing systems. - Develop crawling tools and distributed systems for collecting and cataloging information from the web. - Build backend services for job orchestration, data pipelines, and asynchronous workloads. - Structure captured data and maintain consistency and quality across datasets. - Improve ingestion speed, scalability, and fault tolerance. - Partner with product and engineering teams on new data products and improvements to existing ones.

Requirements - At least seven years of professional experience building or operating backend or infrastructure systems at scale. - Strong programming experience in Python, Go, Rust, or a similar language, including asynchronous or concurrent programming. - Understanding of distributed systems, including parallelism, backpressure, message-driven design, and fault tolerance. - Familiarity with web application architecture, browser rendering, HTTP, DNS, proxies, and network debugging. - Experience with resilient ingestion, API integration, or ETL systems, plus Linux or Unix command-line tools. - Ability to work independently in a remote-first setting, communicate clearly in writing, document designs, and break complex work into milestones.

Nice to have - Experience with large-scale crawling, ingestion, indexing, or streaming systems. - Familiarity with message queues and task orchestration, distributed data platforms, SQL, data warehousing, cloud services, and modern storage patterns. - A degree in a quantitative field or experience in red teaming.

Benefits and work setup Remote-first work, stock, competitive pay, flexible paid time off, health coverage, and health, fitness, or office stipends are listed.