Data Engineer

Role overview Build and operate large scale data systems that combine information from multiple sources into reliable, usable datasets. The work spans ingestion, transformation, entity matching, and production quality controls, with room to shape solutions to open ended data engineering challenges. The role calls for several years of engineering experience and combines independent ownership with close collaboration.

Responsibilities - Develop scalable pipelines to ingest, clean, transform, and load growing volumes of data from varied sources. - Help build entity resolution systems that reconcile records into consistent datasets. - Create and improve CI/CD workflows and anomaly detection to support dependable production data. - Design data processing and storage patterns, including incremental updates, partitioning, backfills, and rebuilds. - Document pipeline, dataflow, and schema designs; communicate scope, progress, and blockers to collaborators.

Requirements - Around 3–5 or more years of industry experience, with examples of sound technical problem solving and implementation. - Strong software engineering fundamentals and practical experience with Python, SQL, and Apache Spark. - Experience building scalable data cleaning and transformation systems, plus orchestration tools such as Airflow, dbt, or Dagster. - Familiarity with Databricks and lakehouse patterns, cloud platforms such as AWS, GCP, or Azure, and data warehouses. - Understanding of formats such as Parquet, ORC, Avro, or Delta Lake. - Able to work effectively in a remote setting, take ownership, collaborate, write clearly, and manage work and blockers proactively.

Nice to have - A degree in a quantitative field, or experience with entity resolution, record linkage, data acquisition, or integration. - Familiarity with Python data tools such as NumPy or pandas, streaming systems such as Kafka, and systematic data quality measurement.

Benefits and work setup - Remote work with flexibility in where and how work is done. - Equity, health, dental, and vision coverage, paid time off, and health, fitness, and office stipends. - Listed compensation: $180,000–$220,000.