Remote job
Fundable - Senior Data Engineer
Job details
About this role
Role overview Own the architecture, implementation, and production operation of a data platform that consolidates information from APIs, webhooks, files, and web scraping. You’ll evolve existing serial workflows into concurrent, event-driven intake and orchestrated batch processing, delivering traceable, dependable records to downstream products.
Responsibilities - Build ingestion across multiple providers and formats, handling concurrency, rate limits, retries, and failures. - Orchestrate scheduled jobs and historical backfills with a workflow engine; implement event-driven processing with queues, dead-letter handling, and controlled replay. - Preserve original source payloads in cloud storage and transform them into raw, standardized, and curated data layers. - Design idempotent updates and safe recovery paths to support reprocessing without duplicates or corruption. - Create data contracts, automated validation, pipeline tests, and monitoring for freshness and failures. - Design and optimize PostgreSQL schemas, queries, and incremental loading for reliable downstream delivery. - Integrate provider enrichment and structured extraction from text using language models.
Requirements - Experience building production data platforms from initial architecture through deployment and ongoing operations. - Production experience with workflow orchestration tools such as Airflow, Dagster, Prefect, or Temporal, including retries and recovery. - Hands-on experience with dbt and Google Cloud Platform. - Experience with event-driven systems, messaging or task queues, asynchronous processing, and failure recovery. - Strong SQL and PostgreSQL skills, including modeling, query tuning, and incremental loads. - Strong Python skills, including asynchronous HTTP and concurrent processing. - Practical web scraping experience, including concurrency, rate limits, and fault handling. - Familiarity with data contracts, storage retention, and automated quality checks.
Nice to have - Experience in a startup or small engineering team with broad ownership and shifting priorities. - Entity resolution skills, such as matching company or person records and reconciling duplicates. - Experience with language-model extraction or company and people data enrichment. - Familiarity with BigQuery and Dataform.