Data Engineer

Role overview Build and operate production software for a centralized data platform that brings together information from more than ten sources into a dataset of over 300 million company records. The role combines data engineering and software development: you’ll create pipelines, reusable Python components, and services that keep company and site data accurate, structured, and quick to query as usage grows.

Responsibilities - Develop and maintain Python ETL/ELT pipelines, scheduled workflows, and data processing jobs. - Model and transform data with SQL and dbt, and improve query performance on a cloud data platform. - Build reusable, well-typed Python libraries and APIs for downstream applications and other engineers. - Add unit, integration, and data quality checks; maintain automated testing and data contracts in CI. - Profile and improve pipeline reliability, runtime, memory use, and cost; replace ad hoc scripts with tested, scheduled code. - Deploy and operate services in cloud environments, with appropriate access controls and sensitive-data protections.

Requirements - Strong Python skills, including typing, error handling, modular design, packaging, and API fundamentals. - Production experience building and operating ETL/ELT pipelines; familiarity with Snowflake and dbt. - Experience with Apache Airflow or a similar code-based orchestrator, and Apache Spark, ideally on Databricks. - Solid SQL and data modeling skills, including relational databases such as PostgreSQL. - Testing experience with pytest and data tests, plus Docker, Kubernetes, CI/CD, Git, and at least one public cloud. - Ability to debug and profile performance, concurrency, and memory issues; daily experience with AI coding assistants and English at C1 level or higher.

Nice to have - Pydantic or similar schema validation; Python API frameworks such as FastAPI; async or multiprocessing patterns. - Azure AI Search, OpenSearch, LLMs, or agentic AI systems; a second language such as Go, Rust, Scala, or TypeScript.

Benefits and work setup Flexible hours and remote, office, or hybrid work are available. The role also offers structured onboarding, internal training and a training budget, and the possibility of moving between projects.