Remote job
Data Engineer w/ Databricks
Job details
About this role
Role overview
You’ll design and operate scalable data solutions for a customer-facing banking platform, helping make data reliable and accessible for analytics and business intelligence. The role centers on building production-grade pipelines with Databricks and AWS, improving data quality and performance, and collaborating with an international, cross-functional team.
Responsibilities
- Build and maintain large-scale batch and streaming ETL/ELT pipelines using Databricks and Apache Spark. - Structure lakehouse data using Bronze, Silver, and Gold layers, with performance-conscious design patterns. - Create automated data quality checks, monitoring, and testing frameworks. - Apply Delta Lake capabilities such as time travel, vacuum operations, and Z-ordering. - Develop deployment workflows for data pipelines using Databricks Asset Bundles or comparable tools. - Tune Spark jobs for performance and cost, and build integrations with platform APIs and external systems. - Design governed data architectures with cataloging, lineage, and data mesh principles.
Requirements
- At least six years of data engineering experience building and deploying production pipelines. - Strong skills in Python, PySpark, and SQL. - Hands-on experience with AWS services such as Glue, Lambda, Redshift, and S3. - Background in data governance and management tools, including Unity Catalog or comparable platforms. - Experience with data migrations and modernization of legacy systems. - Communicative English for work with an international team.
Nice to have
- Professional-level Databricks data engineering certification or equivalent. - Experience with Docker, Kubernetes, or similar container and orchestration tools. - Strong collaboration, communication, analytical, and problem-solving skills.