Lead Data Engineer

Role overview Lead the technical direction of a client-facing data engineering engagement, owning architecture, standards, and hands-on delivery for a modern cloud data platform. The position combines people leadership with deep technical work, designing and operating large-scale batch pipelines while partnering with stakeholders to translate business needs into a coherent roadmap.

Responsibilities - Define and own the technical direction for a data engagement, setting standards for pipeline architecture, data modelling, testing, and operational readiness - Lead and technically direct a team of data engineers, allocating work, reviewing designs and code, and unblocking complex issues - Design and evolve data platform architecture on Microsoft Fabric, including data pipelines, OneLake, and the wider Azure ecosystem - Set and enforce CI/CD, testing, and operational practices across data engineering workflows using Azure DevOps - Build relationships with key client stakeholders, translating business and data requirements into a coherent technical roadmap - Troubleshoot the most complex issues in live data pipelines and lead incident response when production problems arise

Requirements - Extensive hands-on experience designing production data platforms on a modern cloud data platform such as Microsoft Fabric, Databricks, Snowflake, or Synapse - Deep SQL expertise combined with strong data modelling and architecture skills, including Kimball, data vault, medallion/lakehouse, or domain modelling patterns - Strong proficiency in Python and PySpark for batch and ELT data processing at scale - Proven experience leading or technically directing a team of data engineers and setting standards for pipeline design, testing, and delivery - Ownership of CI/CD and operational practices for data engineering workflows, ideally using Azure DevOps - Experience working directly with client stakeholders, building trust and constructively challenging requirements - Familiarity with the broader Azure ecosystem, including OneLake and Azure ML for model industrialisation

Nice to have - Experience industrialising machine learning models alongside data platform work - Comfort operating across hybrid and remote working patterns with occasional travel to client sites or office hubs

Benefits and work setup - Flexible working arrangements, including hybrid and remote options, with office hubs in Edinburgh, Leeds, Manchester, London, and Bulgaria - Trust-based culture focused on impact rather than activity, with investment in professional and personal growth - Supportive, collaborative environment described by employees as friendly and curious