Data Engineer- Silver Conversions and Scheduling

Role overview Build and operate Silver-layer data conversion pipelines in a medallion Lakehouse, turning raw Bronze data into cleansed, consistent datasets. This contract role focuses on dependable scheduling, data integrity and performance using Microsoft Fabric and Azure data services. The stated term is December 2026 to March 2027, with a good likelihood of extension.

Responsibilities - Develop Silver-layer transformations with Microsoft Fabric, Azure Data Factory, Spark and Delta Lake. - Configure pipeline schedules and dependencies, including retry and failure-handling behavior. - Use reusable PySpark libraries and metadata-driven configuration to support efficient delivery. - Apply consistent logging, monitoring, error handling and unit-testing practices. - Maintain data integrity through reconciliation, checkpoints, replay and recovery processes. - Tune Spark workloads for runtime and cost efficiency.

Requirements - At least five years of experience with Microsoft Fabric, including Data Factory and Lakehouse, alongside Azure Data Factory and Azure Storage. - At least five years of experience using PySpark, Spark SQL and Delta Lake for data transformations. - Experience building and scheduling pipelines with Fabric Data Factory or Azure Data Factory. - Exposure to Microsoft Purview for data cataloguing and lineage. - Experience with Azure DevOps, Git and CI/CD pipelines. - Active Reliability status and eligibility to obtain Secret clearance.

Benefits and work setup Work alongside experienced technology professionals on client projects in a collaborative environment that supports knowledge-sharing, mentorship and inclusive hiring. Accommodation is available during the recruitment process.