Remote job
Senior Data Engineer - US
Job details
About this role
Role overview Take ownership of end-to-end data modernization on Azure Synapse or Fabric, translating client requirements into scalable pipelines and delivering governed, high-quality data products. The work supports downstream AI/ML applications and natural-language search experiences.
Responsibilities - Design, build, and optimize scalable ETL/ELT pipelines using advanced T-SQL and Python within Azure Synapse Dedicated SQL Pools and Azure Data Hub. - Develop and manage medallion architecture schemas (Bronze, Silver, Gold) tuned for high-performance SQL analytics and AI-ready workloads. - Integrate AI capabilities into data operations, including automated incident triage, cost-reduction recommendations, and proactive pipeline monitoring. - Implement data quality checks and monitoring frameworks that keep datasets trustworthy. - Administer data warehouses, data lakes, and relational plus NoSQL databases. - Operate workflow orchestration tools such as Airflow, Prefect, or Dagster for scheduling and monitoring pipelines. - Collaborate with AI/ML engineers to ground large language model applications through vector embeddings and semantic search metadata. - Maintain data security and compliance standards, and tune storage plus processing costs using FinOps automation. - Troubleshoot and resolve data issues independently.
Requirements - Expertise in T-SQL with deep experience using Azure Synapse Dedicated SQL Pools, Azure Data Hub, and relational plus NoSQL databases. - Strong Python proficiency for data manipulation, AI orchestration, and pipeline development, including PySpark and Pandas. - Hands-on experience across the Azure Stack, including Synapse Pipelines, ADLS Gen2, and Azure Data Hub for enterprise data warehouse operations. - Familiarity with AI integration patterns such as prompt engineering, vector databases, and semantic layer management for natural-language query tools. - Experience building and managing data pipelines and ETL/ELT processes at scale. - Solid grasp of data warehousing concepts, data modeling, and data quality principles. - Ability to work independently and take ownership of data infrastructure components.