Remote job
Data Engineer
Job details
About this role
Role overview
Build and operate data infrastructure that makes reliable, governed information available for analytics and AI/ML use. This role contributes to a lakehouse architecture and its data pipelines, balancing data quality, security, performance, cost, and delivery needs in collaboration with engineering and analytics partners.
Responsibilities
- Develop lakehouse components and layered data models designed for analytics and machine learning consumption. - Build and operate ELT pipelines, including orchestration, scheduling, monitoring, and issue resolution. - Apply data quality, testing, and observability practices to keep data workflows dependable. - Implement access controls, encryption, masking, and governance measures for sensitive data. - Improve workflow and storage performance while considering cost and speed. - Partner with engineers, analysts, and stakeholders; document designs, contribute to code reviews, and follow version control and CI/CD practices. - Prepare structured data and platform integrations that support AI/ML use cases.
Requirements
- At least three years of hands-on data engineering experience. - At least two years building and operating data pipelines in Databricks, including data modeling, governance, and CI/CD deployment patterns. - At least two years using analytical SQL and Python for pipeline development, transformations, and automation. - Strong communication and collaboration skills across engineering, analytics, and business teams. - Bachelor’s degree in computer science, information technology, or a related field.
Nice to have
- Experience with streaming or ingestion tools such as Lakeflow Connect, AUTO CDC, Fivetran, or Azure Data Factory. - Familiarity with Terraform, Azure DevOps, or AI coding assistants. - Experience applying generative AI tools to improve engineering productivity.
Benefits and work setup
- Remote position, with preference for candidates on the East Coast.