Remote job
Snr. Data Engineer (Remote in South Africa)
Job details
About this role
Role overview Build and operate a data platform that gives engineering, analytics, product, and machine learning teams reliable access to high-quality data. The role focuses on near-real-time event pipelines and shared platform tooling, with opportunities to improve how teams create, monitor, and maintain data workflows.
Responsibilities - Own streaming data pipelines, from ingestion and modeling through monitoring and recovery. - Develop data models and platform capabilities that support reporting, products, services, and machine learning. - Improve self-service tools and automate pipeline creation, upstream change detection, alert triage, and code review workflows. - Work with stakeholders to clarify business questions, design appropriate solutions, and explain results and limitations to nontechnical audiences. - Identify platform needs, evaluate suitable tools, and make the case for improvements. - Review work, mentor teammates, and establish engineering practices the team can follow.
Requirements - Production experience with Spark streaming, including orchestration, checkpoints, monitoring, alerts, and recovery from failures. - Practical knowledge of modern table formats, including updates, schema changes, and ongoing maintenance. - Strong data modeling judgment and experience with event streams or change-data-capture ingestion, including ordering and duplicate handling. - Ability to write tested, typed Python and deliver it through review and continuous integration; working knowledge of SQL. - Cloud infrastructure experience and comfort reading and modifying infrastructure-as-code. - Ability to turn ambiguous stakeholder requests into designs, communicate tradeoffs, and lead technical improvements. - A bachelor’s degree or equivalent and at least 8 years of experience; or a master’s degree or equivalent and at least 3 years; or a doctorate or equivalent and at least 2 years.
Nice to have - Interest or experience in agent-based tools for engineering tasks such as scaffolding, change detection, triage, and review.
Benefits and work setup - Remote role based in South Africa; applicants are asked to submit a résumé in English. - Benefits include sales-target bonuses, referral bonuses, adoption assistance, tuition support, and certification reimbursement and completion bonuses. - An applicant assessment and background check may be part of the hiring process.