Senior Data Platform Engineer

Role overview Design, build and operate secure, scalable and reliable cloud-native data platform capabilities for clients in the public sector and utilities. This is a hands-on engineering role focused on platform services, distributed streaming, automated infrastructure and operational reliability across multiple client delivery projects. UK Government Security Check (SC) clearance is required due to the nature of the work.

Responsibilities

- Design, develop, test and deploy data platform services and components supporting batch and real-time workloads - Build and operate distributed data services using Kafka, Kinesis, Spark, Flink or equivalent technologies - Maintain cloud-native platform capabilities on AWS and/or Azure, including services such as S3, Glue, Lambda, RDS, Athena and EC2 or Azure equivalents - Support data ingestion, processing, storage and integration across systems, including relational and NoSQL databases - Develop and maintain data models, schemas and platform data structures, and support schema evolution, data quality and integrity - Build and maintain CI/CD pipelines and use Infrastructure as Code to create repeatable, maintainable environments - Operate Docker and Kubernetes-based environments; monitor services and investigate failures, performance issues and operational problems - Participate in incident investigation and root cause analysis, and contribute to platform performance, scalability, resilience and availability improvements - Work closely with data engineers, software engineers, architects and client stakeholders on technical design, code reviews and engineering standards

Requirements

- 5+ years of professional experience in Data Engineering, Data Platform Engineering, Software Engineering or a related discipline - Strong hands-on experience building cloud-native data platforms on AWS and/or Azure - Programming proficiency in Java and/or Python - Hands-on experience with real-time streaming and event-driven architectures, including Kafka, Kinesis or equivalent - Familiarity with distributed data processing such as Spark, Flink, Kafka Streams or equivalent - Solid experience with Infrastructure as Code, Docker and Kubernetes environments - Understanding of CI/CD, monitoring, observability, platform security and access controls

Nice to have

- Experience working in public sector, utilities or other complex, regulated environments

Benefits and work setup

- Competitive salary - 25 days holiday plus bank holidays - Company-paid medical insurance and life assurance, plus pension scheme - Annual training allowance, wellbeing allowance, virtual GP and Employee Assistance plan