Remote job
Senior Data Engineer – Capacity Planning & Forecasting
Job details
About this role
Role overview Build data products that help infrastructure teams plan capacity, forecast demand and manage cloud costs. The work combines large-scale data engineering with utilization analysis, bringing telemetry, workload demand, consumption and financial data together to inform infrastructure investment decisions.
Responsibilities - Build and maintain scalable ETL/ELT pipelines that ingest, transform and validate infrastructure and cloud consumption data from billing, telemetry, workload and financial systems. - Create analytical models and datasets for utilization, capacity tracking and cost reporting, with monitoring and reconciliation to maintain data quality. - Analyze historical and current use of compute, including CPU, GPU and TPU, as well as memory and storage; identify bottlenecks, underuse and capacity risks. - Develop demand forecasts, scenario analyses, dashboards and reports that show available and committed capacity, utilization and projected needs. - Investigate unusual utilization, cost changes and forecast deviations, and automate reporting and reconciliation where possible. - Work with infrastructure, finance, procurement, data science and machine learning teams to translate technical and financial data into recommendations.
Requirements - 10 or more years of relevant experience in data engineering, infrastructure analytics or capacity planning; equivalent practical experience may substitute for a degree. - Strong SQL, including complex queries and optimization, and Python for data processing, automation and analysis. - Experience with Pandas, PySpark or similar frameworks, plus scalable ETL/ELT pipelines, data warehousing, modeling and distributed processing. - Familiarity with tools such as Spark, Airflow, Kafka or Trino, as well as data quality, monitoring and governance practices. - Experience with a major cloud platform and understanding of billing, consumption data and infrastructure resource metrics. - Knowledge of capacity management, demand forecasting, historical trend analysis and scenario planning.
Nice to have Familiarity with GPU or TPU infrastructure and AI/ML workload consumption.
Benefits and work setup Remote role open to candidates in Brazil, Mexico, Chile or Colombia. The stated working coverage is Pacific hours, 8:00 a.m. to 5:00 p.m. PST.