Senior Software Engineer, Big Data

Role overview A senior engineering position focused on building large-scale data systems that power matching intelligence for a high-traffic employment marketplace. The role combines data engineering, ML infrastructure, and distributed systems work to serve millions of users and tens of thousands of business customers. It is a fully remote, US-based opportunity suited to engineers who enjoy evolving complex, data-driven platforms.

Responsibilities

- Design and build data processing and exploration pipelines that support machine learning and business intelligence workloads. - Develop and maintain the ML infrastructure used to power marketplace intelligence at scale. - Deploy and operate cloud-based services supporting critical, high-volume projects. - Write, test, instrument, and deploy production code within a Kubernetes environment. - Contribute to the ongoing innovation and architectural evolution of a growing distributed system. - Collaborate on both streaming and batch data systems that process petabyte-scale datasets.

Requirements

- Five or more years of professional software development experience with a focus on Python and big data technologies. - Hands-on experience with at least one of Spark, Kafka, Hadoop, or Hive, plus broader familiarity with the big data ecosystem. - Strong computer science fundamentals, including object-oriented programming, data structures, and algorithms. - Practical experience with containerization tools such as Docker and Kubernetes. - Track record of writing evolvable, well-instrumented, and efficient code in distributed production systems.

Nice to have

- Eight or more years of professional software development experience centered on Python and big data. - BS, MS, or PhD in Computer Science, Mathematics, Physics, or a closely related technical field, or equivalent practical experience. - Experience with data integration tools such as Apache Flume and NiFi, or related streaming ingestion systems. - Familiarity with data storage technologies including Delta Lake, HBase, Cassandra, and MongoDB. - Exposure to modern data processing frameworks such as Apache Hudi, Apache Beam, Apache Flink, Google Cloud Dataflow, Amazon Kinesis Data Analytics, or Azure Databricks.

Benefits and work setup

- Competitive base compensation with a US salary range of $170,000 to $240,000, plus potential equity, bonus, or commission depending on the offer. - Comprehensive medical, financial, and retirement benefits, including an employer-matched 401(k) plan. - Flexible vacation and paid time off policy. - Fully remote work available for most US-based candidates, with a hybrid option also offered.