Software Engineer - Data Infrastructure
Core
Building and scaling high-throughput, large-scale data processing pipelines to support multimodal AI systems and machine learning research workflows.
Role type
Senior IC data infrastructure engineer (ML research focus)
Builds
Scalable data infrastructure, large-scale data pipelines, and batch processing jobs for ML research
Domain
Artificial Intelligence / Machine Learning Research / Distributed Systems
Deliverable
production ML models
Required skills
Python, distributed computing frameworks (Ray, Spark, Beam), large-scale data pipeline design, data optimization, system architecture, monitoring and tuning
Preferred skills
Open source data infrastructure projects, robotics industry experience
Technologies
Ray, Spark, Beam
Responsibilities
Build and maintain scalable data infrastructure for high-throughput ML workflows; Develop and optimize large-scale data pipelines and batch processing jobs; Integrate open-source tools and improve data infrastructure through monitoring and tuning; Collaborate with ML researchers and product teams to ensure data systems meet evolving needs; Contribute to the architecture and implementation of reliable, high-performance data platforms; Participate in cross-functional projects to improve data reliability, scalability, and operational excellence
Seniority
Open to mid-level ICs through senior engineers and managers