Senior AI Data Pipeline Engineer
Core
Design and operate high-throughput, petabyte-scale data pipelines to ingest and process global data for mission-critical AI workloads on large-scale GPU infrastructure.
Role type
Senior IC AI Data Pipeline Engineer
Builds
Global data pipelines, multi-region data infrastructure, and containerized data environments for AI/ML teams
Domain
AI/ML infrastructure, distributed systems, cloud-native data engineering
Deliverable
production ML models
Required skills
Python, Apache Spark, Databricks, Apache Airflow, Kubernetes, Apache Kafka, distributed messaging systems, cloud-native infrastructure design, system-level optimization
Preferred skills
Ray, Spark Streaming, Flink, Terraform, MLOps, global multi-region pipeline architecture
Technologies
Databricks, Spark, Kubernetes, Airflow, Kafka, Ray, Flink, Terraform
Responsibilities
Design and build high-performance, scalable data pipelines; Architect and implement multi-region data infrastructure; Develop flexible pipeline architectures for concurrent AI projects; Optimize large-scale data processing workloads; Maintain and evolve containerized data environments; Collaborate with AI researchers to streamline data flow into training pipelines
Seniority
Senior, hands-on IC
