Data Platform Engineer
Core
Design, build, and maintain scalable data ingestion pipelines, storage architectures, and ML inference workflows for massive streams of sensor and operational data.
Role type
Senior IC data platform engineer (IoT/ML infrastructure)
Builds
End-to-end data workflows, ingestion pipelines, and model serving infrastructure for AI/IoT products
Domain
Cloud-native data engineering, IoT, and machine learning infrastructure
Deliverable
production ML models
Required skills
Python, Golang, SQL, AWS (S3, Lambda, Glue, Kinesis, Athena, RDS), Terraform, Databricks (PySpark/Spark SQL, MLflow, Delta Lake), real-time/batch data ingestion, fault-tolerant ETL/ELT, event-driven architectures (Kafka/Kinesis), infrastructure-as-code, debugging and optimization
Preferred skills
Time-series/IoT sensor data at industrial scale, gRPC/protobuf, feature stores, Kubernetes concepts, Databricks Machine Learning certification, startup experience
Technologies
AWS, Terraform, Databricks, Kafka, Kinesis, CloudWatch, SumoLogic, DataDog, Delta Lake, MLflow, Argo Workflows, gRPC, protobuf
Responsibilities
Design and maintain high-throughput data ingestion pipelines; Implement secure data lake and SQL storage architectures; Develop internal tools and frameworks for data ingestion and ML workflows; Build and maintain inference pipelines for model serving; Collaborate on data contracts, schemas, and retention policies; Monitor pipeline quality, latency, and failures; Optimize storage and query performance; Participate in architecture reviews for security and compliance
Seniority
Senior, hands-on IC