Data Engineer - AI Pipelines & DataOps
Core
Design and deliver scalable ingestion pipelines, streaming architectures, and APIs to feed AI and analytics systems with high-quality data from manufacturing edge devices and enterprise sources.
Role type
Senior IC Data Engineer (AI Pipelines & DataOps)
Builds
Scalable data ingestion pipelines, streaming architectures, and APIs for AI and analytics systems
Domain
Manufacturing / Industrial IoT / Data Engineering
Deliverable
production ML models
Required skills
Cloud-native data engineering (Azure/AWS), Streaming and batch pipeline development, REST/GraphQL API design, DataOps and CI/CD, SQL and NoSQL, Data quality and observability, IoT/OT data integration
Preferred skills
Manufacturing/OT domain experience, MLOps practices, Infrastructure-as-code (Terraform), Microsoft Fabric/OneLake
Technologies
Azure Data Factory, Azure Event Hubs, Databricks, AWS Glue, Kinesis, Lake Formation, Apache Spark, dbt, Apache Kafka, PostgreSQL, Cosmos DB, MongoDB, Great Expectations, OpenLineage, Terraform
Responsibilities
Design and implement scalable data ingestion pipelines for structured and unstructured data sources; Build and maintain batch and streaming data pipelines; Design and expose REST or GraphQL APIs for data assets and AI services; Implement CI/CD practices and DataOps principles; Ensure data quality, lineage, and observability across all pipelines; Support the integration of manufacturing and edge device data into the central data platform
Seniority
Senior, hands-on IC