Data Engineer
Core
Design, build, and maintain scalable production data infrastructure and pipelines to enable AI solutions for manufacturing.
Role type
Data Engineer
Builds
Production data pipelines, data warehouses/lakehouses, and reliable datasets for Data Scientists and ML systems
Domain
Manufacturing / AI / Cloud Data Engineering
Deliverable
production ML models
Required skills
SQL, Python, data modeling, data warehousing, pipeline orchestration (Airflow, Dagster, Spark, Prefect), version control (Git), cloud platforms (AWS, Azure, GCP), data quality monitoring, CI/CD, distributed processing (Kafka, Kinesis), infrastructure-as-code (Terraform)
Preferred skills
Data transformation frameworks, disaster recovery procedures, custom software components, system design at component level
Technologies
Airflow, Dagster, Apache Spark, Prefect, Git, Databricks, ClickHouse, Snowflake, DuckDB, AWS, Azure, GCP, Kafka, Kinesis, Terraform
Responsibilities
Design and maintain data warehouse/lakehouse structures; integrate data from varied source systems; implement data quality validation and monitoring; diagnose and resolve pipeline failures; build prototypes and proof-of-concept solutions; evaluate hardware, software, and cloud solutions; contribute to data and system architecture; install and update disaster recovery procedures
Seniority
Mid-level, hands-on IC