Data Engineer
Core
Designing and maintaining data pipelines to ingest and transform IoT device and software product data into a central data lake for analytics, model training, and dashboard creation.
Role type
Data Engineer
Builds
Central data lake, computed tables, and data pipelines
Domain
IoT, Cloud Data Infrastructure
Deliverable
production ML models
Required skills
SparkSQL, Pyspark, SQL, Python, REST APIs, Data Orchestration (Airflow/Dagster/Prefect), Git, Software Engineering Fundamentals
Preferred skills
Time series data, Late-arriving data, Databricks, Delta Lakes, Public Cloud (AWS/GCP/Azure), First-party data modeling, ML outputs integration
Responsibilities
Build and maintain highly reliable computed tables incorporating unstructured data (video/audio) and sensor data; Access, manipulate, and integrate external datasets with internal data; Deliver high-quality data with strong uptime and reliability requirements; Collaborate with Data Science, AI/ML, and other Data Engineers to ensure high-quality data for diverse purposes
Seniority
Mid-Senior (4+ years experience)
