Senior Data Engineer II
Core
Architecting scalable data infrastructure and AI-augmented tooling to enable data-driven decisions for physical operations (IoT, logistics, manufacturing).
Role type
Senior Data Engineer II (AI-first platform focus)
Builds
End-to-end data pipelines, Spark-driven workflows, data APIs, and intelligent tooling (MCP servers, AI agents) for the Connected Operations Cloud.
Domain
IoT / Connected Operations / Cloud Infrastructure
Deliverable
production ML models | product features | infrastructure
Required skills
Spark/PySpark, SQL, Python, API design (FastAPI), Cloud platforms (AWS), Data quality & observability, Batch & streaming processing, Performance tuning
Preferred skills
AI agent development, MCP server architecture, Product mindset for data platforms
Technologies
Spark, PySpark, Databricks, AWS (Lambda, RDS), FastAPI, DataDog, Splunk, Python, SQL
Responsibilities
Design and build large-scale Spark workflows for batch and streaming data; Develop and maintain end-to-end data pipelines and ingestion workflows; Design, build, and manage data APIs; Optimize Spark job performance (partitioning, shuffle, caching); Facilitate data integration and transformation across applications and data lakes; Write data transformations to generate data products for business teams.
Seniority
Senior, hands-on IC
