Senior Data Engineer
Core
Architect, build, and scale core data platforms and high-throughput pipelines for a toll management and weigh station bypass system.
Role type
Senior IC data engineer (technical lead)
Builds
Production ETL/ELT pipelines, lakehouse architectures, and data systems on Azure/Databricks
Domain
Transportation / Logistics / Cloud Data Engineering
Deliverable
production ML models
Required skills
Distributed data pipelines, Azure, Apache Spark, Databricks, Delta Lake, SQL, Test-Driven Development (TDD), Extreme Programming (XP), AI-assisted development, CI/CD, Observability, Technical leadership
Preferred skills
MLOps, Feature Store management, Infrastructure as Code (Terraform/Bicep), Event-driven architecture, IoT/Telematics data
Technologies
Azure, Databricks, Delta Lake, PySpark, Spark SQL, Unity Catalog, SSIS, Terraform, Bicep
Responsibilities
Design and operate scalable ETL/ELT pipelines; Re-architect legacy SQL Server Integration Services (SSIS) workloads; Champion TDD and XP practices; Leverage AI tools for code generation and testing; Monitor mission-critical data systems for SLA compliance; Define team-wide engineering standards and mentor engineers
Seniority
Senior, hands-on IC with leadership responsibilities