Senior Data Engineer - Python
Core
Designing and maintaining large-scale ETL/ELT pipelines for automotive test bed data ingestion and processing.
Role type
Senior Data Engineer (Python/Automotive)
Builds
Production ETL pipelines processing large-scale test bed data for automotive clients
Domain
Automotive industry, Test Bed Systems
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Python (3.9-3.11), Object-Oriented Programming, Multiprocessing & Concurrency, Enterprise software for Automotive, Production ETL pipeline experience, Complex distributed system design, Azure, Oracle Database, SQL, MongoDB, PowerShell scripting, Windows service deployment, Grafana logging/monitoring, Network drive mounting (SMB/CIFS), NAS interactions, Sympathy for Data framework, .sydata/.h5 file processing
Technologies
Python, Azure, GitHub Actions, Poetry, Oracle, MongoDB, Grafana, PowerShell
Responsibilities
Build and maintain ETL/ELT pipelines for large-scale test bed data ingestion; Design complex distributed systems for automotive test data processing; Deploy Windows services and manage CI/CD pipelines; Process specialized file formats (.sydata, .h5); Implement logging and monitoring on file systems.
Seniority
Senior, hands-on IC