Software Engineer II, Data
Core
Design and build scalable, reliable, and observable data models and data pipelines across batch and streaming architectures.
Role type
Mid-level data engineer
Builds
End-to-end data pipelines processing structured, semi-structured, and unstructured data
Domain
Automotive financing / Data Engineering
Deliverable
production ML models
Required skills
Python, SQL, PySpark, data modeling, database management (SQL/NoSQL), batch/streaming pipeline design, data governance, Apache Spark, AI tools, ADLC, Databricks
Preferred skills
AWS data services (S3, Glue, Lambda, MSK), CI/CD pipelines, Airflow, dbt, Dagster
Technologies
Python, SQL, PySpark, Apache Spark, Databricks, AWS, Airflow, dbt, Dagster
Responsibilities
Design and build end-to-end data pipelines; Partner with stakeholders to understand data requirements; Write unit tests and validate data; Conduct impact analysis; Troubleshoot production issues; Document code and projects
Seniority
Mid-level, hands-on IC
