Lead Data Software Engineer
Core
Design and build resilient, scalable data systems and pipelines to support critical business decisions.
Role type
Lead Data Software Engineer
Builds
Reliable and well-governed data solutions at scale
Domain
Data Engineering
Deliverable
production ML models
Required skills
PySpark, Spark SQL, Databricks Lakeflow, Auto Loader, JDBC/CDC, Delta Lake, Unity Catalog, pytest, Git, Databricks Asset Bundles
Preferred skills
DataStage migration, Dynamics 365/Dataverse integration, Databricks certification
Technologies
PySpark, Spark SQL, Databricks, Delta Lake, Unity Catalog, DB2, SQL Server, Dynamics 365, Dataverse, DataStage
Responsibilities
Design, build, and maintain scalable data pipelines; Develop and manage ingestion pipelines; Write clean, maintainable, high-quality, and thoroughly tested code; Collaborate with cross-functional teams to translate business requirements into technical solutions; Monitor pipeline performance and maintain system reliability; Participate in code reviews and share technical knowledge; Investigate and resolve data quality issues; Contribute to the continuous improvement of data architecture and engineering standards; Create and maintain technical documentation; Provide technical leadership and support the development team.
Seniority
Lead, hands-on IC with team leadership