Data Engineer
Hybrid
Core
Design, develop, and maintain secure and scalable data pipelines on Azure Databricks, leveraging Python, PySpark, and AI-assisted validation tools.
Role type
Intermediate data engineer
Builds
Scalable data pipelines and data quality frameworks
Domain
Cloud data engineering (Azure Databricks)
Deliverable
production ML models
Required skills
Python, SQL, PySpark, Azure Databricks, AI agents/LLMs for code verification, Git workflows, CI/CD pipelines
Preferred skills
Clean code principles, systematic debugging, data quality concepts, DevOps practices
Responsibilities
Design and optimize data pipelines, integrate AI agents for automated code reviews, implement data quality frameworks, debug pipeline issues, optimize query performance, champion software engineering standards
Seniority
Intermediate, hands-on IC
