Data Engineer / Data Scientist
Core
Design and optimize scalable batch and streaming data pipelines and machine learning solutions on Azure to support analytics and AI initiatives.
Role type
Senior individual contributor data engineer / data scientist
Builds
Production data pipelines, ML models, and MLOps workflows
Domain
Cloud data engineering and machine learning on Microsoft Azure
Deliverable
production ML models (via careerplan.io/jobs/8fd66a39-1527-4cc6-9f3c-cf769ad89b84-data-engineer-data-scientist-at-jobgether)
Required skills
Python, PySpark, Azure Databricks, Delta Lake, MLOps, API development, Spark optimization, Kafka, CI/CD
Preferred skills
MLflow, Event Hubs, Azure Functions, Data quality monitoring
Technologies
Azure, Databricks, PySpark, Kafka, Delta Lake, MLflow, GitHub
Responsibilities
Develop scalable data processing solutions using Python, PySpark, and Azure Databricks; Build, maintain, and optimize batch and real-time streaming data pipelines; Debug, troubleshoot, and optimize Spark applications and Databricks jobs; Implement Delta Lake solutions to improve data reliability, versioning, and query performance; Develop APIs using Python or Scala for data and machine learning applications; Support machine learning initiatives, MLOps workflows, and model deployment activities.
Seniority
Senior, hands-on IC
