Data Engineering Lead – Data Science and AI
Core
Design, build, and govern Azure-based data and technology architecture to enable Data Science, Machine Learning, Generative AI, and agentic AI initiatives, moving solutions from experimentation to production.
Role type
Senior hands-on Data Engineering Lead (Cloud Architecture & MLOps)
Builds
Secure, scalable production data pipelines, reusable data products, feature stores, and MLOps/LLMOps capabilities for AI workloads.
Domain
Cloud Data Engineering, AI/ML Infrastructure, Cross-Cloud Architecture (AWS to Azure)
Deliverable
production ML models
Required skills
Azure data/AI services, AWS data services, Python, SQL, data pipeline orchestration, MLOps/LLMOps, data governance, CI/CD, Infrastructure as Code
Preferred skills
Microsoft Fabric, Synapse Analytics, Databricks, feature stores, model registries, containerization, FinOps
Technologies
Azure Data Factory, Azure Functions, Azure Storage, Azure Machine Learning, AWS S3, Glue, Athena, Redshift, Lambda, SageMaker, Git, Docker, Kubernetes
Responsibilities
Define target data engineering architecture for AI workloads; Design integration patterns for moving data from AWS to Azure; Build reusable pipelines and standardized frameworks for ingestion and transformation; Partner with Data Scientists to establish production-grade MLOps and LLMOps; Provide foundations for GenAI and agentic AI solutions including ingestion and governance; Ensure alignment with governance, privacy, and security standards; Provide technical leadership to Data Engineers and translate roadmap into engineering plans; Establish monitoring, incident management, and cost-optimization practices.
Seniority
Senior, hands-on IC with leadership responsibilities
