CareerPlan

Data Engineering Lead – Data Science and AI

Centurion , Gauteng, South AfricaFull-time2026-08-26 → 2026-11-19

Core

Design, build, and govern Azure-based data and technology architecture to enable Data Science, Machine Learning, Generative AI, and agentic AI initiatives, moving solutions from experimentation to production.

Role type

Senior hands-on Data Engineering Lead (Cloud Architecture & MLOps)

Builds

Secure, scalable production data pipelines, reusable data products, feature stores, and MLOps/LLMOps capabilities for AI workloads.

Domain

Cloud Data Engineering, AI/ML Infrastructure, Cross-Cloud Architecture (AWS to Azure)

Deliverable

production ML models

Required skills

Azure data/AI services, AWS data services, Python, SQL, data pipeline orchestration, MLOps/LLMOps, data governance, CI/CD, Infrastructure as Code

Preferred skills

Microsoft Fabric, Synapse Analytics, Databricks, feature stores, model registries, containerization, FinOps

Technologies

Azure Data Factory, Azure Functions, Azure Storage, Azure Machine Learning, AWS S3, Glue, Athena, Redshift, Lambda, SageMaker, Git, Docker, Kubernetes

Responsibilities

Define target data engineering architecture for AI workloads; Design integration patterns for moving data from AWS to Azure; Build reusable pipelines and standardized frameworks for ingestion and transformation; Partner with Data Scientists to establish production-grade MLOps and LLMOps; Provide foundations for GenAI and agentic AI solutions including ingestion and governance; Ensure alignment with governance, privacy, and security standards; Provide technical leadership to Data Engineers and translate roadmap into engineering plans; Establish monitoring, incident management, and cost-optimization practices.

Seniority

Senior, hands-on IC with leadership responsibilities