CareerPlan

Senior Data Engineer Consultant – Data Quality and Modern Cloud Platform

Durban, KZNFull-time2026-08-05 → 2026-11-19
Hybrid

Core

Design, build, and operationalize secure, scalable, governed cloud data platforms and pipelines for client reporting, analytics, migration, and responsible AI use cases.

Role type

Senior IC data engineer consultant (cloud platforms & data quality)

Builds

Cloud data platforms, ingestion/transformation/orchestration frameworks, data quality pipelines, metadata/catalogue solutions, and AI-ready data foundations.

Domain

Data Engineering, Cloud Platforms, Data Quality, Governance

Required skills

Advanced SQL, Python, PySpark, Cloud data platform architecture (Lake/Lakehouse/Warehouse), Data ingestion/transformation/orchestration, Data profiling/cleansing/reconciliation, Metadata/catalogue/lineage management, Cloud security (IAM, encryption, audit logging), CI/CD, Git, Technical leadership

Preferred skills

Microsoft Fabric/Databricks, Delta Lake, Data quality frameworks (Great Expectations, Soda, dbt), Infrastructure as Code (Terraform, Bicep), Power BI, Legacy SQL Server/SSIS, API/Streaming architectures, Generative AI/RAG

Technologies

Azure Data Factory/Fabric, Azure Data Lake Storage, Azure Databricks, SQL, Python, PySpark, Spark, Git, Terraform, Bicep, Power BI, Great Expectations, Soda, dbt

Responsibilities

Assess client systems and translate requirements into technical designs; Build reusable ingestion, transformation, and orchestration frameworks; Profile datasets and build validation, matching, and deduplication rules; Implement metadata, catalogue, and end-to-end lineage; Build observability, retry, and recovery mechanisms; Lead workstreams from discovery through handover and mentor engineers.

Seniority

Senior, hands-on IC with leadership