Senior Data Engineer Consultant – Data Quality and Modern Cloud Platform
Core
Design, build, and operationalize secure, scalable, governed cloud data platforms and pipelines for client reporting, analytics, migration, and responsible AI use cases.
Role type
Senior IC data engineer consultant (cloud platforms & data quality)
Builds
Cloud data platforms, ingestion/transformation/orchestration frameworks, data quality pipelines, metadata/catalogue solutions, and AI-ready data foundations.
Domain
Data Engineering, Cloud Platforms, Data Quality, Governance
Required skills
Advanced SQL, Python, PySpark, Cloud data platform architecture (Lake/Lakehouse/Warehouse), Data ingestion/transformation/orchestration, Data profiling/cleansing/reconciliation, Metadata/catalogue/lineage management, Cloud security (IAM, encryption, audit logging), CI/CD, Git, Technical leadership
Preferred skills
Microsoft Fabric/Databricks, Delta Lake, Data quality frameworks (Great Expectations, Soda, dbt), Infrastructure as Code (Terraform, Bicep), Power BI, Legacy SQL Server/SSIS, API/Streaming architectures, Generative AI/RAG
Technologies
Azure Data Factory/Fabric, Azure Data Lake Storage, Azure Databricks, SQL, Python, PySpark, Spark, Git, Terraform, Bicep, Power BI, Great Expectations, Soda, dbt
Responsibilities
Assess client systems and translate requirements into technical designs; Build reusable ingestion, transformation, and orchestration frameworks; Profile datasets and build validation, matching, and deduplication rules; Implement metadata, catalogue, and end-to-end lineage; Build observability, retry, and recovery mechanisms; Lead workstreams from discovery through handover and mentor engineers.
Seniority
Senior, hands-on IC with leadership