Senior AI Data Engineer
Core
Design and implement modern data platforms, AI workflows, and Generative AI solutions (RAG, multi-agent) using Databricks and Azure.
Role type
Senior AI Data Engineer
Builds
Production-grade AI pipelines, Lakehouse architectures, and MLOps infrastructure
Domain
Enterprise Data & Generative AI
Deliverable
production ML models
Required skills
Python, SQL, PySpark, Delta Lake, Unity Catalog, Databricks Workflows, MLflow, Azure Service Bus, Azure Functions, Event-driven architecture, CI/CD, Git
Preferred skills
PyTorch, TensorFlow, Enterprise-scale AI platform design, LLMOps (prompt tracking, trace logging), Financial services domain knowledge
Technologies
Databricks, PySpark, Delta Lake, Unity Catalog, MLflow, Python, SQL, Apache Spark, Azure Databricks, Azure Service Bus, Azure Functions, Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Azure DevOps, Git, PyTorch, TensorFlow
Responsibilities
Design scalable ETL/ELT and AI pipelines; Develop and maintain Lakehouse architectures; Build and optimise Delta Live Tables; Develop and deploy production-grade Generative AI and agent-based solutions; Create and maintain RAG pipelines; Support MLOps activities; Implement data quality monitoring and governance; Develop event-driven integrations; Collaborate on feature stores and AI training datasets; Integrate Databricks with Azure services; Mentor engineering teams in AI technologies.
