Senior Data Engineer
Core
Design and build scalable, cloud-native data platforms from greenfield to production, implementing near-real-time ingestion pipelines and laying the architectural foundation for AI-ready data infrastructure.
Role type
Senior IC data engineer (cloud-native & AI platforms)
Builds
Cloud-native data platforms, event-driven ingestion pipelines, self-service tooling, and microservices
Domain
Cloud data engineering, AI/ML infrastructure, event-driven systems
Deliverable
production ML models | infrastructure
Required skills
Python, SQL, Apache Spark/PySpark, Databricks or Snowflake, major cloud provider (Azure/AWS/GCP), stream processing (Kafka/Spark Structured Streaming), ETL/ELT patterns, data modelling, orchestration tools (Airflow/Azure Data Factory), Infrastructure as Code (Terraform), dbt, data governance frameworks
Preferred skills
RAG pipeline design, LLM integration patterns, MLflow, Feature Stores, Unity Catalog, Apache Atlas, Databricks, Azure Data Factory, RAG pipeline design, LLM integration patterns
Technologies
Apache Spark, PySpark, Databricks, Snowflake, Azure, AWS, GCP, Kafka, Spark Structured Streaming, Apache Airflow, Azure Data Factory, Terraform, dbt, Unity Catalog, Apache Atlas, MLflow, Feature Stores
Responsibilities
Design and build scalable, cloud-native data platforms; Implement near-real-time ingestion pipelines using event-driven patterns; Refactor and optimise existing Spark and PySpark scripts; Introduce best practices for code quality, testing, and CI/CD; Drive adoption of AI tooling and agentic workflows; Ensure data quality, observability, and reliability; Develop self-service tooling and microservices
Seniority
Senior, hands-on IC
