Principal Data Engineer (AWS, Databricks, Ai, ML Flow, Data Architecture, Apache Airflow)
Core
Design and build scalable data foundations powering advanced analytics, machine learning, generative AI, and agentic AI solutions for Mastercard Foundry R&D.
Role type
Principal Data Engineer (hands-on IC with technical leadership)
Builds
Scalable batch, streaming, and event-driven data platforms; governed lakehouse architectures; reusable data products, services, and APIs.
Domain
Financial services / Payments / Cloud-native Data Engineering / AI Infrastructure
Deliverable
production ML models | product features | infrastructure
Required skills
Python, PySpark, Advanced SQL, Distributed data processing (Apache Spark), Cloud-native architecture (AWS/Azure), Data pipeline engineering (batch/streaming), Workflow orchestration (Apache Airflow), CI/CD, Data governance, Infrastructure as Code
Preferred skills
MLflow, Azure Machine Learning, Unity Catalog, Kafka, Docker, Kubernetes, Terraform, Regulated industry experience
Technologies
AWS, Databricks, Apache Airflow, PySpark, Python, SQL, Apache Spark, MLflow, Kafka, Docker, Kubernetes, Terraform
Responsibilities
Lead architecture and engineering of scalable data platforms; Build reliable data ingestion, transformation, and consumption pipelines; Create reusable data products and self-service capabilities; Enable full AI/ML lifecycle through feature engineering and deployment pipelines; Optimize data workloads for performance and cost; Define engineering standards for modeling, testing, and observability; Implement data governance and security controls; Provide hands-on technical leadership and mentorship.
Seniority
Principal, hands-on IC with technical leadership
