Principal Data Engineer
Core
Lead the evolution of next-generation data architecture and bridge Data Engineering with production-grade MLOps for AI Agentic workloads in a regulated Financial Services environment.
Role type
Principal Data Engineer (AI & Data Platform)
Builds
Enterprise Lakehouse infrastructure, production ML/AI pipelines, secure cloud environments, and real-time streaming systems.
Domain
Financial Services, Cloud Infrastructure, MLOps
Deliverable
production ML models | infrastructure
Required skills
PySpark, Python, SQL, Scala, Terraform, AWS services (S3, VPC, IAM, Kinesis/Kafka), Delta Lake, Unity Catalog, Delta Live Tables, PyTorch, TensorFlow, Hugging Face, CI/CD pipelines
Preferred skills
Experience in Financial Services or regulated environments, knowledge of data privacy and compliance frameworks
Technologies
Databricks, Mosaic AI, MLflow, Langchain, Vector Databases, AWS Bedrock, Spark, GPU clusters
Responsibilities
Architect and maintain enterprise Lakehouse infrastructure using Delta Lake and Medallion topology; Deploy and scale production-grade ML and AI Agentic workloads; Design secure cloud environments using Terraform and manage AWS configurations; Conduct advanced Spark optimization and tune GPU clusters for AI/ML workloads; Architect real-time streaming pipelines.
Seniority
Principal, hands-on IC
