Principal Data Engineer
Core
Design, build, and run scalable production-grade data pipelines and data products (datasets, feature tables, vector databases) to support analytics, AI applications, and self-service data solutions for business partners and data scientists.
Role type
Principal Data Engineer (AI/ML focus)
Builds
Production data pipelines, reusable datasets, feature tables, retrieval infrastructure (vector databases), and LLM-powered automation agents.
Domain
Pharmaceutical & Healthcare / Data Engineering & AI
Deliverable
production ML models
Required skills
Azure Databricks (Delta tables, Unity Catalog, Workflows), PySpark, SQL, Azure Data Factory, CI/CD pipelines, data quality monitoring, data modeling (dimensional, lakehouse), Power BI
Preferred skills
Azure ML, MLflow, MLOps, dataset versioning, RAG pipelines, LLM-based automation, Databricks Genie
Technologies
Azure Databricks, PySpark, SQL, Azure Data Factory, Azure DevOps, MLflow, Power BI, Collibra, Unity Catalog, Vector Databases
Responsibilities
Design and maintain scalable data pipelines; create data products and feature tables for AI/analytics; implement data quality checks and lineage tracking; prepare training/evaluation datasets for data scientists; build LLM-powered automation tools; mentor junior engineers and set code standards.
Seniority
Principal, hands-on IC with mentorship
