Principal Data Engineer
Core
Architect and lead the design of scalable Azure-based data lake solutions and data transformation pipelines for a high-volume freight management platform.
Role type
Principal Data Engineer (Azure/Databricks)
Builds
Azure data lake architectures, ETL pipelines, and data governance frameworks for freight exposure management.
Domain
Logistics / Freight / Cloud Data Engineering
Required skills
Azure Data Factory, Azure Data Lake Storage, Databricks, Delta Live Tables, Structured Streaming, Unity Catalog, SQL, Python, Scala, Spark, Hadoop, Kafka, MuleSoft integration, data governance, cost optimization, technical leadership, team mentoring.
Preferred skills
Azure Data Engineering certification, Databricks certification, Power BI, QlikSense, AI/ML, NLP, GenAI concepts, Agile/Kanban.
Technologies
Azure, Databricks, Delta Live Tables, Unity Catalog, Hadoop, Kafka, Spark, MuleSoft, Power BI, Qlik, Python, Scala, SQL
Responsibilities
Architect scalable Azure data lake solutions; build capability roadmaps for Databricks and Unity Catalog; lead development of data transformation pipelines; optimize Databricks resources and costs; enforce data governance policies; monitor pipeline performance; mentor team members; support detailed design of the Freight Platform and voyage management system; identify middleware integration solutions to replace Mulesoft.
Seniority
Principal, strategic leadership & hands-on architecture