Senior/Lead Data Engineer - Data Quality
Core
Design and build trusted, scalable data platforms and pipelines to support analytics, operational reporting, and AI/ML initiatives across the organization.
Role type
Senior/Lead Data Engineer (Data Quality)
Builds
Scalable data pipelines, ETL/ELT processes, and governed data assets for AI and analytics
Domain
Shipping industry technology / Data Engineering
Deliverable
production ML models | product features
Required skills
Data pipeline design, Data quality frameworks, SQL, Python, Spark, Scala, Pipeline orchestration, Cloud platforms (AWS/Azure/GCP), Data governance, Metadata management, CI/CD for data
Preferred skills
DataOps/MLOps practices, Streaming data architectures (Kafka/Confluent), Modern orchestration engines (Airflow/Dagster), Vector databases (Pinecone/Qdrant/Weaviate)
Technologies
Python, Spark, Scala, SQL, AWS, Azure, Google Cloud Platform, Apache Airflow, Dagster, GitHub Actions, GitLab CI, Jenkins, Apache Kafka, Confluent, Amazon Kinesis, Pinecone, Qdrant, Weaviate
Responsibilities
Design and optimize scalable data pipelines and ETL/ELT processes; Define and implement enterprise-wide data quality principles and standards; Ensure data reliability and accuracy across platforms; Design strategies for FAIR data principles; Build and maintain scalable datasets supporting AI/ML; Collaborate with AI/Data Science teams on production workloads; Catalog data assets and maintain metadata; Define data quality requirements for dashboards and models; Drive best practices in data architecture, governance, and monitoring; Improve observability and operational excellence across the data platform.
Seniority
Senior, hands-on IC with leadership responsibilities
