Data Engineer - Data Platforms (Databricks)
Core
Designing, developing, and maintaining scalable data pipelines and enterprise data platforms using Databricks and Apache Spark to power analytics and AI use cases.
Role type
Senior Data Engineer (Data Platforms)
Builds
Scalable batch and real-time data pipelines, Delta Lake architectures, and reusable data engineering frameworks.
Domain
Enterprise Data Engineering / Cloud Data Platforms
Deliverable
production ML models | infrastructure
Required skills
Python, SQL, Databricks, Apache Spark, ETL/ELT, Delta Lake, Cloud platforms, Data quality & governance, Infrastructure automation
Preferred skills
Data lineage, Query performance optimization, Technical documentation
Technologies
Databricks, Apache Spark, Delta Lake
Responsibilities
Design and maintain scalable data pipelines for batch and real-time processing; Build and optimize enterprise data platforms; Develop ETL/ELT processes to ingest and transform data; Implement Delta Lake architecture and maintain data lake/lakehouse solutions; Collaborate with Data Scientists and AI Engineers to support analytics initiatives; Optimize data storage, partitioning, and query performance; Implement data quality, validation, lineage, and governance processes; Ensure data security, privacy, and compliance; Monitor platform performance, troubleshoot production issues, and automate deployment.
Seniority
Senior, hands-on IC
