Data Engineer (Databricks)
Core
Design and develop scalable data processing pipelines, data platforms, and governance solutions for global enterprises using Databricks and Big Data technologies.
Role type
Senior Data Engineer (Databricks/Azure)
Builds
Universal data platforms, data transformation pipelines, and data ops infrastructure for aerospace, energy, and automotive clients.
Domain
Big Data, Cloud Data Engineering, AI/ML Infrastructure
Deliverable
production ML models
Required skills
Python (OOP), Databricks, Apache Spark/PySpark, Airflow/Dagster, DBT, Azure, Data Modeling, Data Governance, CI/CD, MLOps
Preferred skills
Power BI, LLM integration, Streaming data processing
Technologies
Databricks, Azure, Apache Spark, Airflow, Dagster, DBT, Delta Live Tables
Responsibilities
Design scalable data processing pipelines for streaming and batch processing; Contribute to CI/CD and MLOps processes; Develop applications to aggregate and analyze data; Collaborate with Data Science teams on ML projects; Develop data transformations using Databricks/DBT; Translate business requirements into technical solutions.
Seniority
Mid-Senior, hands-on IC