Senior Data Engineer
Core
Designing, building, and maintaining scalable end-to-end data pipelines and solutions on Databricks for manufacturing and energy sectors.
Role type
Senior Full Stack Data Engineer
Builds
Production data pipelines, data models, ETL/ELT processes, APIs, and data services
Domain
Manufacturing / Energy / Cloud Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python/PySpark, Scala, SQL, Apache Spark, Databricks, Delta Lake, Unity Catalog, ETL/ELT, data modeling, data warehousing, CI/CD, Git, DevOps, Kafka, Spark Streaming, JSON, Parquet, Avro, time-series data
Preferred skills
Databricks Genie, real-time streaming technologies, data governance and compliance best practices
Technologies
Databricks, Azure, Apache Spark, Kafka, Azure DevOps
Responsibilities
Develop and deploy end-to-end data pipelines on Databricks; Implement data ingestion, transformation, and processing workflows; Build and maintain APIs and data services; Monitor, troubleshoot, and optimize pipeline performance; Integrate data from various sources including APIs, databases, and IoT devices; Ensure adherence to data governance, security, and compliance policies.
Seniority
Senior, hands-on IC
