Senior Data Engineer (Databricks) - Contractor
Core
Design and deliver modern, scalable data platforms using Databricks and cloud-based lakehouse architectures to solve complex data challenges.
Role type
Senior Data Engineer (Databricks)
Builds
Scalable data pipelines, batch and streaming data ingestion, and reusable solution accelerators for enterprise data architecture.
Domain
Cloud data engineering, Lakehouse architecture, Distributed processing
Deliverable
production ML models
Required skills
Databricks, Delta Lake, Delta Live Tables, Unity Catalog, Apache Spark, PySpark, SQL, YAML, Apache Airflow, dbt, Terraform, Amazon EMR, Snowflake, Google BigQuery, Amazon Redshift, MLflow
Preferred skills
Databricks Data Engineer Associate/Professional certification, experience with AI/ML technologies, experience delivering multiple Databricks implementation projects
Technologies
Databricks, Delta Lake, Delta Live Tables, Unity Catalog, Apache Spark, PySpark, YAML, Apache Airflow, dbt, Terraform, Amazon EMR, Snowflake, Google BigQuery, Amazon Redshift, MLflow, Great Expectations, Collibra
Responsibilities
Design, develop, and implement data solutions using Databricks core platform capabilities; Build and optimize scalable data pipelines supporting batch, micro-batch, and streaming data ingestion; Implement configuration-driven data pipelines using YAML and Auto Loader; Develop robust data processing and transformation solutions using distributed computing frameworks; Design and maintain complex job orchestration and workflow patterns; Implement data quality, pipeline auditing, monitoring, and observability solutions; Collaborate with Solution Architects to translate architectural designs into technical components; Contribute to Databricks Center of Excellence initiatives by developing reusable solution accelerators.
Seniority
Senior, hands-on IC