Senior Associate Data Engineering Azure/Databricks Arlington Hybrid
Core
Design, build, and optimize scalable batch and streaming data pipelines and Lakehouse solutions using Databricks and cloud platforms.
Role type
Senior hands-on Data Engineer
Builds
Scalable ingestion and ETL/ELT frameworks, Lakehouse architectures, and data governance controls
Domain
Cloud data engineering (Azure/Databricks)
Deliverable
production ML models
Required skills
PySpark, Spark SQL, Delta Lake, Unity Catalog, CI/CD, Infrastructure as Code, SQL, performance optimization
Preferred skills
Azure Data Factory, Airflow, Kafka, Terraform, AI/ML lifecycle support, data mesh
Technologies
Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog, Azure, AWS, GCP, Git, Kafka, Terraform
Responsibilities
Design and optimize batch and streaming pipelines; Develop scalable ingestion and ETL/ELT frameworks; Implement Lakehouse architectures; Establish automated data-quality controls; Optimize Databricks workloads; Implement governance standards for metadata and security; Support Git-based source control and CI/CD pipelines.
Seniority
Senior, hands-on IC