Senior Data Engineer
Core
Design, build, and maintain scalable data pipelines and integrations within a Databricks-based data platform to deliver analytics-ready datasets.
Role type
Senior Data Engineer
Builds
Production-grade data pipelines, medallion architecture datasets, and enterprise integrations
Domain
Cloud Data Engineering / Cybersecurity Operations
Deliverable
production ML models
Required skills
Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, AWS (S3, IAM), Git, CI/CD
Preferred skills
Databricks Workflows, Auto Loader, Lakeflow Spark Declarative Pipelines, Unity Catalog, Amazon Bedrock, LLMs
Technologies
Databricks, Apache Spark, Delta Lake, AWS, Amazon Redshift, Amazon EC2, Sumo Logic
Responsibilities
Design and implement medallion architecture (Bronze, Silver, Gold); Build and maintain integrations with enterprise applications and REST APIs; Optimize Apache Spark and Delta Lake workloads; Implement Unity Catalog governance; Maintain existing data workflows on Amazon Redshift and EC2; Provide technical guidance to team members.
Seniority
Senior, hands-on IC