D&A - Senior Data Engineer
Core
Design, develop, and maintain AWS and Databricks-based data platforms bridging Data Engineering, DBA, and Analytics Engineering.
Role type
Senior Data Engineer (Cloud & Lakehouse)
Builds
Scalable ETL/ELT pipelines, Medallion architecture data layers, and optimized data models for BI tools.
Domain
Cloud Data Engineering, Lakehouse Architecture
Deliverable
production ML models
Required skills
PySpark, SQL (Window Functions, CTEs), AWS ecosystem, Snowflake, Delta Lake, Data Modeling (Star Schema), CI/CD pipelines, Data Quality testing, Performance tuning
Preferred skills
Delta Live Tables (DLT), dbt, MLOps, Apache Airflow, Unity Catalog (via careerplan.io/jobs/71-008-F2-F6D-da-senior-data-engineer)
Technologies
AWS (S3, IAM, EC2), Databricks, Snowflake, Delta Lake, Parquet, Power BI, Tableau
Responsibilities
Develop and maintain complex ETL/ELT pipelines using Python/PySpark on Databricks and Snowflake; Implement and manage data layers following Medallion architecture using Delta Lake; Set up and manage Databricks SQL Warehouses and perform query optimization; Leverage AWS cloud services to manage data storage, compute resources, and secure data movement; Manage indexing, clustering, vacuuming, and performance tuning across Delta Lake and warehouse environments; Design data models (Star Schema / Snowflake) in the Gold layer for BI tools; Manage metadata, permissions, and lineage using Unity Catalog; Implement automated Data Quality tests in CI/CD data pipelines.
Seniority
Mid-Level, hands-on IC
