Sr Data Engineer
Core
Design, develop, and maintain scalable ETL/ELT pipelines and cloud-native data solutions for enterprise platforms. (via careerplan.io/jobs/45-004-7A-27C-sr-data-engineer)
Role type
Senior IC data engineer (cloud & real-time)
Builds
Scalable data pipelines, data models, and data services for enterprise analytics and reporting
Domain
Enterprise IT services, cloud data engineering
Required skills
SQL, Python, PySpark, Azure Cloud (ADF, Databricks, ADLS, Synapse, Event Hubs), Delta Lake, RESTful APIs, Kafka, Spark, Kubernetes, Apache Airflow
Preferred skills
Delta Live Tables, Auto Loader, Change Data Capture, DataOps, CI/CD, Infrastructure as Code
Technologies
Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Azure Synapse Analytics, Azure Event Hubs, Kafka, Spark, Hadoop, OpenShift, Kubernetes, Docker, Apache Airflow
Responsibilities
Design and develop scalable ETL/ELT pipelines for batch and real-time processing; Build and optimize Delta Tables and Lakehouse architectures; Develop and integrate RESTful APIs and data services; Implement real-time data ingestion frameworks using streaming technologies; Manage cloud-native data solutions on Azure; Develop and optimize Databricks Spark applications; Ensure data quality, governance, and compliance; Collaborate with data scientists and analysts to deliver scalable solutions; Troubleshoot and optimize pipeline performance; Support DataOps and CI/CD practices
Seniority
Senior, hands-on IC