Sr. Data Engineer 4B
Core
Develop and optimize complex data pipelines, integration frameworks, and perform data reconciliation and transformation.
Role type
Senior Data Engineer
Builds
Data pipelines and integration frameworks
Domain
Cloud Data Engineering
Deliverable
production ML models
Required skills
Apache Spark, Databricks Platform, Python, CI/CD, Cloud Cost Optimization
Preferred skills
Unit/Integration Testing, Observability, Mentoring
Technologies
Azure, Databricks, Apache Spark
Responsibilities
Develop and optimize complex data pipelines and integration frameworks; Perform data reconciliation, advanced transformation development, and performance tuning; Design and execute unit, integration, and system testing activities; Implement CI/CD processes and engineering automation standards; Build monitoring, observability, and alerting capabilities; Optimize platform usage through cloud cost-aware engineering practices; Troubleshoot complex production and performance issues; Mentor junior engineers and contribute reusable engineering assets.