CareerPlanGet AI match score →

Lead Data Engineer

Bangalore, Karnataka💼 Full-time🗓 2026-06-13 → 2026-07-31

Core

Designing and modernizing enterprise data engineering architecture, migrating legacy on-premises systems to cloud-native AWS patterns, and ensuring platform health and data quality.

Role type

Senior Lead Data Engineer (Enterprise ETL/ELT & Cloud Migration)

Builds

Enterprise data platforms, cloud-native data pipelines, and modernized ETL/ELT architectures.

Domain

Enterprise Data Engineering, Legacy Modernization, Cloud Migration (AWS)

Deliverable

production ML models | infrastructure

Required skills

Enterprise ETL/ELT architecture, Large-scale data platform operations, SQL (IBM DB2, SQL Server), IBM DataStage, Cloud migration strategy, Automation, Data quality monitoring, Release management, Stakeholder management, Knowledge management

Preferred skills

Apache Airflow, dbt, AWS Glue, Spark, ServiceNow, Capacity planning, Disaster recovery compliance

Technologies

AWS, IBM DB2, SQL Server, IBM DataStage, IBM Workload Scheduler, Oracle GoldenGate, Apache Airflow, dbt, AWS Glue, Spark, ServiceNow

Responsibilities

Define enterprise data engineering architecture and technology standards; Lead multi-year platform modernization roadmap from on-premises to AWS; Govern platform health including capacity planning and disaster recovery; Lead workload rationalization and pipeline consolidation; Drive adoption of modern data engineering capabilities; Own SLA adherence and lead root cause analysis for critical incidents; Lead monthly release cycles and change control governance; Maintain backlog visibility and executive reporting; Define data quality monitoring frameworks; Serve as primary relationship owner for senior stakeholders; Deliver operational and executive reporting on pipeline health; Develop multi-year data engineering roadmap; Lead phased AWS cloud migration strategy; Identify and implement automation opportunities; Lead knowledge management across the engineering team.

Seniority

Senior, hands-on IC with leadership responsibilities

Rewrite
## Responsibilities - Define the enterprise data engineering architecture and technology standards across DB2, SQL Server, IBM DataStage, IBM Workload Scheduler, Oracle GoldenGate, and AWS - Lead the multi-year platform modernization roadmap — phased migration from legacy on-premises patterns to cloud-native AWS data engineering patterns - Govern platform health including capacity planning, performance benchmarks, upgrade management, and disaster recovery compliance with BCP/DR standards - Lead workload rationalization — identifying pipelines, stored procedures, and jobs for consolidation, retirement, or re-architecture - Evaluate and drive adoption of modern data engineering capabilities (Apache Airflow, dbt, AWS Glue, Spark) aligned to Project Catalyst objectives - Own SLA adherence across all data engineering queues — incidents, service requests, small-ticket enhancements, and larger backlog-driven work - Lead root cause analysis (RCA) for critical data incidents and drive permanent fixes to prevent recurrence - Lead monthly release cycles including environment coordination, change control governance, and production readiness sign-off - Maintain full backlog visibility in ServiceNow — classification, aging, capacity tracking, and executive-level reporting - Define and oversee data quality monitoring frameworks, escalation procedures, and continuous improvement programs - Serve as the primary data engineering relationship owner for senior stakeholders across PHP, PDS/PMG, Quality, and System Services - Own CSAT measurement and improvement for the data engineering domain, proactively addressing data trust and availability concerns - Deliver weekly operational and monthly executive reporting on pipeline health, throughput, SLA performance, and platform KPIs - Develop and own the multi-year data engineering roadmap aligned to Project Catalyst's stabilization-to-modernization progression - Lead the phased AWS cloud migration strategy for remaining on-premises data engineering components, ensuring continuity and minimal disruption - Identify and implement automation opportunities to reduce manual pipeline interventions, dataset refreshes, and extract requests - Lead knowledge management across the engineering team — runbooks, architecture diagrams, onboarding playbooks, and continuity documentation ## Requirements - Minimum Degree Required: Bachelor’s Degree in Engineering, Statistics, Mathematics, Computer Science, Data Science, Economics, or a related quantitative field - 8+ years of data engineering experience with deep expertise in enterprise ETL/ELT architecture, pipeline design, and large-scale data platform operations - 3+ years in a formal lead, manager, or technical lead capacity overseeing a data engineering team - Expert-level SQL proficiency in IBM DB2 and SQL Server including complex schema design, query optimization, and stored procedure management - Expert-level IBM DataStage experience including architecture, parallel job design, performance tuning, and enterprise deployment ## Nice to Have ## Benefits
Sourced via adzuna · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.
Apply on Adzuna ↗