Senior Data Engineer – Data Pipeline Development & Operational Support
Core
Design, build, and maintain ETL/ELT pipelines in test environments; analyze logs to identify failures and performance issues; conduct root cause analysis and support regression testing before production release.
Role type
Senior Data Engineer (ETL/ELT & O&M)
Builds
ETL/ELT pipelines, reusable scripts/workflows/notebooks, technical reports, operational runbooks
Domain
Data Engineering / Cloud Platforms
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
SQL, Python, cloud platform experience (AWS/Azure/GCP), data pipeline development, workflow orchestration, log analysis, root cause analysis, regression testing, technical documentation
Preferred skills
Databricks, Apache Spark, Delta Lake, CI/CD pipelines, DevOps practices, data onboarding/integration
Technologies
AWS, Azure, GCP, Databricks, Apache Spark, Delta Lake
Responsibilities
Design and build ETL/ELT pipelines using non-production datasets; analyze logs to identify failures and anomalies; conduct root cause analysis and prepare technical reports; perform sanity checks on test runs and schema changes; maintain operational runbooks and change logs; collaborate with client engineering teams to clarify specifications and share progress
Seniority
Senior, hands-on IC
