Team Lead Data Engineer
Core
Design, build, and optimize scalable data pipelines and analytics solutions using AWS and Databricks to support critical business initiatives.
Role type
Team Lead Data Engineer
Builds
Scalable data pipelines, ETL/ELT processes, and data architecture on AWS/Databricks
Domain
Cloud data engineering, Big Data
Deliverable
production ML models | product features | dashboards & analysis
Required skills
AWS Cloud (S3, Lambda, IAM, EC2), Databricks, Delta Lake, Unity Catalog, Apache Spark (PySpark), Python, SQL, CI/CD (GitHub Actions), Data governance tools (DataHub, Soda)
Preferred skills
Agile environment experience, problem-solving, proactive improvement identification
Technologies
AWS, Databricks, Apache Spark, Python, SQL, GitHub Actions, DataHub, Soda
Responsibilities
Design and maintain scalable data pipelines and transformation workflows; Build and optimize ETL/ELT processes for data ingestion and cleansing; Collaborate with analysts and scientists to translate requirements into technical solutions; Ensure data quality, reliability, and performance across systems; Implement best practices for coding, version control, and environment management; Monitor and troubleshoot pipelines for high availability; Contribute to architectural decisions and process improvements
Seniority
Senior, hands-on IC with leadership responsibilities
