Lead Software Engineer
Core
Lead the design, development, and maintenance of data engineering infrastructure, architecting scalable data pipelines for batch and real-time processing, and migrating on-prem applications to cloud-based PaaS.
Role type
Senior IC data engineering lead (cloud architecture & migration)
Builds
Scalable data pipelines, data warehouses, and cloud-based data solutions for multi-platform consumption
Domain
Data engineering, cloud infrastructure, big data
Required skills
Data architecture, data modeling (dimensional, relational, NoSQL), ETL/ELT design, cloud migration, big data technologies, data orchestration, database optimization, API design, microservices architecture
Preferred skills
Technical leadership, agile development, problem-solving, debugging
Technologies
Python, SQL, Java, Scala, Apache Spark, Hadoop, Kafka, AWS (S3, EC2, RDS, Lambda, DynamoDB, Glue), Azure (Data Factory, Synapse, Databricks), GCP (Big Query, Dataflow, Dataproc), Snowflake, Redshift, Airflow, dbt, Terraform, Docker, Kubernetes, Git, Jenkins, GitHub Actions
Responsibilities
Lead the design, development, and maintenance of data engineering infrastructure; Architect and implement scalable data pipelines for batch and real-time processing; Establish best practices, coding standards, and technical documentation; Mentor and provide technical guidance to junior data engineers; Collaborate with data scientists, analysts, and business stakeholders to understand data requirements; Optimize data models and database designs for performance and reliability; Deploy and manage cloud-based data solutions (AWS/Azure/GCP); Ensure data security, compliance, and governance standards are met
Seniority
Senior, hands-on IC with leadership responsibilities
