Data Engineer II
Core
Design, develop, and maintain complex data engineering projects and pipelines for the G2 data platform, ensuring scalable and efficient data processing.
Role type
Senior IC Data Engineer
Builds
Data pipelines and ETL/ELT processes for the G2 Data Platform
Domain
B2B Software / Data Engineering
Deliverable
production ML models | product features
Required skills
Data modeling, SQL, Python or Java, performance tuning, AWS data services, Snowflake, ELT tools (Glue, Airflow, DBT), distributed computing (Spark, Kafka)
Preferred skills
Docker, Kubernetes, CI/CD automation, TDD, data governance, reporting tools (Tableau, Looker, PowerBI)
Technologies
AWS (DynamoDB, RDS, EMR, Lambda, Glue, ECS), Snowflake, Apache Spark, Apache Kafka, Airflow, Airbyte, DBT
Responsibilities
Lead design and development of data pipelines for seamless data integration; Optimize data pipelines for cost effectiveness, scalability, and reliability; Service data requests from various users; Guide junior engineers with technical support and best practices; Own implementation of data quality and governance initiatives; Document data architecture, models, and workflows.
Seniority
Mid-Senior, hands-on IC
