Online Data Engineer
Core
Build infrastructure to capture customer data, develop algorithms to transform data into actionable information, and maintain database pipeline architectures.
Role type
Online Data Engineer
Builds
Data pipelines, data warehouses, data lakes, and data access APIs
Domain
Retail / Big Data Engineering
Deliverable
production ML models | infrastructure
Required skills
ETL pipeline development, Python, SQL, PySpark, Airflow, Google BigQuery, data orchestration, CI/CD automation, batch and streaming pipeline engineering
Preferred skills
Metadata management, data stitching from multiple sources, pipeline optimization for cost and runtime, product roadmap management
Technologies
Python, PySpark, Airflow, Google BigQuery, DataProc, React, JavaScript, Hadoop, MapReduce
Responsibilities
Design and develop algorithms to transform large datasets into business insights; Build, test, and maintain database pipeline architectures; Create data collection frameworks for structured and unstructured data; Develop APIs to retrieve data; Optimize pipeline runtime and storage consumption
Seniority
Mid-level, hands-on IC
