Data Engineer
Core
Design and build data warehouse layers (ODS, DWD, DWS, ADS), develop offline and near-real-time data pipelines, and optimize SQL and compute costs.
Role type
Senior Data Engineer (Data Warehouse & Batch/Stream Processing)
Builds
Scalable data pipelines, reusable data models, and data services for business reporting.
Domain
Enterprise Data Management / Big Data
Required skills
SQL (complex joins, window functions, optimization), Python, Spark (PySpark/Spark SQL), Java, Data Warehouse Modeling, ETL scripting, Job Scheduling, Data Quality Assurance
Preferred skills
AI development project experience
Technologies
Hive, Spark, Flink, HDFS, Object Storage, Azkaban, SQL, Python, Java
Responsibilities
Design data warehouse layers and define metric definitions; Develop offline and near-real-time data pipelines; Optimize SQL queries and job performance to reduce costs; Ensure data quality through validation and error handling; Collaborate on Java development projects; Monitor and document production data workflows.
