Data Engineer
Core
Design, implement, and optimize large-scale data ingest, extraction, cleansing, transformation, and loading systems in a Big Data environment to support AI/ML workflows and analysis.
Role type
Senior Data Engineer
Builds
Large-scale data pipelines, ETL processes, and data layers for analytics and AI/ML.
Domain
Big Data, Data Engineering, AI/ML support
Deliverable
production ML models | product features
Required skills
Python, Java, SQL, Apache Airflow, ETL development, data profiling, data deduplication, RESTful APIs, Linux environment development
Preferred skills
AI/ML system deployment, BI dashboard publishing
Technologies
Airflow, Big Data, ETL, Java, Linux, Python, SQL, Spring, Java EE
Responsibilities
Design and implement large-scale data ingest systems; optimize data lifecycle stages from planning to display; develop custom code for data ingestion and exploitation; create data profiling and matching logic; maintain Data Layer documentation; collaborate with data scientists on AI/ML pipelines.
Seniority
Senior, hands-on IC
