Data Engineer
Core
Design, develop, and maintain data pipelines and systems to enable data analysis and machine learning for clients.
Role type
Data Engineer
Builds
Data pipelines, data models, schemas, and storage solutions
Domain
Data Engineering, Cloud Computing, Big Data
Deliverable
production ML models | data pipelines
Required skills
SQL, relational databases, cloud infrastructure (Azure/AWS), ETL pipelines, big data technologies (Spark/Hadoop/Kafka), Python frameworks (Flask/Tornado/Django), Docker
Preferred skills
NoSQL databases, reporting tools (Tableau/PowerBI/Qlik), GDPR compliance, Government Service Standard
Technologies
SQL, Azure, AWS, GCP, Luigi, Airflow, Argo, Apache Spark, Hadoop, Kafka, Docker, Flask, Tornado, Django, Tableau, PowerBI, Qlik
Responsibilities
Design and build data models and schemas; develop and maintain data ingestion and processing systems; implement data storage solutions; collaborate with stakeholders to address data issues; stay updated with data engineering trends.
Seniority
Mid-Senior, hands-on IC
