Data Engineer
Core
Designing, implementing, and administering big data clusters and solutions for clients in banking, luxury, insurance, finance, and energy sectors.
Role type
Senior Data Engineer (Big Data)
Builds
Production data pipelines and big data solutions
Domain
Financial services and energy
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Java, Scala, Python, Apache Spark, Hadoop, data ingestion, data quality control, unit testing, load testing, performance optimization
Preferred skills
None stated
Technologies
Azure Cloud, On-premise, Spark, Hadoop
Responsibilities
Analyze and understand business needs; Participate in defining and designing architecture; Manage data preparation, ingestion, processing, and quality control; Develop Spark jobs and automate data lake feeds; Perform load and unit tests; Maintain big data solution performance
Seniority
Senior, 5+ years experience
