Big Data Developer
Core
Design and maintain scalable data pipelines and Data Lake architectures for processing large volumes of information.
Role type
Big Data Engineer
Builds
Scalable data pipelines and Data Lake solutions
Domain
Cloud data engineering
Deliverable
production ML models
Required skills
Apache Spark, Scala, SQL, Data Lake architecture, Data quality, Data lineage, Cloud platforms, Data ingestion, Data transformation, Data validation
Preferred skills
Agile collaboration, Incident management, Operational process improvement
Technologies
Spark, Scala, SQL
Responsibilities
Design and develop data pipelines on Data Lake architectures; Implement data ingestion, transformation, validation, and publication processes; Optimize performance and scalability of Spark-based processes; Analyze technical needs and propose improvement solutions; Collaborate with functional, technical, and support teams; Participate in production incident management; Contribute to continuous improvement of operational and monitoring processes