Big Data Developer (Spark/scala)
Core
Develop end-to-end ETL processes with Spark/Scala to transfer data from/to the data lake, including technical validations and business logic.
Role type
Backend Spark Developer
Builds
Data integration pipelines and ETL solutions
Domain
Big Data, Cloud Data Platforms
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Spark, Scala, software design patterns, TDD, big data processing, data integration, data management, agile/DevOps practices, SQL, Hive, Postgres
Preferred skills
Azure Databricks, Bash scripting, Control-M, SOA architecture, Docker, Kubernetes, AWS
Technologies
Spark, Scala, Hadoop, Hive, Azure Databricks, Postgres, SQL, Git, GitHub, Jenkins, Sonar, Nexus, Jira, Docker, Kubernetes, AWS, Control-M
Responsibilities
Develop end-to-end ETL processes with Spark/Scala; Perform technical validations and implement business logic; Certify delivery and integrate components; Design and perform relevant tests to ensure quality.
Seniority
Mid-level, hands-on IC
