Sr Scala/Spark Developer - Irving
Core
Develop, test, and deploy high-performance data processing and analytics applications using Apache Spark and Scala on large-scale datasets.
Role type
Senior IC Scala/Spark Developer
Builds
Data pipelines and storage solutions within the Cloudera Hadoop ecosystem
Domain
Big Data / Data Engineering
Deliverable
production ML models | product features
Required skills
Apache Spark, Scala, Spark SQL, Cloudera Hadoop (HDFS, Hive, Impala, HBase, Kafka), ETL pipeline development, SQL, NoSQL databases, data warehousing concepts, dimensional modeling, Git, CI/CD tools
Preferred skills
Apache Kafka, Flume, Oozie, Nifi, PostgreSQL
Technologies
Apache Spark, Scala, Cloudera Hadoop, HDFS, Hive, Impala, HBase, Kafka, Sqoop, Jenkins, GitLab
Responsibilities
Develop, test, and deploy data processing applications; Optimize and tune Spark applications for performance; Work with Cloudera Hadoop ecosystem to build data pipelines; Collaborate with data scientists and analysts to deliver solutions; Design and implement high-performance analytics solutions; Ensure data integrity, accuracy, and security; Troubleshoot performance issues; Implement version control and CI/CD pipelines
Seniority
Senior, hands-on IC
