Technical Lead, Spark
Seniority
Mid-Senior level
Domain
Engineering
Technologies
Apache Spark, Livy, Java, Scala, Python, SQL, Apache Parquet, Apache Iceberg
Responsibilities
Design new features for Cloudera's data engineering experience, Contribute to Apache Spark and Livy, Develop new features in Scala/Java/Python, Gain expertise in distributed data processing, Debug system level deployment issues, Work on improving internal infrastructure
Required skills
Bsc/Msc in related field or equivalent experience, 6+ years professional software development, Experience leading and delivering complex product enhancements, Strong understanding of at least one of Java, Scala, Python, Experience with systems design and development, Passionate about programming, clean coding habits, attention to detail, and focus on quality, Strong oral and written communication skills, Strong ability to research and solve problems independently, Open-minded, desire to learn new things and build great products, Experience with distributed systems
Preferred skills
Experience with SQL planners, Experience with using/developing Apache Spark, Livy or other related technologies, Experience with large-scale, distributed systems design and development with an understanding of scaling, performance, and scheduling, Solid experience with at least one cloud
