Software Engineer, Big Data
Core
Build data processing pipelines, ML infrastructure, and scalable cloud applications to connect millions of jobseekers and employers.
Role type
Senior data engineer (big data & ML infrastructure)
Builds
Data processing pipelines, ML infrastructure, cloud-based services
Domain
Job marketplace, big data, machine learning
Deliverable
production ML models
Required skills
Python, Spark, Kafka, Hadoop, Hive, Docker, Kubernetes, data structures, algorithms
Preferred skills
Data integration tools (Flume, NiFi), NoSQL storage (HBase, Cassandra, MongoDB), Stream processing (Apache Beam, Flink, Dataflow, Kinesis, Databricks)
Technologies
Kubernetes, Docker, Spark, Kafka, Hadoop, Hive, Delta Lake, HBase, Cassandra, MongoDB, Apache Hudi, Apache Beam, Apache Flink, Google Cloud Dataflow, Amazon Kinesis Data Analytics, Azure Databricks
Responsibilities
Build data processing and exploration pipelines, deploy cloud-based technologies, write and deploy code to Kubernetes environment
Seniority
Senior, hands-on IC