Quantexa Data Engineer
Core
Design, develop, and optimize big data solutions and search applications using Apache Spark, Scala, and Elasticsearch to support data analytics and machine learning initiatives.
Role type
Senior IC data engineer (big data & search)
Builds
Scalable data processing pipelines, Spark applications, and Elasticsearch search/indexing solutions
Domain
Finance / Compliance (AML) / Big Data
Deliverable
production ML models | product features
Required skills
Apache Spark (RDDs, DataFrames, SQL), Scala, Elasticsearch, Hadoop ecosystem, OpenShift Container Platform (OCP), Kubernetes, CI/CD pipelines, data governance, data lineage, metadata management, performance tuning, distributed computing
Preferred skills
Quantexa software proficiency, Quantexa certification, Graphana, Prometheus, Splunk, cloud platforms (AWS, Azure, GCP), Python, Java
Technologies
Apache Spark, Scala, Elasticsearch, Hadoop, HDFS, Hive, Pig, OpenShift, Kubernetes, Docker, Jenkins, Ansible, BitBucket, Git, AWS, Azure, GCP
Responsibilities
Design and implement Spark Scala applications and data processing pipelines; Integrate Elasticsearch with Spark for efficient indexing and querying; Optimize Spark jobs and Elasticsearch queries for performance; Develop containerized data engineering solutions on OCP; Implement data governance, lineage, and metadata management; Collaborate with DevOps teams on CI/CD and platform stability; Troubleshoot data processing issues and monitor infrastructure health.
Seniority
Senior, hands-on IC
