Data Processing Quality Engineer
Core
Ensure quality of the data processing engine regarding result accuracy, performance fidelity, and robust execution at scale.
Role type
Data Processing Quality Engineer
Builds
Data processing engines and query systems
Domain
Data infrastructure / Cloud platforms
Deliverable
production ML models
Required skills
SQL, Python, Apache Spark, Gluten, Velox, DataFusion, AWS, GCP, MS Azure, Jira, Confluence, Git, Jenkins, Functional testing, Performance testing, Integration testing, System Level testing, Automation, Debugging, Root cause analysis, Test strategy
Preferred skills
BS EE/CS or equivalent, 5+ years in data processing quality or performance testing
Technologies
Apache Spark, Gluten, Velox, DataFusion, AWS, GCP, MS Azure, Jira, Confluence, Git, Jenkins
Responsibilities
Ensure result accuracy, performance fidelity, and robust execution at scale; Develop test strategies for complex systems; Debug and narrow down failures; Execute Functional, Performance, Integration, and System Level testing.
Seniority
Mid-Senior, hands-on IC