QA Data Engineer
Core
Ensuring quality, reliability, and accuracy of data pipelines powering analytics, reporting, and machine learning applications by designing validation frameworks and automating data quality tests.
Role type
QA Data Engineer
Builds
Automated test frameworks and validation services for large-scale distributed machine learning systems
Domain
Grocery retail / Data Engineering
Deliverable
production ML models
Required skills
Data pipeline validation, Test automation, Unit testing, Data modeling, CI/CD, Python, SQL, Apache Spark
Preferred skills
Peer review processes, Scalable data processing technologies
Technologies
Spark, Python, SQL
Responsibilities
Building test and validation cases for data engineering and data science code changes; Designing, monitoring, and maintaining QA reports, KPIs & quality trends; Creating test automation frameworks to validate data engineering pipelines; Scripting and automating tests for data pipelines used for machine learning models, dashboards, and reports; Developing scalable tools and services for conducting peer reviews and unit testing; Collaborating with data scientists and data engineers to test and deploy data pipelines; Identifying and addressing unexpected pipeline production issues; Bridging the gap between project and production teams for smooth roll-out changes.
Seniority
Mid-level, hands-on IC
