Senior Software Development Test Engineer - AI
Core
Build confidence in the quality, correctness, and reliability of machine learning models and AI agents powering the Automotive Retail Cloud (ARC) by designing evaluation frameworks for non-deterministic outputs, automating testing across the model lifecycle, and using AI/LLMs to accelerate quality engineering.
Role type
Senior AI Software Development Engineer in Test (AI-SDET)
Builds
Machine learning models, AI agents, and AI-powered features for the Automotive Retail Cloud (ARC)
Domain
Automotive retail technology, Machine Learning, Generative AI
Deliverable
production ML models
Required skills
Python, Java, test automation framework design, ML model evaluation, LLM testing, hallucination detection, CI/CD, API testing, root cause analysis
Preferred skills
MLflow, Weights & Biases, LangSmith, Ragas, DeepEval, AWS/GCP/Azure, Docker, Kubernetes, model observability, agentic system testing
Technologies
Python, Java, MLflow, Weights & Biases, LangSmith, Ragas, DeepEval, AWS, GCP, Azure, Docker, Kubernetes
Responsibilities
Design and implement automated testing frameworks for model inference, prompt pipelines, agentic workflows, and RAG systems; Create and curate evaluation datasets and ground-truth sets to measure response accuracy and consistency; Use AI/LLMs to generate test cases, synthetic data, and automation scaffolding; Build regression suites to detect quality drift when models or data change; Validate integration of ML models into ARC products and define quality metrics and monitoring to detect degradation
Seniority
Senior, hands-on IC
