Software Development Engineer in Test
Core
Execute functional, performance, stability, and compatibility testing for core server-side modules, AI agents, and LLM products including recommendation, search, and retrieval systems.
Role type
Senior IC Software Development Engineer in Test (AI/LLM focus)
Builds
Automated testing pipelines, quality gates, and evaluation strategies for Context OS and AI model releases.
Domain
Artificial Intelligence, Large Language Models (LLM), Vector Retrieval, Server-side Architecture
Deliverable
production ML models
Required skills
API regression testing, chaos testing, automated evaluation of AI agents, vector-retrieval system testing, LLM/embedding/RAG testing, CI/CD integration, performance testing, microservices testing, message queue testing
Preferred skills
Experience with Elasticsearch, Faiss, Milvus, vLLM, Triton, Selenium, Playwright, Pytest, Kubernetes, Docker
Technologies
Elasticsearch, Faiss, Milvus, vLLM, Triton, Selenium, Playwright, Pytest, Kubernetes, Docker
Responsibilities
Lead API regression and data-consistency checks during architecture upgrades; Own evaluation strategies for recommendation, search, and retrieval systems; Create automated evaluations for AI agents covering multi-turn dialogue and tool calling; Develop custom automated testing tools and performance scripts; Integrate agent-testing pipelines into CI/CD.
Seniority
Senior, hands-on IC
