Software Engineer, Test Automation
Core
Build test infrastructure and evaluation frameworks for an AI-agentic Security Operations Center (SOC) to ensure system reliability and performance.
Role type
Senior IC software engineer (test automation & AI quality)
Builds
Scalable testing ecosystems, AI-based evaluation agents, and benchmarking protocols for complex security systems.
Domain
Cybersecurity / AI / Test Automation
Deliverable
production ML models | product features
Required skills
Python, Go, large-scale testing ecosystem design, security compliance frameworks, modern performance testing methodologies, AI-driven automation
Preferred skills
Contract testing frameworks, Playwright, LLM APIs (Claude, Codex), prompt engineering, LangChain, LangSmith, LLM-as-judge frameworks, K6, Terra Virtual Machine, JMeter
Technologies
Python, Go, Playwright, LangChain, LangSmith, K6, Terra Virtual Machine, JMeter, Open Cybersecurity Schema Framework (OCSF), Caldera
Responsibilities
Develop PQ foundations and evaluation frameworks for AI agents and workflows; implement AI-based solutions to mitigate risk; collaborate on code and design reviews; execute proof-of-concepts and validate testing paradigms; maintain quality processes and alignment with goals; implement validation techniques like LLM-as-judge and behavioral regression suites; partner on contract testing and security compliance.
Seniority
Senior, hands-on IC