Software Engineer
Core
Build scalable evaluation infrastructure and dashboards to enable rigorous testing and measurement of LLM applications.
Role type
Senior Software Engineer (AI Infrastructure & Evaluation)
Builds
Evaluation infrastructure, experiment management flows, analytics dashboards, and Python SDKs for LLM testing.
Domain
AI/ML infrastructure, LLM evaluation, developer tooling
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Modern frontend (React/Next.js), Backend (Django/FastAPI), API design, Data modeling, Testing, Performance optimization, Docker/Kubernetes, CI/CD, Async task processing, PostgreSQL
Preferred skills
LLM app development, Evaluation frameworks, Prompt engineering, Async workers (Celery), Privacy-sensitive deployments
Technologies
Python, Django, Django Ninja, Next.js, React, PostgreSQL, Docker, Kubernetes, Celery
Responsibilities
Develop scalable APIs and interfaces for structured experiments and automated evaluations; Design intuitive flows for user inputs and evaluation data management; Create dashboards for result summaries and compliance reports; Extend Python SDKs for modern AI dev workflows; Optimize async task processing and containerized deployments; Design UX for batch testing and collaborative reviews; Improve CI/CD and code health practices; Integrate cutting-edge AI tools to improve LLM app workflows.
Seniority
Senior, hands-on IC