Applied AI Engineer
Core
Build evaluation infrastructure and cost engineering systems for Velixo Intelligence, an MCP-based gateway enabling finance teams to query ERPs and perform governed writebacks via LLMs.
Role type
Senior Applied AI Engineer (LLM Evaluation & Cost Optimization)
Builds
Velixo Intelligence (MCP gateway for ERP querying and writebacks)
Domain
Finance technology, Large Language Models (LLMs), ERP integration
Deliverable
production ML models
Required skills
LLM API production experience, evaluation harness design, tracing/observability (Langfuse), eval frameworks (promptfoo), token accounting, context window management, caching strategies, Python, TypeScript
Preferred skills
MCP framework experience, ERP/accounting domain knowledge, usage-based pricing models, startup environment experience
Technologies
Langfuse, promptfoo, MCP, GIQL, Claude, ChatGPT, Copilot
Responsibilities
Build and maintain regression suites for query generation and writeback correctness; Instrument Gateway actions for tokens, latency, and cost; Optimize costs via prompt compression and model routing; Track model performance against actual workloads; Partner with Finance to model margin impact of pricing changes.
Seniority
Senior, hands-on IC