Applied AI Engineer
Core
Own the quality, safety, and performance of AI-powered consumer features including Personalized Credit Insights and an AI chat assistant.
Role type
Applied AI Engineer (LLM & Backend)
Builds
AI assistants, evaluation frameworks, guardrails, and monitoring tooling for credit insights.
Domain
Fintech / Credit Scoring / LLM Operations
Deliverable
production ML models
Required skills
Python, Java (Spring Boot), LLM application development, prompt engineering, LLM evaluation frameworks, SQL, AWS (Bedrock, S3, Lambda, DynamoDB, Redshift), CI/CD, A/B testing
Preferred skills
Guardrail tooling, LLM security testing, RAG evaluation, fairness and bias evaluation, regulated domain experience (FCRA, ECOA, UDAAP, CFPB), observability (Datadog), BI tools (Tableau)
Technologies
Python, Java, Spring Boot, Amazon Bedrock, AWS, Redshift, Datadog, Tableau, Claude Code, Cursor, GitHub Copilot
Responsibilities
Design and tune system prompts, tool descriptions, and evaluation prompts; build regression suites and evaluation frameworks; measure LLM evaluator reliability against human labels; design and monitor guardrails for PII redaction and compliance; lead model upgrades via A/B tests; build tools for analysts and QA engineers; partner on fairness and bias evaluation.
Seniority
Mid-Senior, hands-on IC