CareerPlan

Lead Data Scientist (Context Engineering)

Irvine, California; El Segundo, CaliforniaFull-time$154k2026-07-07 → 2026-11-20

Core

Own the quality, safety, and economics of AI agent retrieval and generation by building evaluation frameworks, observability tooling, security controls, and memory systems.

Role type

Staff/Lead Data Scientist (Context Engineering)

Builds

Intelligent, autonomous systems connecting Marketing, Sales, and Market Intelligence into a reasoning ecosystem.

Domain

Automotive + Applied AI / LLM Operations

Deliverable

production ML models

Required skills

LLM evaluation framework design, RAG pipeline instrumentation, vector database management, Python, SQL, context security (prompt injection defense), token economics optimization, episodic memory system design

Preferred skills

Master's degree in CS/IS/Data Engineering

Technologies

LangChain, LlamaIndex, FAISS, LangSmith, Arize, Phoenix

Responsibilities

Build and own evaluation frameworks for retrieval and generation quality including benchmarks and human-in-the-loop review; Instrument retrieval-to-answer paths for root-cause analysis and operational ownership of quality degradation; Design re-indexing and cache-invalidation strategies for data freshness; Defend generation layers against prompt injection and context leakage via isolation and sandboxing; Optimize context compression, prompt caching, and dynamic windowing for token economics; Design episodic and long-term memory systems for conversational agents.

Seniority

Staff/Lead, hands-on IC