CS27 - Bac+5 - Évaluation et Monitoring des Agents IA en Production (H/F)
Core
Structuring and industrializing AI agent evaluation processes and building a common evaluation/monitoring platform for generative AI agents.
Role type
Junior IC AI evaluation engineer (intern)
Builds
Common evaluation framework and monitoring platform for AI agents
Domain
Automotive + Generative AI
Deliverable
production ML models
Required skills
Python, Generative AI (LLM, NLP, RAG, AI agents), Data aggregation and synthesis, Metric analysis, Framework implementation
Preferred skills
LLM evaluation frameworks (Ragas, Vertex AI), Observability and governance, Data pipelines, GCP
Responsibilities
Map and analyze existing evaluation processes, Design and implement a common evaluation platform, Collect and compare quality/robustness metrics, Define evaluation standards and recommendations, Maintain evaluation datasets, Conduct research on evaluation methodologies
