AI Lead / 12 Months day rate contract / Sydney
Core
Build evaluation frameworks, lead pre-release testing, and monitor production AI services for drift and risk within a public sector education AI Hub.
Role type
Senior AI Lead (Evaluation & Governance)
Builds
Evaluation frameworks, testing methodologies, and governance sign-off processes for generative AI and LLMs.
Domain
Public sector education / Generative AI / Responsible AI
Deliverable
production ML models
Required skills
Data science, Machine Learning, NLP, Generative AI, LLMs, Agentic AI frameworks, Evaluation framework design, Monitoring and regression testing, Data integrity, Compliance
Preferred skills
Postgraduate qualifications, Agile methodology, Stakeholder management
Technologies
LLMs, Agentic AI frameworks
Responsibilities
Build and improve AI evaluation frameworks and standards; Design and govern evaluation datasets, metrics, and testing approaches; Lead pre-release testing and assurance for deployment approval; Run ongoing monitoring and regression testing on production AI services; Partner with technical teams and governance stakeholders on data integrity and compliance.
