Sr Manager Software Dev, Advertising Full Funnel Agentic Intelligence
Core
Found and lead a standalone team to build a shared, rigorous evaluation system for AI agents and models in advertising, creating benchmarks, rubrics, and infrastructure to measure performance against real-world tasks.
Role type
Senior Manager, Software Development (Team Lead)
Builds
Evaluation infrastructure, advertising benchmarks, rubrics, and task environments for AI agents and models
Domain
Advertising technology + AI/ML evaluation
Deliverable
production ML models
Required skills
Engineering team leadership, evaluation system design, benchmark creation, rubric design, distributed systems, software development lifecycle, stakeholder management, expert-in-the-loop program management
Preferred skills
AI/ML technologies, advertising technology sales, AWS services, large-scale high-traffic applications
Technologies
AWS (EC2, Lambda, S3, DynamoDB, SQS), AI/ML frameworks
Responsibilities
Hire and lead a team of 10-12 engineers and PMs; define technical vision and roadmap for evaluations; partner with scientists to turn criteria into training signals; run expert-annotation programs; set and maintain evaluation standards; publish trusted results
Seniority
Senior, hands-on IC with management responsibilities