About the Role:

The Manual Evaluations Program Leader will own the end-to-end strategy, design and execution of human evaluations for Uber's GenAI-powered products, including conversational AI, voice AI, agent workflows and auto-evaluation systems. This role sits within the Global Digital Experience team, the operational arm of Uber's customer support tech organisation, and is a critical driver of quality, safety, and performance across Uber's next-generation AI solutions.

This leader will build and scale Uber's Manual Evaluation framework: defining methodologies, creating evaluation rubrics, ensuring annotation quality, and generating the insights that shape model tuning, product improvements, and release decisions. They will partner closely with Product, Engineering, Data Science and Product Ops to translate evaluation outcomes into clear technical and operational actions.

The role includes both strategic leadership and operational execution. The Program Leader will directly manage a team of three and indirectly oversee a distributed network of evaluators across global business sites. They will be responsible for setting the quality bar for evaluations, ensuring consistent delivery at scale, and driving continuous improvement of the evaluation pipeline.

The ideal candidate brings strong technical literacy in GenAI systems, exceptional program design and operational skills, and the ability to lead high-impact cross-functional initiatives. They are comfortable navigating ambiguity, building strong partnerships across Uber and influencing product direction through rigorous evaluation insights. This is a rare opportunity to play a leading role in one of Uber's most transformative technology programs and help shape the future of Uber's AI-driven experiences.

What the Candidate Will Do:

1. Own the end-to-end strategy, design, and execution of Manual Evaluations for Uber's GenAI-powered products (chatbots, voice AI, automated workflows, autoeval systems)2. Develop and continuously improve evaluation methodologies, including rubrics, taxonomies, annotation guidelines, quality standards and success metrics3. Partner with Product, Engineering, Data Science and Product Operations to ensure human evaluations directly inform model tuning, safety improvements, product design changes, and release decisions, as well as scaled operations teams to delivery on time, at short notice and to a high quality standard4. Lead evaluation projects across multiple AI products simultaneously**, ensuring timelines, quality and delivery expectations are met5. Package insights into clear, actionable narratives and present them to cross-functional leaders, influencing product and operational strategy6. Oversee a global manual evaluations operation, including direct management of a core team, indirect leadership of evaluators at multiple business sites and ongoing assessment of internal vs external resources to deliver the best evaluation outcomes7. Establish processes and tools that scale, including workflow optimization, evaluator training, QA systems and feedback loops.**8. Serve as Uber's subject-matter expert in human evaluation for GenAI, staying current with best practices in safety testing, multimodal evaluation and human-in-the-loop systems.

Basic Qualifications:

1. Bachelors degree in engineering or similar
2. 5+ years of experience inprogram management, product operations, quality operations, research operations, or technical program leadership**, ideally in a technology or AI-related environment.3. Experience with GenAI systems, LLM evaluation, model safety, failure pattern analysis, prompt evaluation, or AI product quality.4. Experiencedesigning or running structured evaluation or quality frameworks, such as human labeling, annotation, audit workflows or manual review processes.5. Familiarity withevaluation methodologies(rubric design, taxonomies, annotation guidelines, reliability scoring, inter-rater agreement, etc.).6. Proven track record ofmanaging teams, including coaching, performance management and resource planning.7. Strong project management abilities, with experiencerunning multiple complex programs simultaneously.**8. Proven experience managing outsourced teams to execute high-quality manual evaluation processes

Preferred Qualifications:

1. Demonstrated ability towork cross-functionally **with Product, Engineering, Data Science, and Operations teams.2. Knowledge ofautomated evaluation systems, LLM-as-judge frameworks, or hybrid human+machine evaluation pipelines.3. Background inservice design, conversational AI, voice UX, or agent workflows.4. Stronganalytical and problem-solving skills, with experience turning ambiguous data into clear insights.5. Excellentwritten and verbal communication skills, capable of translating technical evaluation outputs into business-relevant insights.**6. Experience inglobal operations, including scaling teams, training processes, and quality management across regions.

• For San Francisco, CA-based roles: The base salary range for this role is USD**$162,000 per year**
• USD**$180,000 per year**.

You will be eligible to participate in Uber's bonus program, and may be offered an equity award & other types of comp. All full-time employees are eligible to participate in a 401(k) plan. You will also be eligible for various benefits. More details can be found at the following link https://jobs.uber.com/en/benefits.

Uber's mission is to reimagine the way the world moves for the better. Here, bold ideas create real-world impact, challenges drive growth, and speed fuels progress. What moves us, moves the world - let's move it forward, together.

Uber is proud to be an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you have a disability or special need that requires accommodation, please let us know by completing this form.

Offices continue to be central to collaboration and Uber's cultural identity. Unless formally approved to work fully remotely, Uber expects employees to spend at least half of their work time in their assigned office. For certain roles, such as those based at green-light hubs, employees are expected to be in-office for 100% of their time. Please speak with your recruiter to better understand in-office expectations for this role.