Product Manager, Evals & Improvement

The Agent Transformation Accelerator (ATA) Product team is building Meta’s future of work: Metamate, an internal full-stack platform that lets people direct, review, and help agents improve their work across the entire company. We’re building the platform components, memory systems, improvement loops, and shared product experiences that make agentic AI genuinely useful for product development. This role owns how we measure and raise the quality of agentic systems (multi-step trajectories, tool use, and partial credit).
This is an early-stage, high-leverage role that contributes directly to Metamate's topline revenue, growth, and velocity. You'll shape the approach from the ground up in a space where the right metrics don't yet exist, defining what "good" means for agents and owning the eval verdict that gates model upgrades, harness changes, and major launches. Expect to be constantly learning, working shoulder-to-shoulder with engineering, data science, and ML, including our partners across ATA and Meta Superintelligence Labs (MSL), to turn signal from real usage into a continuous improvement loop that makes the product measurably better every week.
Product Manager, Evals & Improvement Responsibilities
Own the end-to-end feedback and improvement system: collection, analysis, prioritization, and resolution.
Define how we measure agent quality across dimensions (accuracy, helpfulness, reliability, safety).
Build product mechanisms that capture implicit and explicit user signal at scale.
Partner with ML and data science to turn feedback into model improvements, eval frameworks, and training data.
Drive the evals and measurement strategy: what "good" looks like, how we detect regressions, and how we track progress.
Create transparency and accountability for quality across the ATA Product team.

Minimum Qualifications:

Strong quantitative skills and experience defining complex metrics
Comfort with ambiguous problem spaces where the right metrics do not yet exist
5+ years of relevant industry experience with at least 2 years in Product Management
Bachelor's degree (or relevant degree equivalent): STEM subject ideal but not essential (Computer Science, Engineering, Information Systems, Analytics, Mathematics, Physics, Applied Sciences)
Experience partnering closely with data science and ML engineering teams
Track record of driving product improvements through data and experimentation
Direct experience building evals for AI products end to end: task set construction, rubric design, instrumentation, and analysis
Experience running eval-driven development cycles, with a clear view of where evals are informative and where they mislead
Willingness to get into the weeds of eval data and rubrics, with high standards for eval rigor and an obsession with quality
Deep understanding of LLM capabilities and failure modes

Preferred Qualifications:

Experience building annotation, labeling, or crowd-sourcing systems
Experience with reinforcement learning from human feedback (RLHF) or similar human-in-the-loop systems
Background in evals, trust and safety measurement, or ML quality infrastructure
Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies

About Meta

Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and Whats App further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.
For those who live in or expect to work from California if hired for this position, please click here for additional information.
United States of America: $146,000/year to $204,000/year + bonus + equity + benefits
Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.

Equal Employment Opportunity:

Meta is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or other applicable legally protected characteristics. You may view our Equal Employment Opportunity notice here.
Meta is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, fill out the Accommodations request form.
Apply for this job
Take the first step toward a rewarding career at Meta.
Recruiters can view your conversations with AI. Using AI is optional and does not impact the outcome of your application process.Learn more about AI usage and settings.
Sign up
to ask follow-up questions