Overview

The Copilot Offline Evaluation Platform (COEP) is the foundational system that empowers Copilot product teams to run deterministic, reproducible, and representative evaluations at scale. As a PM on the COEP team, you will help define and deliver capabilities that streamline evaluation workflows across our platform components. You will partner closely with engineering, data science, and scenario teams to build the tools and abstractions that ensure Copilot quality keeps pace with rapid product innovation.

This role is ideal for a systems thinker who loves simplifying complexity, building for scale, and driving clarity across deeply technical problem spaces.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities:

Own Product Areas Within the Evaluation Lifecycle

You will drive strategy, execution, and customer outcomes for one or more pillars of COEP, such as:

• Evalset generation and configuration flows (tenant profiles, grounding data, assertions, ingestion graphs)
• Evaluation environment provisioning and reuse
• Integration with EGP, EMP, and C‑Store orchestrators
• Data/DP Pipeline integration for scenario‑specific query pattern generation
• Agentic UX and natural‑language-driven evaluation tooling enabling end-to-end flow without manual engineering intervention

Deeply Understand COEP Customer Personas

You will advocate for users across the platform:

• Scenario Owners, who define evaluation goals, author prompts/assertions, and drive evaluation demand
• Feature Developers, who execute eval jobs and review metrics without needing deep infra knowledge
• Quality teams and connector owners (e.g., GCO) who depend on precision, recipes, and repeatability for high-quality evaluations

Drive Clarity & Execution Across Engineering and Partner Teams

• Break down ambiguous, cross-system work into crisp requirements, aligned around customer workflows
• Partner with EGP, EMP, DP Pipeline, and C‑Store engineering teams to ship high‑quality platform capabilities
• Contribute to shared COEP reviews, PM team syncs, and cross-org syncs
• Produce specs, requirements, UX flows, and success metrics for your feature areas

Deliver Platform Reliability, Quality, and Usability

• Ensure features support determinism, reproducibility, and tenant coherence
• Define MVP criteria, success KPIs, backward compatibility expectations, and instrumentation needs
• Drive E2E dogfooding sessions, bug bashes, and quality validation efforts for major milestones (e.g., COEP Agent MVP)

Champion Platform Simplification and Scale:

• Identify opportunities to reduce friction, automate manual steps, and unify evaluation workflows
• Simplify onboarding and enable self-serve evaluation flows for hundreds of Copilot product teams
• Create documentation, recipes, templates, and customer-ready guidance to accelerate adoption

Qualifications:

Required Qualifications:

• Bachelor's Degree AND 3+ years experience in product/service/program management or software developmentOR equivalent experience.
  Preferred Qualifications:- Bachelor's Degree AND 8+ years experience in product/service/program management or software developmentOR equivalent experience.
• 2+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework).
• 4+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn).
• 4+ years experience disrupting a market for a product, feature, or experience (e.g., competitive disruption, taking the place of an established competing product).
• Experience with ML/AI systems, evaluation tools, or developer platforms Familiarity with Copilot products and workflows.
• Experience building natural‑language or agentic workflows.
• Solid systems thinking, execution skills, and technical aptitude.

Product Management IC3 - The typical base pay range for this role across the U.S. is USD $100,600 - $199,000 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $131,400 - $215,400 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.