Overview

Core AI is at the forefront of Microsoft’s mission to redefine how software is built and experienced. We are responsible for building the foundational platforms, services, programming models, and developer experiences that power the next generation of applications using Generative AI. Our work enables developers and enterprises to harness the full potential of AI to create intelligent, adaptive, and transformative software.

In this role, you will help in driving production‑critical model quality and API Functionality across AI serving pipelines powering Copilot and Azure OpenAI Foundry.

At Microsoft, our mission is to empower every person and every organization on the planet to achieve more. We live this mission every day through a culture that embraces a growth mindset, values diverse perspectives, and encourages continuous learning. We believe in creating an environment where individuals bring their best selves to work, collaborate openly, and build technology that makes a meaningful impact. Join us and help shape the future of the world.

This role is targeting an immediate start date.

Responsibilities:

• Designs, implement, and operate model quality evaluationfor advanced AI and agentic systems.
• Validates API correctness, reliability, and contract stability as model capabilities evolve, including tool use, agents, and multimodal workflows.
• Builds automated test harnesses, quality gates, and telemetry pipelines to detect regressions across model versions and deployments.
• Partners closely with infrastructure, safety, and downstream product teams to ensure evaluation signals directly inform release readiness.
• Drives clarity in ambiguous problem spaces by identifying gaps and proposing new evaluation methodologies as AI capabilities scale.

Qualifications:

Required Qualifications

• Bachelor's Degree in Computer Science, or related technical discipline with proven experience coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Other Requirements

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years

Preferred Qualifications:

• Dependable expertise in solving complex technical challenges in one or more domains such as distributed systems, AI/ML infrastructure, developer platforms, or cloud services.
• Commited to operate with high ownership and autonomy in fast‑moving, ambiguous environments.
• Proven experience with ML model evaluation, quality metrics, or testing frameworks for AI systems.
• Proven experience building CI/CD validation pipelines, large‑scale test automation, or data‑driven quality dashboards.

#aiplatform #coreai #modelquality #mlsystems #EiP #IC2SWE

Software Engineering IC2 - The typical base pay range for this role across the U.S. is USD $84,200 - $165,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $109,000 - $180,400 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.