Senior Software Engineer

• AI Frameworks

Microsoft | Software Engineering IC4 | Hybrid

Compensation: Base pay range across U.S. is USD $119,800 - $234,700/year. San Francisco Bay Area and New York City metropolitan areas: USD $158,400 - $258,000/year.

About the Team

The Artificial Intelligence Frameworks team at Microsoft develops AI software that enables running AI models everywhere—from the world's fastest AI supercomputers to servers, desktops, mobile phones, IoT devices, and internet browsers. We collaborate with hardware teams and partners at the intersection of AI algorithmic innovation, purpose-built AI hardware, systems, and software.

We own inference performance of OpenAI and other state-of-the-art LLM models and work directly with OpenAI on models hosted on Azure OpenAI service, serving some of the largest workloads on the planet with trillions of inferences per day in major Microsoft products including Office, Windows, Bing, SQL Server, and Dynamics.

Responsibilities

• Drive innovation across the AI software stack with a strong focus on high-performance inference systems
• Lead benchmarking and optimization of state-of-the-art LLMs across GPUs and Microsoft's custom AI hardware
• Architect improvements to large-scale serving pipelines
• Diagnose complex performance bottlenecks across distributed systems
• Own critical metrics, design and implement robust optimizations
• Partner closely with research, hardware, and production teams to ship cutting-edge capabilities
• Software development in C++, Python, and other languages for specialized accelerators and host systems
• Software design, development, and optimization to execute AI workloads at cloud scale
• Co-design with hardware partner teams on HW/SW interfaces
• Perform design and code review of peer work

Required Qualifications

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python (or equivalent experience)
• Ability to meet Microsoft, customer, and/or government security screening requirements
• Microsoft Cloud Background Check required upon hire/transfer and every two years thereafter

Preferred Qualifications

• Master's Degree in Computer Science or related technical field AND 6+ years technical engineering experience; OR Bachelor's Degree AND 8+ years technical engineering experience; OR PhD Degree in Computer Science or related technical field AND 1+ years technical engineering experience
• 1+ year of experience with Large Language Models (LLMs) and large-scale execution on AI workloads
• 3+ years of technical design, problem-solving, and debugging skills

Additional Information

Applications are accepted on an ongoing basis with a minimum posting period of 5 days. Microsoft is an equal opportunity employer committed to building a diverse and inclusive workforce.