Overview
As Microsoft continues to push the boundaries of AI, we are seeking brilliant and passionate engineers to work with us on the most interesting and challenging problems of AI Infrastructure development. We are a team focusing on large language model optimization, and we are at the forefront of driving innovation in large-scale AI infrastructure. You will be instrumental in designing and implementing the high-performance, massively scalable infrastructure required to deploy frontier LLM models through innovative GPU kernel, compression, scheduling and parallelization optimizations, directly contributing to groundbreaking advancements in the field. This role is not just about maintaining systems; it's about architecting the future. If you are passionate about AI systems, low-level performance optimization, and solving hard cross-discipline engineering problems, we invite you to join us and help shape the future of AI at Microsoft. This is your opportunity to make a defining impact.

Responsibilities

- Keep up to date with and utilize the latest developments in LLM system optimization.

- Take the lead in designing innovative system optimization solutions for internal LLM workloads.

- Optimize LLM inference workloads through innovative kernel, algorithm, scheduling, and parallelization technologies.

- Continuously develop and maintain internal LLM inference infrastructure. - Discover new LLM system optimization needs and innovations.

Qualifications

- A bachelor’s degree or higher in computer science, engineering, or a related field, PhD is preferred

- Strong programming skills in Python and C/C++- 5+ years of experience in machine learning system development and optimization

Preferred Qualifications:

- 5+ years of experience in CUDA kernel development and optimization

- Experience in optimizing communication layer / kernels for deep learning systems

- Experience in machine learning model compression - Experience on different hardware such as both NVIDIA and AMD GPUs is a plus

- A growth mindset and a passion for learning new things

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.