Principal Software Engineer - AI Frameworks

The Artificial Intelligence (AI) Frameworks team at Microsoft develops AI software that enables running AI models everywhere, from world’s fastest AI supercomputers, to servers, desktops, mobile phones, internet of things (IoT) devices and internet browsers.

• Identify and drive improvements to end-to-end inference performance of OpenAI and other state of the art LLMs
• Measure, benchmark performance on Nvidia/AMD GPU's and first party Microsoft silicon
• Optimize and monitor performance of LLMs and build software tooling to enable insights into performance opportunities ranging from the model level to the systems and silicon level
• Enable fast time to market of LLMs/models and their deployments at scale by building software tools that afford velocity in porting models on new Nvidia, AMD GPUs and Maia silicon
• Design, implement, and test functions or components for our AI/deep neural networks (DNN)/LLM frameworks and tools
• Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency of our systems
• Communicate and collaborate with our partners both internal and external

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to C/C++, Python
• OR equivalent experience
• 4+ years’ practical experience working on high performance applications and performance debugging and optimization on CPU's/GPU's
• Experience in DNN/LLM inference and experience in one or more DL frameworks such as PyTorch, Tensorflow, or ONNX Runtime and familiarity with CUDA, ROCm, Triton

• Health insurance
• 401k matching
• Flight privileges
• Discounts on products and services
• Generous time away
• Giving programs
• Opportunities to network and connect

Microsoft’s mission is to empower every person and every organization on the planet to achieve more.