Software Engineer II

Microsoft builds innovative technology solutions to empower every person and organization on the planet to achieve more.
$94,300 - $182,600
Machine Learning
Mid-Level Software Engineer
Remote
5,000+ Employees
2+ years of experience
AI · Enterprise SaaS

Description For Software Engineer II

Microsoft's AI Platform organization is seeking a Software Engineer II to join their team building the end-to-end Azure AI stack. This role focuses on GPU performance analysis and optimization for large-scale AI models. The position is part of the AI Frameworks team, which develops software for training and deploying advanced AI models, including work with OpenAI on Azure OpenAI service.

The role requires strong technical skills in C/C++, Python, and GPU development, with a focus on optimizing performance for large language models. You'll work with cutting-edge hardware, including Microsoft's next-generation supercomputers and the Maia-100 AI accelerator. The position offers the opportunity to impact Microsoft's flagship products, from Office to Teams and Xbox.

The ideal candidate will have experience with GPU optimization, deep learning frameworks, and a strong foundation in software engineering principles. You'll collaborate with researchers and developers, working on fundamental abstractions, programming models, and APIs for large-scale AI training and inference.

This position offers competitive compensation, comprehensive benefits, and the chance to work on transformative AI technology. Microsoft values diversity and inclusion, seeking different perspectives to drive innovation. The role provides up to 100% work-from-home flexibility with minimal travel requirements (0-25%).

Last updated 4 hours ago

Responsibilities For Software Engineer II

  • Engage with key partners to understand and implement performance analysis and optimization for state-of-the-art large language models (LLMs)
  • Work with cutting edge hardware stacks and fast-moving software stack to deliver best of class inference and optimal cost
  • Perform software development in C/C++, Python, and GPU development in languages such as CUDA, ROCm, or Triton
  • Embody Microsoft's Culture and Values

Requirements For Software Engineer II

Python
Kubernetes
  • Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience
  • 1+ years' experience with C/C++
  • Experience in optimizing GPU kernels for performance
  • Proficiency using profiling tools such as NVIDIA Visual Profiler, nvprof, and NVIDIA Nsight Compute
  • Experience in low-level program behavior, including performance and memory usage
  • Cross-team collaboration skills
  • Experience with deep learning frameworks (PyTorch, Tensorflow, ONNX)

Benefits For Software Engineer II

Medical Insurance
Parental Leave
Education Budget
  • Industry leading healthcare
  • Educational resources
  • Discounts on products and services
  • Savings and investments
  • Maternity and paternity leave
  • Generous time away
  • Giving programs
  • Opportunities to network and connect

Interested in this job?

Jobs Related To Microsoft Software Engineer II

Software Engineer II

Software Engineer II position at Microsoft's Gaming ML team, focusing on AI and Machine Learning applications in gaming and graphics, offering competitive salary and hybrid work arrangement in Vancouver.

Software Engineer II - AI Engineering Systems

Software Engineer II position at Microsoft's AI Frameworks team, focusing on developing AI infrastructure and systems for next-generation supercomputers and AI accelerators.

Software Engineer II - Bing Platform - Microsoft AI

Software Engineer II position at Microsoft's Bing Platform, focusing on AI infrastructure development and optimization of ML workloads, offering competitive compensation and comprehensive benefits.

Software Engineer

Software Engineer role at Microsoft Digital focusing on AI/ML development, offering hybrid work and competitive compensation, building next-gen employee experience solutions.

Software Engineer II

Microsoft is hiring a Software Engineer II to build and optimize their Azure ML model-serving platform, working with large-scale AI models including OpenAI's generative models.