Software Engineer II

Microsoft's Azure Machine Learning team is seeking a Software Engineer II to join their Inference team, focusing on next-generation model serving including hosting OAI models like ChatGPT, Bing, and Office scale implementations. The role involves building highly reliable and available platforms for model inferencing at scale, working on high throughput/low latency scenarios, and driving performance optimization capabilities. The position is part of Microsoft's broader mission to democratize ML and make it available to every enterprise, developer, and data scientist. The team currently serves billions of requests per day on cutting-edge scenarios and models across the company. This is an excellent opportunity for engineers passionate about AI, distributed systems, and large-scale applications, offering a hybrid work environment with up to 50% work from home flexibility. The role provides competitive compensation, comprehensive benefits, and the chance to work with diverse, remote teams on innovative AI solutions.