Software Engineer II

Microsoft's Azure Machine Learning team is seeking a Software Engineer II to join their Models-As-A-Service team within the AI Platform division. This role focuses on building and operating the industry's largest-scale engineering system for Large Language Models (LLMs) and Generative AI Services. The team is responsible for providing serverless access to third-party models from providers like Mistral AI, Cohere, and Meta.

The position offers an exciting opportunity to work at the forefront of large-scale AI inferencing, dealing with state-of-the-art models and ensuring the platform can deliver AI capabilities to both individual developers and enterprise customers. The role combines technical challenges in high-performance computing with the latest advances in AI technology.

As a Software Engineer II, you'll be involved in designing and developing features for large-scale model inferencing, optimizing AI models, and working closely with product managers and model providers. The position offers competitive compensation ranging from $98,300 to $193,200 per year (higher in SF and NYC areas), comprehensive benefits, and the opportunity to work remotely.

This role is perfect for someone with strong programming skills, experience in high-performance systems, and a passion for AI technology. You'll be part of Microsoft's mission to empower every person and organization on the planet, working in an inclusive environment that values growth mindset, innovation, and collaboration.