Senior Software Engineer, AWS Neuron Inference

Amazon is a global technology company providing cloud computing, AI, and e-commerce services.
$151,300 - $261,500
Machine Learning
Senior Software Engineer
In-Person
5+ years of experience
AI · Enterprise SaaS

Description For Senior Software Engineer, AWS Neuron Inference

AWS Neuron is Amazon's complete software stack for cloud-scale machine learning accelerators, specifically the AWS Inferentia and Trainium. This Senior Software Engineering role is part of the Machine Learning Inference Applications team, focusing on developing and optimizing core LLM inference components.

The position involves working with cutting-edge LLM technology, including attention mechanisms, MLP, quantization, speculative decoding, and mixture of experts. You'll collaborate with chip architects, compiler engineers, and runtime engineers to maximize performance on Neuron devices for various models like Llama 3.3 70B, 3.1 405B, DBRX, and Mixtral.

The team culture emphasizes knowledge-sharing and mentorship, with senior members providing one-on-one mentoring and thorough code reviews. Career growth is prioritized through strategic project assignments that help develop engineering expertise. The role offers competitive compensation ranging from $151,300 to $261,500 based on location, plus equity and comprehensive benefits.

This is an excellent opportunity for experienced engineers passionate about machine learning optimization and looking to work on large-scale, impactful projects. The position requires strong programming skills, understanding of ML fundamentals, and the ability to work collaboratively across teams. Amazon's inclusive culture and commitment to diversity make it an ideal workplace for innovation and professional growth.

Last updated 3 days ago

Responsibilities For Senior Software Engineer, AWS Neuron Inference

  • Development and performance optimization of core building blocks of LLM Inference
  • Work on Attention, MLP, Quantization, Speculative Decoding, Mixture of Experts
  • Adapt latest research in LLM optimization to Neuron chips
  • Work across teams and organizations
  • Extract best performance from both open source and internally developed models

Requirements For Senior Software Engineer, AWS Neuron Inference

Python
Java
  • Bachelor's degree in computer science or equivalent
  • 5+ years of full software development life cycle experience
  • 5+ years of programming using modern programming languages (Java, C++, or C#)
  • Experience with object-oriented design
  • Understanding of Machine learning models, architectures, training and inference lifecycles

Benefits For Senior Software Engineer, AWS Neuron Inference

Medical Insurance
Dental Insurance
Vision Insurance
401k
Equity
  • Medical benefits
  • Financial benefits
  • Equity compensation
  • Sign-on payments
  • Comprehensive benefits package

Interested in this job?

Jobs Related To Amazon Senior Software Engineer, AWS Neuron Inference

Senior Software Development Engineer, Ring & Blink AI

Senior Software Engineer position at Amazon's Ring & Blink AI team focusing on computer vision and machine learning software development for smart home devices.

Senior Software Developer, Amazon Games AI

Senior Software Developer position at Amazon Games focusing on implementing ML, RL, and Generative AI techniques for game development.

Product Development Engineer, Annapurna Labs Silicon Operations

Senior Product Development Engineer role at AWS-Annapurna Labs focusing on silicon yield optimization for machine learning accelerator servers.

Senior Software Development Engineer - Amazon Music Machine Learning

Senior ML Engineer role at Amazon Music, leading recommendation systems development and team mentoring in Berlin.

Emulation Engineer, AWS Annapurna Labs

Senior Emulation Engineer position at Amazon working on AWS Inferentia and ML acceleration hardware, offering competitive compensation and benefits.