Senior Software Engineer-Distributed Inference

World leader in accelerated computing, pioneering AI and digital twins technology to transform industries.
$184,000 - $356,500
Distributed Systems
Senior Software Engineer
Remote
5,000+ Employees
8+ years of experience
AI · Enterprise SaaS

Description For Senior Software Engineer-Distributed Inference

NVIDIA is seeking a Senior Software Engineer to join their GPU-accelerated deep learning software team, focusing on the Triton Inference Server. This role combines distributed systems expertise with cutting-edge AI technology, working on tools that make deep learning model deployment more accessible. The position offers the opportunity to work with state-of-the-art technology in AI, particularly in areas like LLM and Generative AI. The role involves developing performance analysis tools, collaborating with researchers, and implementing efficient algorithms for deep learning systems. NVIDIA's legacy of innovation in computer graphics and AI computing makes this an exciting opportunity for someone passionate about distributed systems and AI. The company offers competitive compensation including a base salary range of $184,000-$356,500, plus equity and benefits. The position is remote-friendly, allowing for flexible work arrangements while being part of a team that's driving innovation in AI and accelerated computing. NVIDIA's commitment to diversity and inclusion, combined with their position as a leader in AI computing, makes this an attractive opportunity for talented engineers looking to make a significant impact in the field of distributed AI systems.

Last updated 13 days ago

Responsibilities For Senior Software Engineer-Distributed Inference

  • Develop and enhance functionalities within GenAI-Perf, Triton Performance Analyzer and Model Analyzer tools
  • Collaborate with researchers and engineers to understand performance analysis needs
  • Set up, execute, and analyze performance of LLM, Generative AI and deep learning models
  • Develop algorithms for measuring deep learning throughput and latency
  • Integrate tools for deep learning performance analysis
  • Automate testing processes
  • Contribute to technical documentation and user guides

Requirements For Senior Software Engineer-Distributed Inference

Python
Kubernetes
  • Bachelor's, Masters or PhD or equivalent experience
  • 8+ years in Computer Science, computer architecture, or related field
  • Knowledge of distributed systems programming
  • Excellent Python programming and software design skills
  • Experience with deep learning frameworks (PyTorch, TensorFlow, TensorRT)
  • Experience with cloud computing platforms (AWS, Azure, GCP)
  • Experience with HTTP REST, gRPC, protobuf, JSON

Benefits For Senior Software Engineer-Distributed Inference

Equity
Medical Insurance
  • Equity
  • Medical Insurance

Interested in this job?

Jobs Related To NVIDIA Senior Software Engineer-Distributed Inference

Senior System Software Engineer, NCCL - Partner Enablement

Senior System Software Engineer position at NVIDIA focusing on NCCL partner enablement, requiring expertise in distributed systems and HPC.

Senior AI-HPC Storage Engineer

Senior AI-HPC Storage Engineer position at NVIDIA focusing on designing and implementing distributed storage solutions for AI and HPC workloads.

Senior Software Engineer - HPC

Senior Software Engineer position at NVIDIA focusing on HPC infrastructure, requiring 10+ years of experience in distributed systems and cloud computing.

Systems Engineer, Enterprise

Senior Systems Engineer position at NVIDIA focusing on enterprise HPC server deployment, requiring 6+ years experience and strong hardware/software expertise.

Senior System Software Engineer, Distributed Systems - DGX Cloud

Senior System Software Engineer position at NVIDIA focusing on distributed systems and DGX Cloud infrastructure.