Manager, LLM Accuracy Evaluation

World leader in accelerated computing, pioneering AI and digital twins technology.
Machine Learning
Staff Software Engineer
Remote
5,000+ Employees
8+ years of experience
AI · Enterprise SaaS

Description For Manager, LLM Accuracy Evaluation

NVIDIA is seeking a visionary Manager to lead their LLM Accuracy Evaluation team, focusing on pioneering new methodologies for evaluating cutting-edge deep learning models. This role offers a unique opportunity to shape the future of AI at one of the world's leading technology companies. The position involves leading a team of exceptional engineers working on flagship models like Gemma and Llama-3, optimizing them as NVIDIA Inference Microservices (NIM). The role requires deep expertise in AI technologies, leadership experience, and the ability to drive innovation while maintaining high engineering standards. The successful candidate will work with state-of-the-art GPU clusters and have early access to unreleased hardware, directly influencing NVIDIA's roadmap and the broader AI landscape. The position combines technical leadership with team management, requiring both deep technical knowledge and strong interpersonal skills. The role offers the opportunity to work with cutting-edge AI technology while developing and implementing new evaluation methodologies for AI model performance. NVIDIA provides a collaborative environment focused on pushing the boundaries of AI technology and maintains a strong commitment to diversity and inclusion in their workplace.

Last updated a month ago

Responsibilities For Manager, LLM Accuracy Evaluation

  • Lead and mentor a team of highly skilled engineers
  • Drive accuracy evaluation of flagship models as NVIDIA Inference Microservices (NIM)
  • Collaborate with internal customers across the organization
  • Supervise development and implementation of new methodologies for evaluating AI models
  • Build a culture of continuous improvement

Requirements For Manager, LLM Accuracy Evaluation

Kubernetes
Python
  • BS, MS, or PhD in Computer Science, AI, Applied Math, or related field
  • 8+ years of industry experience, including 3+ years in leadership/management
  • Proven experience leading engineering teams
  • Strong understanding of AI technologies, including NLP, LLMs, and deep learning
  • Exceptional communication and interpersonal skills

Interested in this job?

Jobs Related To NVIDIA Manager, LLM Accuracy Evaluation

Machine Learning Software Platform Architect

Senior ML Platform Architect role at NVIDIA focusing on LLM infrastructure for chip design

Software Manager, Deep Learning Frameworks

Lead NVIDIA's Deep Learning Frameworks team, developing core ML software and algorithms while managing engineering talent.

Gen AI Product Evangelist Engineer, Retail

Senior technical role combining AI expertise with developer advocacy to create educational content and demos for NVIDIA's retail-focused Gen AI applications.

Manager, Neural Reconstruction Software

Lead NVIDIA's Neural Reconstruction Software team, developing cutting-edge AI technologies for computer vision and robotics applications.

Senior Manager, Visual AI Microservices

Lead the development of Visual AI Microservices at NVIDIA, managing teams building cutting-edge computer vision and AI services for developer enablement.