Taro Logo

Manager, LLM Accuracy Evaluation

World leader in accelerated computing, pioneering AI and digital twins technology.
Machine Learning
Staff Software Engineer
Remote
5,000+ Employees
8+ years of experience
AI · Enterprise SaaS
This job posting may no longer be active. You may be interested in these related jobs instead:

Description For Manager, LLM Accuracy Evaluation

NVIDIA is seeking a visionary Manager to lead their LLM Accuracy Evaluation team, focusing on pioneering new methodologies for evaluating cutting-edge deep learning models. This role offers a unique opportunity to shape the future of AI at one of the world's leading technology companies. The position involves leading a team of exceptional engineers working on flagship models like Gemma and Llama-3, optimizing them as NVIDIA Inference Microservices (NIM). The role requires deep expertise in AI technologies, leadership experience, and the ability to drive innovation while maintaining high engineering standards. The successful candidate will work with state-of-the-art GPU clusters and have early access to unreleased hardware, directly influencing NVIDIA's roadmap and the broader AI landscape. The position combines technical leadership with team management, requiring both deep technical knowledge and strong interpersonal skills. The role offers the opportunity to work with cutting-edge AI technology while developing and implementing new evaluation methodologies for AI model performance. NVIDIA provides a collaborative environment focused on pushing the boundaries of AI technology and maintains a strong commitment to diversity and inclusion in their workplace.

Last updated 7 months ago

Responsibilities For Manager, LLM Accuracy Evaluation

  • Lead and mentor a team of highly skilled engineers
  • Drive accuracy evaluation of flagship models as NVIDIA Inference Microservices (NIM)
  • Collaborate with internal customers across the organization
  • Supervise development and implementation of new methodologies for evaluating AI models
  • Build a culture of continuous improvement

Requirements For Manager, LLM Accuracy Evaluation

Kubernetes
Python
  • BS, MS, or PhD in Computer Science, AI, Applied Math, or related field
  • 8+ years of industry experience, including 3+ years in leadership/management
  • Proven experience leading engineering teams
  • Strong understanding of AI technologies, including NLP, LLMs, and deep learning
  • Exceptional communication and interpersonal skills

Interested in this job?