ML Engineer — LLM Evaluation
Description For ML Engineer — LLM Evaluation
At Dynamo AI, we are at the forefront of developing LLMs with a focus on safety, privacy, and real-world responsibility. Our ML team combines academic research expertise with industry applications to empower Fortune 500 companies in adopting frontier research for their next-generation LLM products.
As an ML Engineer specializing in LLM Evaluation, you will play a crucial role in our mission to democratize AI advancements responsibly. You'll be working on the premier platform for private and personalized LLMs, providing the fastest end-to-end solution to deploy research in the real world.
Key responsibilities include:
- Owning LLM evaluation processes and methods
- Generating high-quality synthetic data and conducting rigorous benchmarking
- Delivering robust, scalable, and reproducible production code
- Developing innovative benchmarking methods for assessing LLMs' harmlessness and helpfulness
- Collaborating with our research team on papers, patents, and presentations
We're looking for candidates with strong domain knowledge in LLM evaluation, extensive experience in LLM benchmarking, and the ability to adapt quickly to new research findings. You'll be part of a fast-paced team of ML Ph.D.'s and builders, free from Big Tech and academic constraints.
Join us if you're excited about:
- Working on cutting-edge AI technology with real-world impact
- Democratizing state-of-the-art research on safe and responsible AI
- Seeing your work influence end customers in weeks, not years
- Building a platform that empowers fair, unbiased, and responsible development of LLMs
At Dynamo AI, we're committed to maintaining compliance with all applicable local and state laws regarding job listings and salary transparency. We strive to ensure our practices promote fairness, equity, and transparency for all candidates.
If you're passionate about pushing the boundaries of LLM evaluation and want to make a significant impact in the field of responsible AI, we encourage you to apply and be part of our innovative team at Dynamo AI.
Responsibilities For ML Engineer — LLM Evaluation
- Own LLM evaluation processes and methods with a focus on generating benchmarks representative of real-world usage and safety vulnerabilities
- Generate high quality synthetic data, curate labels, and conduct rigorous benchmarking
- Deliver robust, scalable, and reproducible production code
- Develop methods for benchmarking that revamps how we assess the best LLMs for harmlessness and helpfulness
- Co-author papers, patents, and presentations with our research team
Requirements For ML Engineer — LLM Evaluation
- Domain knowledge in LLM evaluation and data curation techniques
- Extensive experience in designing and implementing LLM benchmarking, extending previous methods
- Comfortability with leading end-to-end projects
- Adaptability and flexibility to learn, implement, and extend state-of-the-art research
- Preferred: past research or projects in benchmarking LLMs
Jobs Related To Dynamo AI ML Engineer — LLM Evaluation
