Senior Software Engineer (ML Inference and Performance)

Baseten is seeking a Senior Software Engineer focused on ML inference and performance to join their dynamic team. This role is ideal for someone passionate about advancing AI frontiers and thrives in a fast-paced startup environment.

Key responsibilities include:

• Implementing and refining cutting-edge techniques for ML model inference and infrastructure
• Deep diving into codebases of TensorRT, PyTorch, Transformers, CUDA, and other libraries
• Applying and scaling optimization techniques across various ML models, especially large language models
• Collaborating with a diverse team to design and implement innovative solutions
• Owning projects from idea to production

The ideal candidate should have:

• 3-10 years of experience with programming languages like Python, C++, or Go
• Deep understanding of software engineering principles and AI/ML inference solutions
• Strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM
• Experience with Docker and Kubernetes
• Deep understanding of GPU Architecture

Bonus points for:

• Proficiency in enhancing LLM performance
• Familiarity with LLM optimization techniques
• Experience with CUDA or similar technologies

Baseten offers a competitive compensation package, including unlimited PTO, 401k, and covered healthcare premiums. They provide a unique opportunity to be part of a rapidly growing startup in an exciting engineering field, fostering an inclusive work culture that supports learning and growth.

The company uses a tech stack including AWS, Kubernetes, Istio, Knative, Prometheus stack for infrastructure; Python, Django, Postgres, Redis for backend; and React, TypeScript, GraphQL for frontend.

Baseten is committed to fostering a diverse and inclusive workplace, providing equal employment opportunities to all employees and applicants.