ML Engineer - LLM Evaluation

August 27

🇺🇸 United States – Remote

🏄 California – Remote

Although this job is listed in California, this company may be open to hiring remote in other states.

⏰ Full Time

🟡 Mid-level

🟠 Senior

🤖 Machine Learning Engineer

Apply Now

Receive Emails with Similar Jobs

Dynamo AI

WebsiteLinkedInAll Job Openings

11 - 50 employees

🤖 Artificial Intelligence

📋 Compliance

🔐 Security

💰 $15.1M Series A on 2023-08

Description

• Own LLM evaluation processes and methods with a focus on generating benchmarks representative of real-world usage and safety vulnerabilities. • Generate high quality synthetic data, curate labels, and conduct rigorous benchmarking. • Deliver robust, scalable, and reproducible production code. • Push the envelope by developing methods for benchmarking that revamps how we assess the best LLMs for harmlessness and helpfulness. Your research will directly empower our customers to more feasibly deploy safe and responsible LLMs. • Co-author papers, patents, and presentations with our research team by integrating other members’ work with your vertical.

Requirements

• Domain knowledge in LLM evaluation and data curation techniques. • Extensive experience in designing and implementing LLM benchmarking, extending previous methods. Comfortability with leading end-to-end projects. • Adaptability and flexibility. In both the academic and startup world, a new finding in the community may necessitate an abrupt shift in focus. You must be able to learn, implement, and extend state-of-the-art research. • Preferred: past research or projects in benchmarking LLMs.