VB Transform 2024 returns this July! More than 400 business leaders will gather in San Francisco July 9-11 to learn about advances in GenAI strategy and engage in thought-provoking discussions within the community. Find out how to attend here.
Galileo, a pioneer in enterprise generative AI, has unveiled Galileo Luna, a groundbreaking evaluation-driven model (EFM) suite that will transform the way enterprises evaluate GenAI systems. With Luna, Galileo aims to solve the critical issues of speed, cost, and accuracy that hinder the widespread adoption of generative AI in production environments.
“Galileo created Luna to address the limitations of current GenAI evaluation methods, which are slow, expensive, and often inaccurate,” Vikram Chatterji, co-founder and CEO of Galileo, told VentureBeat. “We were motivated by the need for very low-latency, cost-effective, and highly accurate assessments in a production environment.”
The development of Luna is a significant milestone for Galileo, which has been at the forefront of enterprise GenAI since its founding in early 2021. The company's commitment to pushing the boundaries of AI evaluation is evident in an intensive R&D process spanning nearly a year. To Luna's creations.
Purpose-built models that redefine speed, cost, and accuracy
At the heart of Luna's innovation are small-scale language models carefully tailored to specific evaluation tasks, such as hallucination detection, context quality assessment, data leak prevention, and malicious prompt identification. This special design allows Luna to deliver unparalleled performance across three key metrics: speed, cost, and accuracy.
VB Transform 2024 registration is now open
Join business leaders at a major AI event in San Francisco July 9-11. Connect with your peers, explore the opportunities and challenges of Generative AI, and learn how to integrate AI applications into your industry. Register now
“Luna outperforms GPT-3.5 in speed, cost and accuracy through several innovations,” Chatterji explained. “Luna significantly reduces computational overhead and cost by leveraging small language models purpose-built for specific evaluation tasks. These design choices allow for evaluations that are 97% cheaper and 11 times faster than those performed with GPT-3.5.”
But it's not just about speed and cost. Luna also boasts industry-leading accuracy, up to 20% better than previous methods, for detecting hallucinations, immediate injections, personally identifiable information (PII), and more. “Using advanced techniques such as multiple compact language models and intelligent chunking allows the Luna model to better maintain context and provide more accurate evaluations,” Chatterji added.
Transform evaluation without ground truth datasets.
One of the most surprising aspects of Luna is that it can work without any existing real-world data sets. By leveraging pre-trained evaluation models that are fine-tuned on a variety of domain-specific datasets, Luna eliminates the time-consuming and costly process of creating custom test sets. These innovations streamline the assessment process and reduce reliance on extensive human-generated data.
Luna's potential applications are broad, and Chatterji highlights its relevance in industries that require high reliability and speed in AI evaluation. “Luna is especially powerful for large enterprise applications that require volume and throughput (e.g. millions of queries per month). “Fortune 100 companies in healthcare, finance and telecommunications are finding Luna particularly useful,” he said.
Customization and continuous evolution following rapid GenAI advancements
Use cases range from real-time monitoring of AI output and hallucination detection in AI-generated content to ensuring the safety and quality of chatbot interactions. Additionally, Galileo's Fine Tune product allows Luna to be tailored to specific customer requirements, achieving accuracy levels of 95% or better for critical tasks in industries such as pharmaceuticals and financial services.
As the generative AI landscape continues to rapidly evolve, Galileo is committed to staying at the forefront of innovation. Chatterji emphasized that Luna will expand in three key ways: expanding support for more types of assessment tasks, continuing to improve accuracy, and further reducing costs and latency.
“Galileo is committed to expanding the boundaries of what is possible in AI evaluation and helping organizations bring trustworthy AI into production,” said Chatterji. “As the generative AI landscape continues to evolve, Galileo is committed to providing our customers with cutting-edge assessment capabilities that help businesses deploy AI pragmatically and inspire confidence and trust among consumers.”
With the launch of Luna, Galileo solidifies its position as a leader in enterprise GenAI evaluation. As more organizations seek to harness the power of generative AI, Luna's ability to provide fast, cost-effective, and accurate assessments will be a critical factor in driving widespread adoption of this innovative technology and unlocking its full potential. no see.