The Tonic Validate application and Validate SDK (tonic_validate) allow you to measure how well your RAG LLM system performs.
Validate calculates rigorous LLM-assisted RAG evaluation metrics. You can also use tonic_validate to compute RAG metrics outside the context of a Validate project.
In Validate:
Development projects track the performance of a RAG system that is under development.
Production monitoring projects track how well a production RAG system answers questions from actual users.
For development projects, Validate also provides an integration with Ragas, tonic_ragas_logger, that allows you to visualize Ragas evaluation results in Validate.
Need help with Validate? Contact support@tonic.ai.
Start your Validate account
Sign up for a Validate account and create your first project.
Set up the Validate SDK
Install the SDK. Provide Validate and Open AI API keys.
Quickstart example
Use tonic_validate to log RAG metrics to a development project.
Types of RAG metrics
RAG metrics measure the quality of RAG LLM responses.
Create and manage benchmarks
A benchmark is a set of questions, optionally with expected responses to send to a development project.
Create and manage projects
A development project consists of a set of runs.
A production monitoring project tracks performance over time.
Start a new run
Start a new Validate run to calculate metrics for RAG LLM answers to questions.
View run results
Review average metric scores, and the grouping of values for the questions.
Connect your RAG system to Validate
Configure your RAG system to send user questions and system answers to Validate.
Track RAG system performance
View average metric scores over time for the RAG system.
End-to-end example with a llama index
Demonstrates an end-to-end Validate development project flow.