Validate workflows
Last updated
Was this helpful?
Last updated
Was this helpful?
The overall process to use a Tonic Validate development project to evaluate your RAG system consists of the following:
A Validate run analyzes a RAG system performance against a set of questions and optional ideal answers.
You can use the Validate application or SDK to add the benchmark to Validate.
The run configuration includes:
The project
The questions for to analyze the RAG performance. A Validate benchmark is one way to provide the question data.
Any metadata about the RAG data, such as the type of LLM, the embedder, or the retrieval algorithm
The metrics to calculate
Based on the run results, you update the RAG system to improve the results, then create another run.
You compare the run results to see if your changes improved the quality of the answers.
After you release your RAG system, you can use a Validate production monitoring project to track how well it answers user questions.
Each question that a user asked
The answer that the RAG system provided
The context that the RAG system used
As it receives the questions, Validate generates metric scores.
One way to provide the questions and answers is to .
Next, use the Validate application to .
Use the Validate SDK to .
From the Validate application, .
Use the Validate application to .
In your RAG system, you to send the following to the production monitoring project:
In the Validate application, you can for the questions that Validate received from the RAG system.
You can also .
