Validate workflow

The overall process to use Tonic Validate to evaluate your RAG system consists of the following:

Overview diagram of the Tonic Validate workflow

Create your benchmark (optional)

A Validate run analyzes a RAG system performance against a set of questions and optional ideal answers.

One way to provide the questions and answers is to configure a benchmark in Validate.

You can use the Validate application or SDK to add the benchmark to Tonic Validate.

Create your project

Next, use the Validate application to create a project.

Create a run

Use the Validate SDK to create a run for the project.

The run configuration includes:

• The project

• The questions for to analyze the RAG performance. A Validate benchmark is one way to provide the question data.

• Any metadata about the RAG data, such as the type of LLM, the embedder, or the retrieval algorithm

• The metrics to calculate

Review the run results

From the Validate application, review the scores and metrics from the run.

Update and iterate

Based on the run results, you update the RAG system to improve the results, then create another run.

You compare the run results to see if your changes improved the quality of the answers.