Managing projects in Validate

A Tonic Validate project contains the results of Tonic Validate runs.

For a set of questions, which can be from a Tonic Validate benchmark, the run assesses the quality of the answers from your RAG system. If you also provide expected answers, then the run compares the answers from your RAG system against the provided answers. It also analyzes how the RAG system used additional context to answer the questions.

Each run generates an overall score and metrics.

For information about starting a run, go to Starting a Validate run.

For information about viewing run results, go to Viewing and managing runs.

You can also use our Ragas integration, tonic_ragas_logger, which allows you to display Validate visualizations of Ragas results.

Displaying the list of projects

The Validate home page includes the list of projects.

Tonic Validate home page

For each project, the list displays:

• The project name

• When the most recent run occurred

• A chart that maps the average overall score for each run over time

Displaying details for a project

To display the details for a project, click the project tile.

New project without runs

For a new project that does not have any runs, the project details page guides you through the required steps to create a run.

If you are using tonic_ragas_logger to visualize Ragas results in Validate, then select Ragas as the logging framework.

If you are using Validate runs to generate and visualize metrics, then select Validate as the logging framework.

Project details page for a project without runs

Project with runs

For a project that has completed runs, the project details page displays the list of runs, and provides an overview of the scores across the runs and questions.

Project details page with the overall scores and list of runs and questions

At the left is the list of runs for the project. From there, you can display details for the run results.

The tiles across the top contain the average overall score and average metrics scores for the most recent run.

By default, the graph displays the overall score across all of the runs over time. When you click a metric score tile, the graph updates to show the average metric score across the runs.

Graph that shows a specific metric score over time

Below the graph is the list of questions in the project benchmark. For each question, the list shows the overall score for a month ago and for the most recent run.

To filter the question list, in the filter field, type text from the question.

Filtering the list of questions

When you click a question, the graph is updated to show the average overall or metric score across runs for that specific question. To deselect the question, click it again.

Graph filtered to show a metric score over time for a specific question

Creating a project

For a new project, you provide a name.

To create a project:

1. On the Validate home page, click Create a Project.

2. In the Name field, type the name of the project, then click Save.

Create a new project panel to provide the project name

Validate displays the project details panel.

The details panel includes the project identifier, which you provide to identify the project when you make calls to the API. To copy the the project identifier to the clipboard, click Copy ID.

Project name with options to copy the project ID and name

Changing the project name

For an existing project, you can change the name.

To edit the project name:

1. Either:

   • From the Validate home page, click the options menu for the project, then click Edit.
   • From the project details page, click Edit Name.
2. On the Edit Project panel, in the Project Name field, type the new name for the project.

Edit Project page to update the project name

3. Click Save.

Deleting a project

To delete a project, from the projects list:

1. Click the options icon for the project.

2. In the options menu, click Delete.

Options menu for a project