# Iterating over the guidelines to use for model training On the **Guidelines refinement** page, you prepare the guidelines that define a model. To test each version of the guidelines, Textual uses the guidelines to detect entity values in the test data. It then generates scores to indicate how closely those detection results match the values that you established during your initial review. Textual also generates recommendations to improve the guidelines. ## **Viewing the initial version of the guidelines** To work on the guidelines, click **Guidelines refinement**. The **Guidelines refinement** option is enabled when you complete the review on the initial set of test files. The first time you display the **Guidelines refinement** page, Textual uses the guidelines that you provided during the entity type creation to populate the **Version 1** tab.

Guidelines refinement page for a model-based custom entity type

At the left are the guidelines. At the right is the list of test data files. ## **File statuses for the guidelines refinement** For each version of the guidelines, Textual uses the guidelines to detect entity values in the test data. The file statuses are: * **Queued for annotation** - Textual has not yet scanned the file. * **Annotating** - Textual is in the process of scanning the file. * **Annotated** - The scan is complete. ## **Reviewing the test scores for the guidelines** When Textual uses guidelines to detect entity values in the test files, it sets the number of detected entities and a set of scores. The scores reflect how well the detections match the entity values that you established in the test data setup. If you change the established values in the test data, Textual updates the scores for the guidelines. The overall entity count and scores across all files are displayed across the top of the page. The file list displays the entity count and scores for each file.

Overall scores and file scores in the **Guidelines Refinement** list

The scores are: * **Precision score** - Measures the accuracy of positive predictions. Indicates how many of the detected entities were correctly identified. For example, the guidelines detect 10 values. If only 3 of those are correct, then the precision score is lower than if 7 of those are correct. * **Recall score** - Measures the model's ability to find all of the entities. Indicates how many of the actual entities it detected. For example, the guidelines detect 10 correct values. If the total number of correct values is 20, then the recall score is lower than if the total number of correct values is 12. * **F1 score** - The harmonic mean of precision and recall. The goal is to have a balance between precision and recall. The guidelines should produce annotations that are both accurate and complete. Detecting all of the correct values is not useful if the guidelines also detect a large number of incorrect values. And detecting only correct values is not useful if the guidelines only detect a fraction of the total number of correct values. ## **Reviewing the guideline detections** To review the entity values that Textual detected based on the current version of the guidelines, click the file name. ## **Editing the guidelines** Based on how accurately Textual detected the entity values, Textual generates suggested changes to the guidelines. For example, it might suggest additional language to more specifically identify values that the previous version either missed or detected incorrectly. To start a new version of the guidelines: 1. Click **Edit**. If there are suggestions, you can also click **Review**.

Edit and Review options for the annotation guidelines

2. On the **Annotation guidelines** panel, the current guidelines are displayed in an editable text area on the left.\ \ On the right is a summary of the suggested updates to the guidelines.\ \ To display the proposed replacement guidelines, toggle **Show diff** to the on position.

Annotation guidelines panel with AI suggestions to improve the guidelines in the next version

3. To update the guidelines, you can either: * Update the guidelines manually. * Accept all of the suggestions, and replace the current guidelines. To do this, click **Accept changes**. * Manually copy text from the suggestions and paste it into the guidelines. 4. To save the guidelines version, and start the detection and scoring, click **Save new version**. Textual creates a new tab for the new version of the guidelines. The tab label is **Version *****n***, where ***n*** is incremented for each new version. The most recent version is at the left.