Selecting the training data for your models

Before you start training models, on the Model data setup page, you select the training data to use.

Model Data Setup page with selected training files

About training data

The training data is a much larger set of files than the test data, and can include hundreds of files or more. The data should ideally contain at least 1,000 values for the entity type. For example, for an entity type to identify health conditions, you might use 5 medical appointment reports in your test data, but several hundred medical reports for your training data.

Similar to the test files, the training files should be relatively small - no more than 5,000 words.

For training data, there is no option to paste in text. Training data files are either uploaded from a local file system or selected from a cloud storage solution.

If you selected the test data from a cloud storage solution, then you must use the same cloud storage solution for the training data. For example, if you selected the test data from Amazon S3, then you must select the training data from Amazon S3.

You can add files to the training data at any time. New files are only used for models that are trained after the files are added.

Uploading files from a local file system

If there are no training files, then on the Model data setup page, click Upload files, then search for and select the files to upload.

To add more uploaded files to the training data:

1. Click Add training data.

Add training data options to select additional training files

1. Click Upload files.

2. Search for and select the files to add to the training data.

Selecting files from cloud storage

If there are no training files, then on the Model data setup page, click the cloud storage solution to use, then select the files to add.

If the test data came from a cloud storage solution, then you must use the same cloud storage option for the training data.

To add more cloud storage files to the training data:

1. Click Add training data.

Add training data options to select additional training files

1. Select the cloud storage solution.

2. Select the files to add to the training data.

For training data, you can select entire folders. Textual then adds all of the files in the folder.

Displaying the content of a training data file

On the Model data setup page, to display the content of an uploaded file, click the file name.

Training data file statuses

Each training data file goes through the following statuses:

• Queued for upload - The file is not yet uploaded.

• Uploading - Textual is uploading the file.

• Ready - The file is uploaded and is used for subsequent model training.

Model training cannot start until all of the currently uploaded files are Ready.

Deleting training files

On the Model data setup page, to delete a training file:

1. Click its delete icon.

2. On the confirmation panel, you can choose to skip the confirmation when you delete training files. If you select this option, then the next time you delete a training file, the file is deleted immediately, and the panel does not display.

3. Click Delete.

When you delete a training file:

• For existing models that annotated the file:

  • The entity counts continue to reflect the entities that were detected in the file

  • The file name remains in the list on the model details.

  • The file name is dimmed, and you cannot display the file details.

• For models that are created after the file is deleted, the file is not annotated and is not displayed in the list on the model details.