Previewing Textual detection and redaction

The Tonic Textual Home page provides a tool that allows you to see how Textual detects and replaces values in plain text or an uploaded file.

It also provides a preview of the redaction configuration options, including:

• How to replace the values for each entity type.

• Added and excluded values for each entity type.

The Home page displays automatically when you log in to Textual. To return to the Home page from other pages, in the navigation menu, click Home.

Providing the content to redact

To provide the content to redact, you can enter text directly, or you can upload a file.

Entering text

As you enter or paste text in the Original Content text area, Textual displays the redacted version in the Results panel at the right.

Using one of the samples

Textual also provides sample text options for some common use cases. To populate the text with a sample, under Try a sample, click the sample to use.

Uploading a file

You can also redact .txt or .docx files.

To provide a file, either:

• Drag and drop the file to the Original Content text area.

• Click the upload prompt, then search for and select the file.

Textual processes the file and then displays the redacted version in the Results panel. The Original Content text area is removed.

Clearing the text

To clear the text, click Clear.

Selecting the handling option for an entity type

The handling option indicates how Textual replaces a detected value for an entity type. You can experiment with different handling options.

Note that the updated configuration is only used for the current redacted text. When you clear the text, Textual also clears the configuration.

The options are:

• Redact - This is the default value. Textual replaces the value with the name of the entity type. For example, the first name John is replaced with NAME_GIVEN.

• Synthesize - Textual replaces the value with a realistic generated value. For example, the first name John is replaced with the first name Michael. The replacement values are consistent, which means that a given value always has the same replacement. For example, Michael is always the replacement value for John.

• Off - Textual ignores the value and copies it as is to the Results panel.

To change the handling option for an entity type:

1. In the Results panel, click an instance of the entity type.

2. On the configuration panel, click the handling option to use.

Textual updates all instances of that entity type to use the selected handling option.

For example, if you change the handling option for NAME_GIVEN to Synthesize, then all instances of first names are replaced with realistic values.

Defining added and excluded values

For each entity type in entered text, you can use regular expressions to define added and excluded values.

• Added values are values that Textual does not detect for an entity type, but that you want to include. For example, you might have values that are specific to your company or industry.

• Excluded values are values that you do not want Textual to identify as a given entity type.

Note that the configuration is only used for the current redacted text. When you clear the text, Textual also clears the configuration.

Also, this option is only available for text that you enter directly. For an uploaded file, to do additional configuration or to download the file, you must create a dataset from the file.

Displaying the configuration panel

To display the configuration panel for added and excluded values, click Fine-tune Results.

The Fine-Tune Results panel displays the list of configured rules for the current text. For each rule, the list includes:

• The entity type.

• Whether the rule adds or excludes values.

• The regular expression to identify the added or excluded values.

Adding a rule to add or exclude values

On the Fine-Tune Results panel, to create a rule:

1. Click Add Rule.

1. From the entity type dropdown list, select the entity type that the rule applies to.

2. From the rule type dropdown list:

   • If the rule adds values, then select Include.

   • If the rule excludes values, then select Exclude.

3. In the regular expression field, provide the regular expression to use to identify the values to add or exclude.

4. To save the rule, click the save icon.

Editing a rule

To edit a rule:

1. On the Fine-Tune Results panel, click the edit icon for the rule.

2. Update the configuration.

3. Click the save icon.

Deleting a rule

On the Fine-Tune Results panel, to delete a rule, click its delete icon.

Creating a dataset from an uploaded file

From an uploaded file, you can create a dataset that contains the file.

You can then provide additional configuration, such as added and excluded values, and download the redacted file.

To create a dataset from an uploaded file:

1. Click Download.

2. Click Create a Dataset.

Textual displays the dataset details for the new dataset. The dataset name is Playground Dataset <number>, where the number reflects the number of datasets that were created from the Home page.

The dataset contains the uploaded file.

Viewing and copying the request code

When Textual generates the redacted version of the text, it also generates the corresponding API request. The request includes the entity type configuration.

To view the API request code, click Show Code.

To hide the code, click Hide Code.

Selecting the request code type

On the code panel:

• The Python tab contains the Python version of the request.

• The cURL tab contains the cURL version of the request.

Copying the request code

To copy the currently selected version of the request code, click Copy Code.

Enabling and using additional LLM processing of detected entities

Textual offers an option to send detected entity information to a custom Large Language Model (LLM) to synthesize accurate replacements.

LLM synthesis methods

Textual provides the following LLM synthesis methods.

ReplacementSynthesis

ReplacementSynthesis redacts sensitive values. It uses the LLM to generate contextually appropriate replacements based on the surrounding text.

When you use this method:

1. Textual identifies sensitive values in the text.

2. Textual redacts the values and sends the following to the LLM:

   • Redacted placeholders for the detected values, such as ORGANIZATION or NAME_GIVEN

   • The positions of the detected entities

   • The surrounding text context

   Textual does not send the original sensitive values to the LLM.

3. The LLM analyzes the context.

4. The LLM generates realistic replacement values that fit naturally within the text.

GroupingSynthesis

GroupingSynthesis does the following:

• Groups related entities

• Generates new entity names

• Uses the LLM to reproduce the original format of the value

When you use this method:

1. Textual sends the detected entity values and surrounding text to the LLM. To enable grouping and format pattern recognition, Textual must send the original sensitive values.

2. The LLM groups entities based on whether they refer to the same thing, concept, or person. Grouping is only done within each entity type. For example, Lyon the person and Lyon the city are never grouped together.

3. The LLM chooses a representative value for each group. For example, if the content includes Will, William, and W.I.L.L, it chooses William as the most complete form.

4. The representative value is sent to Textual's standard, non-LLM synthesis generators to get a replacement value.

5. The LLM formats the replacement to match the original format. For example, if Will is replaced with Rob, then W.I.L.L becomes R.O.B.

Making the LLM processing available

To enable the LLM processing, set the environment variable ENABLE_EXPERIMENTAL_SYNTHESIS to True. If this is not set to True, then the LLM processing does not work.

You must also set up the Solar.LLM container.

Configuring the Solar.LLM container

To configure the container, you can use the following Docker Compose content as a reference:

services:
  textual-llm:
    image: textual-llm:[textual-version-here]
    container_name: textual-llm
    volumes:
      - llm-models:/app/models
    ports:
      - "11443:11443"
    secrets:
      - llm_aws_key_id
      - llm_aws_access_key
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
    networks:
      - llm-network

volumes:
  llm-models:

networks:
  llm-network:
    driver: bridge

secrets:
  llm_aws_key_id:
    environment: "LLM_AWS_KEY_ID"
  llm_aws_access_key:
    environment: "LLM_AWS_ACCESS_KEY"

The AWS keys are used to download the Textual custom models. To obtain a copy of the keys, contact your Tonic.ai support representative.

Enabling the LLM processing for entered text

After you enter text in the Original Content panel, to enable the LLM processing, in the Results panel, click Use an LLM to perform AI synthesis.

You cannot use this option for text that contains more than 100 words.

By default, the LLM processing applies the following synthesis methods to the entity types:

generator_config = {
    "NUMERIC_VALUE": PiiState.ReplacementSynthesis,
    "LANGUAGE": PiiState.ReplacementSynthesis,
    "MONEY": PiiState.ReplacementSynthesis,
    "PRODUCT": PiiState.ReplacementSynthesis,
    "EVENT": PiiState.ReplacementSynthesis,
    "WORK_OF_ART": PiiState.ReplacementSynthesis,
    "LAW": PiiState.ReplacementSynthesis,
    "US_PASSPORT": PiiState.ReplacementSynthesis,
    "MEDICAL_LICENSE": PiiState.ReplacementSynthesis,
    "DATE_TIME": PiiState.GroupingSynthesis,
    "US_BANK_NUMBER": PiiState.ReplacementSynthesis,
    "NRP": PiiState.ReplacementSynthesis,
    "US_SSN": PiiState.GroupingSynthesis,
    "IP_ADDRESS": PiiState.Synthesis,
    "ORGANIZATION": PiiState.GroupingSynthesis,
    "PHONE_NUMBER": PiiState.GroupingSynthesis,
    "US_ITIN": PiiState.ReplacementSynthesis,
    "LOCATION": PiiState.GroupingSynthesis,
    "LOCATION_ADDRESS": PiiState.GroupingSynthesis,
    "LOCATION_CITY": PiiState.GroupingSynthesis,
    "LOCATION_STATE": PiiState.GroupingSynthesis,
    "LOCATION_ZIP": PiiState.GroupingSynthesis,
    "LOCATION_COUNTRY": PiiState.ReplacementSynthesis,
    "CREDIT_CARD": PiiState.GroupingSynthesis,
    "US_DRIVER_LICENSE": PiiState.ReplacementSynthesis,
    "EMAIL_ADDRESS": PiiState.ReplacementSynthesis,
    "IBAN_CODE": PiiState.ReplacementSynthesis,
    "URL": PiiState.ReplacementSynthesis,
    "NAME_GIVEN": PiiState.GroupingSynthesis,
    "NAME_FAMILY": PiiState.GroupingSynthesis,
    "PERSON": PiiState.GroupingSynthesis,
    "GENDER_IDENTIFIER": PiiState.ReplacementSynthesis,
    "OCCUPATION": PiiState.ReplacementSynthesis,
    "USERNAME": PiiState.ReplacementSynthesis,
    "PASSWORD": PiiState.ReplacementSynthesis,
    "PERSON_AGE": PiiState.GroupingSynthesis,
    "DOB": PiiState.GroupingSynthesis,
    "CC_EXP": PiiState.GroupingSynthesis,
    "CVV": PiiState.GroupingSynthesis,
    "PROJECT_NAME": PiiState.ReplacementSynthesis,
    "MICR_CODE": PiiState.ReplacementSynthesis,
    "HEALTHCARE_ID": PiiState.ReplacementSynthesis,
    "NUMERIC_PII": PiiState.ReplacementSynthesis,
    "LOCATION_COMPLETE_ADDRESS": PiiState.ReplacementSynthesis,
}

When you clear the text, Textual reverts to the default processing.

Processing with the SDK

In the Python SDK, to use LLM synthesis, call the redact function.