LogoLogo
Release notesPython SDK docsDocs homeTextual CloudTonic.ai
  • Tonic Textual guide
  • Getting started with Textual
  • Previewing Textual detection and redaction
  • Entity types that Textual detects
    • Built-in entity types
    • Managing custom entity types
  • Language support in Textual
  • Datasets - Create redacted files
    • Datasets workflow for text redaction
    • Creating and managing datasets
    • Assigning tags to datasets
    • Displaying the file manager
    • Adding and removing dataset files
    • Reviewing the sensitivity detection results
    • Configuring the redaction
      • Configuring added and excluded values for built-in entity types
      • Working with custom entity types
      • Selecting the handling option for entity types
      • Configuring synthesis options
      • Configuring handling of file components
    • Adding manual overrides to PDF files
      • Editing an individual PDF file
      • Creating templates to apply to PDF files
    • Sharing dataset access
    • Previewing the original and redacted data in a file
    • Downloading redacted data
  • Pipelines - Prepare LLM content
    • Pipelines workflow for LLM preparation
    • Viewing pipeline lists and details
    • Assigning tags to pipelines
    • Setting up pipelines
      • Creating and editing pipelines
      • Supported file types for pipelines
      • Creating custom entity types from a pipeline
      • Configuring file synthesis for a pipeline
      • Configuring an Amazon S3 pipeline
      • Configuring a Databricks pipeline
      • Configuring an Azure pipeline
      • Configuring a Sharepoint pipeline
      • Selecting files for an uploaded file pipeline
    • Starting a pipeline run
    • Sharing pipeline access
    • Viewing pipeline results
      • Viewing pipeline files, runs, and statistics
      • Displaying details for a processed file
      • Structure of the pipeline output file JSON
    • Downloading and using pipeline output
  • Textual Python SDK
    • Installing the Textual SDK
    • Creating and revoking Textual API keys
    • Obtaining JWT tokens for authentication
    • Instantiating the SDK client
    • Datasets and redaction
      • Create and manage datasets
      • Redact individual strings
      • Redact individual files
      • Transcribe and redact an audio file
      • Configure entity type handling for redaction
      • Record and review redaction requests
    • Pipelines and parsing
      • Create and manage pipelines
      • Parse individual files
  • Textual REST API
    • About the Textual REST API
    • REST API authentication
    • Redaction
      • Redact text strings
  • Datasets
    • Manage datasets
    • Manage dataset files
  • Snowflake Native App and SPCS
    • About the Snowflake Native App
    • Setting up the app
    • Using the app
    • Using Textual with Snowpark Container Services directly
  • Install and administer Textual
    • Textual architecture
    • Setting up and managing a Textual Cloud pay-as-you-go subscription
    • Deploying a self-hosted instance
      • System requirements
      • Deploying with Docker Compose
      • Deploying on Kubernetes with Helm
    • Configuring Textual
      • How to configure Textual environment variables
      • Configuring the number of textual-ml workers
      • Configuring the number of jobs to run concurrently
      • Configuring the format of Textual logs
      • Setting a custom certificate
      • Configuring endpoint URLs for calls to AWS
      • Enabling PDF and image processing
      • Setting the S3 bucket for file uploads and redactions
      • Required IAM role permissions for Amazon S3
      • Configuring model preferences
    • Viewing model specifications
    • Managing user access to Textual
      • Textual organizations
      • Creating a new account in an existing organization
      • Single sign-on (SSO)
        • Viewing the list of SSO groups in Textual
        • Azure
        • GitHub
        • Google
        • Keycloak
        • Okta
      • Managing Textual users
      • Managing permissions
        • About permissions and permission sets
        • Built-in permission sets and available permissions
        • Viewing the lists of permission sets
        • Configuring custom permission sets
        • Configuring access to global permission sets
        • Setting initial access to all global permissions
    • Textual monitoring
      • Downloading a usage report
      • Tracking user access to Textual
Powered by GitBook
On this page
  • Providing the content to redact
  • Entering text
  • Using one of the samples
  • Uploading a file
  • Clearing the text
  • Selecting the handling option for an entity type
  • Defining added and excluded values
  • Displaying the configuration panel
  • Adding a rule to add or exclude values
  • Editing a rule
  • Deleting a rule
  • Creating a dataset from an uploaded file
  • Viewing and copying the request code
  • Selecting the request code type
  • Copying the request code
  • Enabling and using additional LLM processing of detected entities
  • About the LLM processing
  • Making the LLM processing available
  • Configuring the model to use for the LLM processing
  • Enabling the LLM processing for entered text

Was this helpful?

Export as PDF

Previewing Textual detection and redaction

Last updated 14 days ago

Was this helpful?

Required global permission: Use the playground on the Home page

The Tonic Textual Home page provides a tool that allows you to see how Textual detects and replaces values in plain text or an uploaded file.

It also provides a preview of the redaction configuration options, including:

  • How to replace the values for each entity type.

  • Added and excluded values for each entity type.

The Home page displays automatically when you log in to Textual. To return to the Home page from other pages, in the navigation menu, click Home.

Providing the content to redact

To provide the content to redact, you can enter text directly, or you can upload a file.

Entering text

As you enter or paste text in the Original Content text area, Textual displays the redacted version in the Results panel at the right.

Using one of the samples

Textual also provides sample text options for some common use cases. To populate the text with a sample, under Try a sample, click the sample to use.

Uploading a file

You can also redact .txt or .docx files.

To provide a file, either:

  • Drag and drop the file to the Original Content text area.

  • Click the upload prompt, then search for and select the file.

Textual processes the file and then displays the redacted version in the Results panel. The Original Content text area is removed.

Clearing the text

To clear the text, click Clear.

Selecting the handling option for an entity type

The handling option indicates how Textual replaces a detected value for an entity type. You can experiment with different handling options.

Note that the updated configuration is only used for the current redacted text. When you clear the text, Textual also clears the configuration.

The options are:

  • Redact - This is the default value. Textual replaces the value with the name of the entity type. For example, the first name John is replaced with NAME_GIVEN.

  • Synthesize - Textual replaces the value with a realistic generated value. For example, the first name John is replaced with the first name Michael. The replacement values are consistent, which means that a given value always has the same replacement. For example, MIchael is always the replacement value for John.

  • Off - Textual ignores the value and copies it as is to the Results panel.

To change the handling option for an entity type:

  1. In the Results panel, click an instance of the entity type.

  2. On the configuration panel, click the handling option to use.

Textual updates all instances of that entity type to use the selected handling option.

For example, if you change the handling option for NAME_GIVEN to Synthesize, then all instances of first names are replaced with realistic values.

Defining added and excluded values

For each entity type in entered text, you can use regular expressions to define added and excluded values.

  • Added values are values that Textual does not detect for an entity type, but that you want to include. For example, you might have values that are specific to your company or industry.

  • Excluded values are values that you do not want Textual to identify as a given entity type.

Note that the configuration is only used for the current redacted text. When you clear the text, Textual also clears the configuration.

Displaying the configuration panel

To display the configuration panel for added and excluded values, click Fine-tune Results.

The Fine-Tune Results panel displays the list of configured rules for the current text. For each rule, the list includes:

  • The entity type.

  • Whether the rule adds or excludes values.

  • The regular expression to identify the added or excluded values.

Adding a rule to add or exclude values

On the Fine-Tune Results panel, to create a rule:

  1. Click Add Rule.

  1. From the entity type dropdown list, select the entity type that the rule applies to.

  2. From the rule type dropdown list:

    • If the rule adds values, then select Include.

    • If the rule excludes values, then select Exclude.

  3. In the regular expression field, provide the regular expression to use to identify the values to add or exclude.

  4. To save the rule, click the save icon.

Editing a rule

To edit a rule:

  1. On the Fine-Tune Results panel, click the edit icon for the rule.

  2. Update the configuration.

  3. Click the save icon.

Deleting a rule

On the Fine-Tune Results panel, to delete a rule, click its delete icon.

Creating a dataset from an uploaded file

From an uploaded file, you can create a dataset that contains the file.

You can then provide additional configuration, such as added and excluded values, and download the redacted file.

To create a dataset from an uploaded file:

  1. Click Download.

  2. Click Create a Dataset.

Textual displays the dataset details for the new dataset. The dataset name is Playground Dataset <number>, where the number reflects the number of datasets that were created from the Home page.

The dataset contains the uploaded file.

Viewing and copying the request code

When Textual generates the redacted version of the text, it also generates the corresponding API request. The request includes the entity type configuration.

To view the API request code, click Show Code.

To hide the code, click Hide Code.

Selecting the request code type

On the code panel:

  • The Python tab contains the Python version of the request.

  • The cURL tab contains the cURL version of the request.

Copying the request code

To copy the currently selected version of the request code, click Copy Code.

Enabling and using additional LLM processing of detected entities

For entered text on the Home page, Textual offers an option to send the following to an OpenAI large language model (LLM):

  • The detected entity values.

  • The text that surrounds each value.

The LLM processing is not available for uploaded files.

It is also limited to text that contains 100 or fewer words.

About the LLM processing

The LLM processing is intended to improve the detection and the replacement values. The LLM:

  1. Verifies that the assigned entity type is correct.

  2. If it is not, determines the correct entity type.

  3. Standardizes an entity value that has different formats, such as Main St. versus Main Street.

  4. Generates replacement values that use the same format as the original value.

Making the LLM processing available

Configuring the model to use for the LLM processing

By default, the LLM processing uses the gpt-4o model.

Using a ChatGPT model

Using Amazon Bedrock

To use Amazon Bedrock for LLM processing, set the following environment variables:

  • AWS_ACCESS_KEY_ID - An AWS access key that is associated with an IAM user or role. The role must have read permissions for Amazon Bedrock AmazonBedrockReadOnly.

  • AWS_SECRET_ACCESS_KEY - The secret key that is associated with the access key.

  • AWS_DEFAULT_REGION - The AWS Region to send the authentication request to.

Using Azure OpenAI

  • LLM_MODEL - azure/<model name>, where <model name> is the Azure deployment name.

  • AZURE_OPENAI_API_KEY - The API key.

  • AZURE_API_BASE - The URL of your Azure OpenAI deployment.

Enabling the LLM processing for entered text

After you enter text in the Original Content panel, to enable the LLM processing, in the Results panel, click Use an LLM to perform AI synthesis.

You cannot use this option for text that contains more than 100 words.

When you clear the text, Textual reverts to the default processing.

Also, this option is only available for text that you enter directly. For an uploaded file, to do additional configuration or to download the file, you must .

To enable the LLM processing, set the ENABLE_EXPERIMENTAL_SYNTHESIS to True. Without this set to true, the LLM processing will not work.

To use a different model, configure the LLM_MODEL.

If you use a ChatGPT model, you must also provide an OpenAI key as the value of the OPENAI_API_KEY.

To use a ChapGPT model other than gpt-4o configure the LLM_MODEL. For example, to use gpt-4o-mini, set LLM_MODEL to openai/gpt-4o-mini.

LLM_MODEL - bedrock/<model name>, where <model name> is one of the .

To use Azure OpenAI, you configure the following :

AZURE_API_VERSION - The of Azure to use.

environment variable
environment variable
environment variable
environment variable
models that Amazon Bedrock supports
environment variables
API version
create a dataset from the file
Initial view of the Textual Home page
Home page with redacted text
Sample text options for the Home page
Home page with the content of an uploaded file
Selecting the handling option for an entity type
Redacted text with given name values synthesized
Fine-Tune Results panel for added and excluded values
Row to define a new rule for added or excluded values
Code to create the redaction request, including the entity type handling and added and excluded values