1 of 100

Tonic Structural

Tonic Structural User Guide

The Tonic Structural platform creates safe, realistic datasets to use in staging environments or for local development. The Structural web application and API can be used by engineers, data analysts, or security experts.

Structural connects to source databases that contain sensitive data such as personally identifiable information (PII) or protected health information (PHI). To protect that data, Structural transforms the sensitive values and then writes the transformed data to a destination location.

New to Structural? Review the Tonic Structural workflow overview. For information on how to create a Structural account and start a Structural free trial, go to Getting started with the Structural free trial.

Want to know what's in the latest Structural releases? Go to the Tonic Structural release notes.

The Structural application heading includes a feature updates icon, which displays a summary of the newest features, and includes a link to the Structural release notes.

Connect to your data

Configure and generate transformed data

Manage a self-hosted Tonic Structural instance

Need help with Structural? Contact .

About Tonic Structural

The Tonic Structural synthetic data platform combines sensitive data detection and data transformation to allow users to create safe, secure, and compliant datasets.

Common Structural use cases include creating staging and development environments, and trying out a new cloud provider without complex data agreements.

Structural allows you to reduce bug counts, shorten testing life cycles, and share data with partners, all while helping to ensure security and compliance with the latest regulations, from GDPR to CCPA.

You can use the Structural API to integrate with CI/CD pipelines or to create automated processes that ensure that the generated data is available on demand.

Structural data generation workflow

Tonic Structural data generation combines sensitive data detection and data transformation to create safe, secure, and compliant datasets.

The Structural data generation workflow involves the following steps:

You can also view this video overview of the Structural data generation workflow.

To get started, you create a workspace. When you create a workspace, you identify the type of source data, such as PostgreSQL or MySQL, and establish the connections to the source database and the destination location. The source database contains the original data that you want to synthesize. The destination location is where Structural stores the synthesized data. It might be a database, a storage location, or a container repository.
Next, you . The sensitivity scan identifies columns that contain sensitive data. These columns need to be protected by a generator.
Based on the sensitivity scan results, you configure the data generation. The configuration includes:
- The table mode controls the number of rows and columns that are copied to the destination database.
- You can make adjustments to the initial sensitivity assignments. For example, you can mark additional columns as sensitive that the initial scan did not identify as sensitive.
After you complete the configuration, you . The data generation job uses the configured table modes and generators to transform the data from the source database and write the transformed data to the destination location. You can track the job progress and view the job results.

Logging into Structural for the first time

When you go to Tonic Structural for the first time, you create an account. How you create an account depends on the type of user you are.

A new Structural user can be one of the following:

A completely new user who is starting a Structural 14-day free trial. Free trial users use Structural Cloud to explore and experiment with Structural before they decide whether to purchase it.
A new user on a self-hosted Structural instance. Self-hosted instances are installed on-premises. The customer administers the Structural users.
New users are added to existing organizations based on their email domain.

Creating and managing workspaces

Workspace configuration settings

The workspace settings for a new workspace (New Workspace view) or edited workspace (Workspace Settings tab) provide information about the workspace and its data.

Workspace identification and connection type

Every workspace includes the following settings to identify the workspace and to select the type of data connector.

Fields to identify the workspace

All workspaces have the following fields that identify the workspace:

In the Workspace name field, enter the name of the workspace.
In the Workspace description field, provide a brief description of the workspace. The description can contain up to 200 characters.
In the Tags field, provide a comma-separated list of tags to assign to the workspace. For more information on managing tags, go to .

Connection type

Under Connection Type, select the type of data connector to use for the workspace data. You cannot change the connection type on a .

The Basic and Professional licenses limit the number and type of data connectors you can use.

A Basic instance can only use one data connector type, which can be either PostgreSQL or MySQL. After you create your first workspace, any subsequent workspaces must use the same data connector type.
A Professional instance can use up two different data connector types, which can be any type other than Oracle or Db2 for LUW. After you create workspaces that use two different data connector types, any subsequent workspaces must use one of those data connector types.

If the database that you want to connect to isn't in the list, or you want to have different database types for your source and destination database, contact [email protected].

When you select a connector type, Structural updates the view to display the connection fields used for that connector type. The specific fields vary based on the .

Using secrets managers for authentication

Required license: Enterprise

Your organization might use a secrets manager to secure credentials, including database connection credentials.

You can configure a set of available secrets managers. In the workspace configuration, users can then select a secret name from a secrets manager.

Assigning tags to a workspace

Required workspace permission: Configure workspace settings

You can associate custom tags with each workspace. Tags can help to organize and provide a quick glance into the workspace configuration.

Tags are accessible to every user that has access to the workspace.

Tags are stored in the workspace JSON, and are included in the workspace export. You can also use the API to get access to tags.

Managing tags from workspace settings

You can add and edit tags in the Tags field on the New Workspace and Workspace Settings views.

To add tags, enter a comma-separated list of the tags to add.
To remove a tag, click its delete icon.

Managing tags from Workspaces view

You can also manage tags directly from Workspaces view.

Assigning tags

To add tags to a workspace that does not currently have tags:

Hover over the Tags column for the workspace.
Click Add Tags.
In the tag input field, type a comma-separated list of tags to apply.

Editing the assigned tags

To edit the assigned tags:

Click the Tags column for the workspace.
In the tag input field, to remove tag, click its delete icon.
To add tags, type a comma-separated list of the tags to add.

Managing access to workspaces

When you create a workspace, you become the owner of the workspace, and by default are assigned the built-in Manager workspace permission set for the workspace. The Manager permission set provides full access to the workspace configuration, data, and results.

With a Professional or Enterprise license, you can also assign workspace permission sets to other users and to SSO groups. You can also transfer a workspace to a different owner.

If you are granted access to any workspace permission set for a workspace, then you have access to all of the workspace management views for that workspace. However, you can only perform tasks that you have permission for in that workspace.

Workspace access is managed from the Workspaces view. You cannot assign workspace permission sets from Structural Settings view.

You can also view an .

Transferring ownership of a workspace

Required permission

Global permission: View organization users. This permission is only required for the Tonic Structural application. It is not needed when you use the Structural API.

Managing the workspace schema cache

You can configure a workspace to cache the source database schema, to reduce the number of times that Tonic Structural needs to query the source database.

Viewing the schema cache status

When schema caching is enabled for a workspace, then in the expanded version of the workspace management view header, below the workspace name, Structural shows the current status of the schema cache.

The status indicates when:

Structural is retrieving the schema information.
Structural is checking for schema updates.
Structural is refreshing the schema cache.
Structural fails to connect to the source database.

Refreshing the schema cache

When the schema cache is updated, or when Structural has detected updates to the schema, then the schema cache status includes an option to refresh the schema cache.

To refresh the schema:

Click the schema cache status.
On the panel, click Refresh Schema.

Structural starts a new schema retrieval job. To track the progress of the job, go to the .

Configuring data generation

Database View

Database View provides a complete view of your source database structure and configuration.

To display Database View, either:

On the workspace management view, in the workspace navigation bar, click Database View.
On

Displaying sample data for a column

Required workspace permission:

Source data: Preview source data

Identifying similar columns

During sensitivity scans and schema change scans, Tonic Structural identifies groups of similar columns.

To identify similar columns, Structural uses a text embedding model to calculate the semantic similarity between any two column names in the database. When a column name's semantic similarity to the name of a given column is above a specified threshold, then the column is similar to the given column.

If a column has similar columns, then the Applied Generator column contains an icon that includes the count of similar columns.

By default, the similar columns icon is hidden. To display the similar columns icon, hover over the column row.

When you assign a generator to a column, the similar columns icon for that column remains visible during your current session.

Commenting on columns

Required license: Professional or Enterprise

From Database View, you can add comments to columns. For example, you might use a comment to explain why you selected a particular generator or marked a column as sensitive or not sensitive.

Creating a new comment

If a column does not have any comments, then to add a comment:

In the Applied Generator column, click the comment icon.
In the comment field, type the comment text.
Click Comment.

Responding to existing comments

When a column has existing comments, the comment icon is green. To add comments:

Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user.
In the comment field, type the comment text.
Click Reply.

Working with document-based data

For document-based data connectors - currently MongoDB and Amazon DynamoDB - Database View and Table View are replaced by Collection View. "Collection" is the term that Structural uses to refer to MongoDB collections and DynamoDB tables.

For JSON columns in file connector and PostgreSQL workspaces, you can use Document View to view and assign generators to JSON fields.

Identifying sensitive data

Tonic Structural uses its sensitivity scan to identify source data columns that contain sensitive information. The scan ignores truncated tables.

The sensitivity scan identifies Structural's built-in sensitivity types. It also looks for custom types that you define.

You can also manually mark a column as sensitive or not sensitive.

Manually indicating whether a column is sensitive

You can also manually indicate that a column is sensitive or not sensitive.

For example, the sensitivity scan might incorrectly identify a column as sensitive. Or a column might contain data that you consider sensitive but that does not match a detected sensitivity type.

When you manually change a column from not sensitive to sensitive, Structural marks the sensitivity detection as full confidence.

For information on how to change whether a column is sensitive:

For Privacy Hub, go to .
For Database View, go to:
- For a single column,
- For multiple selected columns,
For Table View, go to .

The Structural API also provides .

Algebraic

The algebraic generator identifies the algebraic relationship between three or more numeric values and generates new values to match. At least one of the values must be a non-integer.

If a relationship cannot be found, then the generator defaults to the Categorical generator.

This generator can be linked with other Algebraic generators.

Characteristics

How to configure

To configure the generator, from the Link To dropdown list, select the columns to link this column to. You can select other columns that are assigned the Algebraic generator.

You must select at least three columns.

The column values must be numeric. At least one of the columns must contain a value other than an integer.

If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Alphanumeric String Key

Generates unique alphanumeric strings of the same length as the input.

For example, for the origin value ABC123, the output value is a six-character alphanumeric string such as D24N05.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

By default, the generator is not consistent.

If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Array Character Scramble

A version of the generator that can be used for array values.

This generator replaces letters with random other letters, and numbers with random other numbers. Punctuation and whitespace are preserved.

For example, for the following array value:

["ABC.123", 3, "last week"]

The output might be something like:

["KFR.860", 7, "sdrw mwoc"]

Character Substitution

Performs a random character replacement that preserves formatting (spaces, capitalization, and punctuation).

Characters are replaced with other characters from within the same Unicode Block. A given source character is always mapped to the same destination character. For example, M might always map to V.

For example, for the following input string:

Miami Store #162

The output would be something like:

Continuous

Generates a continuous distribution to fit the underlying data.

This generator can be linked to other Continuous generators to create multivariate distributions and can be partitioned by other columns.

Characteristics

File Name

This generator scrambles characters, but preserves formatting and keeps the file extension intact.

For example, for the following input value:

DataSummary1.pdf

The output value would look something like:

RsnoPwcsrtv5.pdf

This generator securely masks letters and numbers. There is no way to recover the original data.

Find and Replace

This generator replaces all instances of the find string with the replace string.

For example, you can indicate to replace all instances of abc with 123.

Characteristics

How to configure

To configure the generator:

In the Find field, type the string to look for in the source column value. To use a regular expression to identify the source value, check the Use Regex checkbox. If you use a regular expression, use backslash ( \ ) as the escape character.
In the Replace field, type the string to replace the matching string with.
If is enabled, then to use it for this column, in the advanced options section, toggle

FNR

The FNR generator transforms Norwegian national identity numbers. In Norwegian, the term for national identity number abbreviates to FNR.

The first six digits of an FNR reflects the person's birthdate. You can choose to preserve the birthdates from the source values in the destination values. If you do not preserve the source values, the destination values are still within the same date range as the source values.

Another digit in an FNR indicates whether the person is male or female. You can specify whether to preserve in the generated value the gender indicated in the source value.

The last digits in an FNR are a checksum value. The last digits in the destination value are not a checksum - the values are random.

Hostname

Generates random host names, based on the English language.

Characteristics

Running the Structural sensitivity scan

The Structural sensitivity scan identifies sensitive columns in source data. The scan ignores truncated tables.

When sensitivity scans run

For most data connectors, Structural runs sensitivity scans automatically based on specific events. You can also run manual sensitivity scans on demand.

On a self-hosted instance, sensitivity scans can also run automatically at the same time each day.

Event-based sensitivity scans

Structural does not run automatic sensitivity scans for Databricks and Spark SDK workspaces.

By default, Structural does not run automatic sensitivity scans for file connector workspaces. To enable automatic sensitivity scans for the file connector, set the TONIC_ENABLE_FILES_PRIVACY_SCAN_AUTORUN to true.

Structural automatically runs a sensitivity scan when you:

Create a completely new workspace and connect a data source.
Change the data connection details for the source database.
Add a file group to a file connector workspace.

A child workspace always inherits the sensitivity designations from its parent workspace.

When you copy a workspace, Structural runs a new sensitivity scan on the copy to identify sensitive columns. However, it keeps the sensitivity designation for columns that you specifically marked as sensitive or not sensitive.

Manual sensitivity scans

In addition to the automatic scans, from Privacy Hub, you can .

Scheduling daily sensitivity scans

On self-hosted instances, Structural can also run scheduled daily sensitivity scans in the background.

The daily scans only run on the 10 workspaces that had the most recent activity. Activity includes:

User-initiated updates that are included in the .
Data generation jobs.

By default, Structural runs the sensitivity scans each day at midnight.

To enable and configure the daily sensitivity scans, use the following . You can add these settings to the Environment Settings list on Structural Settings.

TONIC_ENABLE_SCHEDULED_SENSITIVITY_SCAN - Boolean to indicate whether to enable the scheduled daily sensitivity scans. The default value is true. To disable the scheduled daily scan, set this to false.
TONIC_SENSITIVITY_SCAN_HOUR - When scheduled scans are enabled, the hour at which to run the scans. The setting uses the local time zone. The value is an integer between 0 and 23, where 0 is midnight and 23 is 11:00 PM. For example, a value of 14 indicates to run the job at 2:00 PM. The default value is 0.

Configuring parallel processing for sensitivity scans

For improved performance, sensitivity scans can use parallel processing.

For relational databases such as PostgreSQL and SQL Server, to configure parallel processing, you use the TONIC_PII_SCAN_PARALLELISM_RDBMS. The default value is 4.

For document-based databases such as MongoDB, you use the environment setting TONIC_PII_SCAN_PARALLELISM_DOCUMENTDB. The default value is 1.

How Structural identifies sensitive values

The Structural sensitivity scan uses the following rules and processes to:

Identify sensitive columns.
Recommend generators for those columns. For information about applying recommended generators to columns, go to .
Indicate its confidence that an identified column is sensitive and is of the detected sensitivity type.

Note that this process cannot guarantee perfect precision and recall. We strongly recommend that a human reviews the sensitivity scan results and the broader dataset to ensure that nothing sensitive was missed.

Rule-based data type, column name, and value analysis - High, medium, or low confidence

To identify that a column contains sensitive information for a , Structural looks at the data type, column name, and column values.

This part of the sensitivity scan uses regular expression matching and dictionary lookups. It produces high, medium, or low confidence detections.

When this part of the sensitivity scan determines that a column contains sensitivity data, it:

Marks the column as sensitive
Assigns the sensitivity type to the column
Recommends the generator configuration for the identified sensitivity type. Note that if the recommended generator is not compatible with the column, then Structural discards the recommendation.

Custom sensitivity rules - Full confidence

The sensitivity scan also looks for any columns that match custom sensitivity types that you define in your custom sensitivity rules.

Custom sensitivity rules are based on the column data type and column name. For more information about custom sensitivity rules, go to .

Custom sensitivity rules always produce full confidence detections.

When a column matches a custom sensitivity rule, Structural:

Marks the column as sensitive.
Assigns the sensitivity rule name as the sensitivity type.
Recommends the generator preset from the sensitivity rule.

Model-based analysis - Medium and low confidence

To identify additional sensitive columns that might not be captured by the other parts of the scan, the sensitivity scan uses an artificial intelligence (AI) model. Note that the model is pre-trained. Structural does not use customer data to train the model, and it does not send any customer data externally.

This part of the scan produces medium or low confidence detections for built-in entity types.

The model considers the table and column name. If the combination of table and column name is similar in meaning to a sensitivity type that Structural has a recommended generator for, then Structural:

Marks the column as sensitive.
Assigns the sensitivity type to the column.
Recommends the generator configuration for that sensitivity type.

Downloading the sensitivity scan log

To download the log of the most recent sensitivity scan, either:

On the workspace management view, from the download menu, select Download Sensitivity Scan Log.
On Privacy Hub, click Reports and Logs, then select Scan Log.

The log tracks the progress of the scan.

Configuring an individual column

For an individual column in Database View, you can configure the assigned generator and determine the column sensitivity.

Displaying the generator configuration panel

From the column list, to display the generator configuration panel, in the Applied Generator column, click the generator name tag.

Indicating whether a column is sensitive

Required workspace permission: Configure column sensitivity

The Structural sensitivity scan provides an initial indication of whether a column is sensitive and, if it is sensitive:

The type of sensitive data that is in the column.
The confidence level of the sensitivity detection.

For more information, go to .

In a child workspace, you cannot configure whether a column is sensitive. A child workspace always inherits the sensitivity designations from its parent workspace.

Status column

From the Status column, to confirm or change the column sensitivity, click the Status value.

The status panel indicates whether the column is sensitive. It identifies the sensitivity type, and indicates how the sensitivity was determined - by a sensitivity scan or by a user.

Built-in sensitivity type

For a column that matches a built-in sensitivity type, the first time that you display the panel, the Sensitive data? setting displays Yes and No options for you to confirm or change the sensitivity.

To indicate that the column is sensitive, click Yes.
To indicate that the column is not sensitive, click No.

When you click Yes or No, the Yes and No options change to a simple toggle. When you click Yes, the sensitivity confidence level changes to full.

After that:

To indicate that the column is sensitive, toggle Sensitive data? to the on position.
To indicate that the column is not sensitive, toggle Sensitive data? to the off position.

Sensitivity rule match

When a column matches a sensitivity rule, the sensitivity panel indicates that the column matched a sensitivity rule.

You use the Sensitive data? toggle to indicate whether the column is actually sensitive.

No built-in sensitivity type or sensitivity rule match

When a column does not match a built-in sensitivity type or a custom sensitivity rule, the sensitivity panel indicates that column is not sensitive.

The Sensitive data? setting displays Yes and No options for you to confirm or change the sensitivity.

To indicate that the column is sensitive, click Yes.
To confirm that the column is not sensitive, click No.

When you click Yes or No, the Yes and No options change to a simple toggle.

If you click Yes:

The panel updates to indicate that a user confirmed that the column is sensitive.
The sensitivity confidence level is set to full confidence.

After that:

To indicate that the column is sensitive, toggle Sensitive data? to the on position.
To indicate that the column is not sensitive, toggle Sensitive data? to the off position.

Column configuration panel

To configure the sensitivity, you can also use the Sensitive Data toggle on the column configuration panel.

To indicate that a column is sensitive, toggle the sensitivity setting to the on position.
To indicate that the column is not sensitive, toggle the sensitivity setting to the off position.

When you change the sensitivity from the generator configuration panel, the Sensitive data? setting on the sensitivity panel also changes from the Yes and No options to the toggle.

Assigning or ignoring the recommended generator

Required workspace permission: Configure column generators

When a sensitivity scan identifies a column, Structural recommends a generator for the column. For example, when the sensitivity scan identifies a column as a first name, Structural recommends the Name generator configured to generate a first name value.

In the Assigned Generator column on Database View, columns that do not have an assigned generator, and that have a recommended generator, display the available recommendation icon.

When you click the generator dropdown, the column configuration panel includes the following information:

The sensitivity confidence level.
The recommended generator.
Sample source and destination values based on the recommended generator.

From the panel, you choose whether to assign or ignore the recommended generator for that type.

To assign the recommended generator, click Apply.
To ignore the recommendation, click Ignore. Structural clears the recommendation.

Changing the column generator configuration

Required workspace permission: Configure column generators

To change the generator that is assigned to a selected column:

Click the generator name tag for the column.
To assign a different generator to the column, from the Generator Type dropdown list, select the generator.
Configure the generator options.

To reset an assigned generator to Passthrough, which indicates to not transform the data, click Reset, then click Reset to Passthrough.

For details about the configuration options for each generator, go to the .

For more information about selecting and configuring generators and generator presets, go to .

Enabling Document View for JSON columns

Supported only for the file connector and PostgreSQL.

For a JSON column, instead of assigning a generator, you can enable Document View.

From Document View, you can view the JSON schema structure and assign generators to individual JSON fields. For more information, go to .

To enable Document View, on the column configuration panel, toggle Use Document View to the on position. Note that if you have , or enabled , then the Use Document View toggle is in the advanced options.

When Document View is enabled, the generator dropdown is replaced with the Open in Document View option.

Assigning generators to path expressions

On Collection View and Document View, Structural can automatically assign generators to fields that match a configured path expression. Each collection or JSON column has its own set of path generators.

Structural always applies the path generator to matching fields that do not have an assigned generator (are set to Passthrough).

Structural does not apply the path generator to matching fields that have a generator configuration applied directly.

However, in a child workspace, Structural does apply the path generator to matching fields that inherit their current generator configuration from the parent workspace.

Displaying the list of path generators

To display the list of path generators for the current collection or JSON column, click Path Generators.

For each path generator, the list includes:

The priority order. Structural checks the fields against the paths in the order that the paths are displayed. The first matching path wins.
The path expression to identify matching fields.
The data type filter for matching fields. You can configure a path generator to only apply to fields of a specific type or types.

Creating a path generator

To create a path generator, you can either create a completely new path generator, or start from a duplicate of an existing path generator.

Structural saves the new generator automatically when the configuration is complete. The Saved button at the bottom right indicates when the generator is saved.

Creating a completely new path generator

To create a path generator:

On the path generators panel, click Add path generator.
On the path generator details panel, in the Path Expression field, provide the path expression to use to identify matching paths. The path expression uses the syntax. Note that for a path generator, you cannot use the expression to check for a field value. For more information about the supported operators and some examples, go to . When you provide a path expression, the matching fields list displays the fields that match the expression.
You can optionally filter the matching fields based on the data type. For example, you might only want to apply a generator to text or integer fields. By default, the data type filter list is empty. The available data types are general types that map to specific data types in a given database. Under

Copying an existing path generator

You can create a path generator based on an existing one. For example, for the same path expression, you might want to assign a different generator based on the data type.

To create a new path generator based on an existing path generator:

On the path generators list, click the options menu for the path generator to copy.

Click Duplicate path generator.
On the path generator details panel, edit the configuration.

Updating a path generator

To update a path generator:

On the path generators list, click the options menu for the path generator.
Click Edit path generator.
On the path generator details panel, edit the configuration.

Structural saves the changes automatically.

For fields that were assigned a generator based on the previous configuration, but that do not match the updated path generator configuration:

If the field matches other path generators, then the next matching configuration is applied.
If the field does not match any other path generators, then the field reverts to Passthrough.

Deleting a path generator

When you delete a path generator, the generator assignment is removed from the matching fields. If a field matched more than one path generator, then the next match is used.

To delete a path configuration:

On the path generators list, click the options menu for the path configuration.
Click Delete path generator.

For fields that were assigned a generator based on the path generator:

If the field matches other path generators, then the next matching path generator is applied.
If the field does not match any other path generators, then the field reverts to Passthrough.

Supported JSONPath operators and examples

For a path generator path expression, Structural supports the following operators:

$ - Root
. - Child operator
.. - Recursive descent operator

For example, a document includes an array of objects. Each object contains name, address, and email address fields.

You can configure a path generator that assigns a generator to the address field in all of the array objects. You cannot only assign a generator to the address field in one of the array objects.

Here are some example path expressions, based on the following JSON:

Determining the priority order for the path generators

When Structural looks for matching fields, it checks the path generators in the order that they are displayed on the Path Generators panel.

For each field, it uses the first matching configuration.

To change the order of the path generators, drag and drop each configuration to the appropriate location in the list.

Identifying matching fields on Collection View and Document View

On Collection View and Document View, when the assigned generator comes from a path generator, the generator assignment is marked with an icon.

When you click the generator, the configuration panel indicates that the generator is assigned based on a path generator.

Overriding the path generator assignment

When you change the configuration for the field, the icon is removed. The override tooltip indicates that the path generator was overridden.

If you set the generator to Passthrough, then the field reverts to the path generator.

Viewing the column list

The column list on Database View contains information about the sensitivity and generator configuration for each column.

Column - Column name and type

The Column column provides general information about the columns and their content, including:

Table and column name. When you click the column name, for the column table displays.
The name of the schema that contains the table.
The data type for the column.
An indicator when the column is a primary key

The Column column also contains the option to .

Status - Protection and sensitivity status

The Status column provides information about whether the column contains sensitive data and whether it has an assigned generator.

The protection status can be one of the following values:

Protected - The column has an assigned generator.
Not Sensitive - The column is marked as not sensitive.
At Risk - The column is sensitive and does not have an assigned generator.

At the right of the Status column is a confidence indicator. For At Risk columns, the confidence indicator shows how confident Structural is that the column is sensitive and contains values of the displayed sensitivity type. Protected columns also reflect the original confidence level.

For more information about how Structural identifies values and assigns the confidence level, go to .

From the Status column, you can .

Applied Generator - Column configuration

The Applied Generator column is where you .

The generator dropdown indicates the currently assigned generator. It also indicates when an unprotected column has a recommended generator.

For foreign key columns, the generator dropdown is disabled and the column is marked as a foreign key. Foreign key columns always inherit the generator that is assigned to the primary key.

In a child workspace, when the generator configuration overrides the parent workspace, the generator dropdown displays the override icon.

The Applied Generator column also contains the option to .

Filtering the column list

To filter the column list, you can:

Use the table list to filter the displayed columns based on the table that the columns belong to.
Use the filter field to filter the columns by table or column name.
Use the Filters panel to filter the columns based on column attributes and generator configuration.

You can use column filters to quickly find columns that you want to verify or to update the configuration for.

Filter by table

To filter the column list to only include columns for specific tables, either:

.
Check the checkbox for each table to display columns for.

Filter by table or column name

To filter the column list by table or column name, in the filter field, begin to type text that is in the table or column name.

As you type, Structural filters the column list.

Using the Filters panel

The Filters panel provides access to column filters other than the table and column name.

To display the Filters panel, click Filters. The list only includes the filters that apply to the displayed data.

Searching for a filter

To search for a filter or a filter value, in the search field, start to type the value. The search looks for text in the individual settings.

For each filter, the Filters panel indicates the number of matching columns, based on the selected tables and the current filters.

Adding a filter

To add a filter, depending on the filter type, either check the checkbox or select a filter option. As you add filters, Structural applies them to the column list. Above the list, Structural displays tags for the selected filters.

Clearing the selected filters

To clear all of the currently selected filters, click Clear All.

Filters panel filters

Columns with generator recommendations

To only display detected sensitive columns for which there is a recommended generator, on the Filters panel, check Has Generator Recommendation.

At-risk columns

An at-risk column:

Is marked as sensitive.
Is included in the destination data.
Is assigned the Passthrough generator.

To only display at-risk columns, on the Filters panel, check At-Risk Column.

When you check At-Risk Column, Structural adds the following filters under Privacy Settings:

Sets the sensitivity filter to Sensitive
Sets the protection status filter to Not protected
Sets the column inclusion filter to Included

Sensitivity

You can filter the columns based on the column sensitivity.

On the Filters panel, under Privacy Settings, the sensitivity filter is by default set to All, which indicates to display both sensitive and non-sensitive columns.

To only display sensitive columns, click Sensitive.
To only display non-sensitive columns, click Not sensitive.

Note that when you check At-risk Column, Structural automatically selects Sensitive.

Protection status

You can filter the columns based on whether they have any generator other than Passthrough assigned. To filter the columns based on specific assigned generators, use the Applied Generator filter.

On the Filters panel, under Privacy Settings, the column protection filter is by default set to All, which indicates to display both protected and not protected columns.

To only display columns that have an assigned generator, click Protected.
To only display columns that do not have an assigned generator, click Not protected.

Note that when you check At-Risk Column, Structural automatically selects Not protected.

Inclusion in the destination database

You can filter the columns based on whether they are populated in the destination database. For example, if a table is truncated, then the columns in that table are not populated.

On the Filters panel, under Privacy Settings, the column inclusion filter is by default set to All, which indicates to display both included and not included columns.

To only display columns that are populated in the destination database, click Included.
To only display columns that are not populated in the destination database, click Not included.

Note that when you check At-Risk Column, Structural automatically selects Included.

Assigned generator

To only display columns that are assigned specific generators, on the Filters panel, under Applied Generator, check the checkbox for each generator to include.

The list of generators only includes generators that are assigned to the currently displayed columns and that are compatible with other applied filters.

To search for a specific generator, in the Filters search field, begin to type the generator name.

Column data type

You can filter the columns by the column data type. For example, you can only display varchar columns, or only columns that contain either numeric or integer values.

To only display columns that have specific data types, on the Filters panel, under Database Data Types, check the checkbox for each data type to include.

The list of data types only includes data types that are present in the currently displayed columns and that are compatible with other applied filters.

To search for a specific data type, in the Filters search field, begin to type the data type.

Unresolved schema changes

When the source database schema changes, you might need to update the configuration to reflect those changes. If you do not resolve the schema changes, then the data generation might fail. The data generation fails if there are unresolved conflicting changes, or if you configure Structural to always fail data generation when there are any unresolved changes.

For more information about schema changes, go to .

To only display columns that have unresolved schema changes, on the Filters panel, check Unresolved Schema Changes.

Sensitivity type

For detected sensitive columns, the sensitivity type indicates the type of data that was detected. Examples of sensitivity types include First Name, Address, and Email.

To only display columns that contain specific sensitivity types, on the Filters panel, under Sensitivity Type, check the checkbox for each sensitivity type to include.

The list of sensitivity types only includes sensitivity types that are present in the currently displayed columns.

To search for a specific sensitivity type, in the Filters search field, type the sensitivity type.

Sensitivity confidence

When the Structural sensitivity scan , it also determines how confident it is in that determination. The Status column displays the confidence level.

You can filter the columns based on the confidence level.

To only display columns that have a specific confidence level, on the Filters panel, under Sensitivity confidence, check the checkbox next to each confidence level to include.

Column nullability

You can filter the column list based on whether the column is nullable.

On the Filters panel, under Data Attributes, the nullability filter is by default set to All, which indicates to display both nullable and non-nullable columns.

To only display columns that are nullable, click Nullable.
To only display columns that are not nullable, click Non-nullable.

Column uniqueness

You can filter the column list based on whether the column must be unique.

On the Filters panel, under Data Attributes, the uniqueness filter is by default set to All, which indicates to display both unique and not unique columns.

To only display columns that must be unique, click Unique.
To only display columns that do not require uniqueness, click Not unique.

Primary or foreign keys

You can filter the column list to indicate whether to include:

Columns that are not primary or foreign keys.
Columns that are foreign keys.
Columns that are primary keys.

On the Filters panel, under Column Type:

To display columns that are neither a primary key nor a foreign key, check Non-keyed.
To display columns that are primary keys, check Primary key.
To display columns that are foreign keys, check Foreign key.

Generator overrides in a child workspace

In a child workspace, to only display columns that override the generator configuration that is in the parent workspace, on the Filters panel, check Overrides Inheritance.

Uses Structural data encryption

You can enable Structural data encryption, a configuration that allows Structural to:

Decrypt source data before applying the generator.
Encrypt generated data before writing it to the destination database.

For more information, go to .

When Structural data encryption is enabled, the generator configuration panel includes an option to use Structural data encryption.

To only display columns that are configured to use Structural data encryption, on the Filters panel, check Uses Data Encryption.

Columns with automatically assigned generators

You can filter the columns to only display those for which Structural automatically assigned the generator. Based on the , this can happen during data generation in response to changes to the source data schema.

To only display columns that have automatically applied generators, on the Filters panel, toggle Automatically Applied Generators to the on position.

Sorting the column list

By default, the column list is sorted first by table name, then by column name. The columns for each table display together. Within each table, the columns are in alphabetical order.

You can also sort the column list by column name first, then by table. Columns that have the same name display together. Those columns are sorted by the name of the table.

The button at the right of the Column column heading indicates the current sort order.

T.C indicates that the table is sorted by table, then by column
C.T indicates that the table is sorted by column, then by table

To switch the sort order, click the button.

Table modes

Each table is assigned a table mode. The table mode determines at a high level how the table is populated in the destination database.

Selecting the table mode for a table

Required workspace permission: Assign table modes

Both Database View and Table View allow you to view and update the selected table mode for a table.

For Database View, go to .

For Table View, go to .

Available table modes

De-Identify

This is the default table mode for new tables.

In this mode, Tonic Structural copies over all of the rows to the destination database.

For columns that have the generator set to Passthrough, Structural copies the original source data to the destination database.

For columns that are assigned a generator other than Passthrough, Structural uses the generator to replace the column data in the destination database.

Truncate

This mode drops all data for the table in the destination database. Sensitivity scans ignore truncated tables.

For data connectors other than Spark-based data connectors, the table schema and any constraints associated with the table are included in the destination database.

For Spark-based data connectors (, ), the table is ignored completely.

For the , file groups are treated as tables. When a file group is assigned Truncate mode, the data generation process ignores the files that are in that file group.

Any existing data in the destination database is removed. For example, if you change the table mode to Truncate after an initial data generation, the next data generation clears the table data. For Spark-based data connectors, the table is removed.

If you assign Truncate mode to a table that has a foreign key constraint, it fails during data generation. If this is a requirement, contact [email protected] for assistance.

When is enabled, the Truncate table mode does not actually truncate the destination table. Instead, it works more like Preserve Destination table mode, which preserves existing records in the destination table.

Preserve Destination

This mode preserves the data in the destination database for this table. It does not add or update any records.

This feature is primarily used for very large tables that don't need to be de-identified during subsequent runs after the data exists in the destination database.

When you assign Preserve Destination mode to a table, Structural locks the generator configuration for the table columns.

The destination database must have the same schema as the source database.

You cannot use Preserve Destination mode when you:

Enable upsert for a workspace.
Write destination data to a container repository.
Write destination data to an Ephemeral snapshot.

Incremental

Incremental mode only processes the changes that occurred to the source table since the most recent data generation or other changes in the destination. This can greatly reduce generation time for large tables that don't have a lot of changes.

For Incremental mode to work, the following conditions must be satisfied:

The table must exist in the destination database. Either Structural created the table during data generation, or the table was created and populated in some other way.
A reliable date updated column must be present. When you select Incremental mode for a table, Structural prompts you to select the date updated column to use.
The table must have a primary key.

To maximize performance, we recommend that you have an index on the date updated field.

For tables that use Incremental mode, Structural checks the source database for records that have an updated date that that is greater than the maximum date in that column in the destination database.

When identifying records to update, Structural only checks the updated date. It does not check for other updates. Records where the generator configuration is changed are not updated if they do not meet the updated date requirement.

For the identified records, Structural checks for primary key matches between the source and destination databases, then does one of the following:

If the primary key value exists in the destination database, then Structural overwrites the record in the destination database.
If the primary key value does not exist in the destination database, then Structural adds a new record to the destination database.

This mode currently only updates and adds records. Rows that are deleted from the source database remain in the destination database.

To ensure accurate incremental processing of records, we recommend that you do not directly modify the destination database. A direct modification might cause the maximum updated date in the destination database to be after the date of the last data generation. This could prevent records from being identified for incremental processing.

Incremental mode is currently supported on PostgreSQL, MySQL, and SQL Server. If you want to use this table mode with another database type, contact .

You cannot use Incremental mode when you:

Enable upsert for a workspace.
Write destination data to a container repository.
Write destination data to an Ephemeral snapshot.

Scale

In this mode, Structural generates an arbitrary number of new rows, as specified by the user, using the generators that are assigned to the table columns.

You can use linking and partitioning to create complex relationships between columns.

Structural generates primary and foreign keys that reflect the distribution (1:1 or 1:many) between the tables in the source database.

You cannot use Scale mode when you enable upsert for a workspace.

In Scale mode tables, you can only use the following generators:

Indicating whether to return an error when destination data already exists (Databricks only)

For the Databricks data connector, the table mode configuration includes an Error on Overwrite setting. The setting indicates whether to return an error when Structural attempts to write data to a destination table that already contains data. The option is not available when you write destination data to Databricks Delta tables.

To return the error, toggle the setting to the on position.

To not return the error, toggle the setting to the off position.

Applying a filter to tables

For workspaces that use following data connectors, the table mode configuration for De-Identify mode includes an option to apply a filter to the table:

Table filters provide a way to generate a smaller set of data when a data connector does not support subsetting. For more information, go to .

Configuring partitioning for the destination database

This option is only available for workspaces that use the following data connectors:

On the table mode configuration panel, you can use the Repartition or Coalesce option to indicate a number of partitions to generate.

By default, the destination database uses the same partitioning as the source database. The partition option is set to Neither.

Using the Repartition option

The Repartition option allows you to provide a specific number of partitions to generate.

To use the Repartition option:

Click Repartition.
In the field, enter the number of partitions.

Using the Coalesce option

The Coalesce option allows you to provide a maximum number of partitions to generate. If the source data has fewer partitions than the number you specify, then Structural only generates that number.

The Coalesce option should be more efficient than the Repartition option.

To use the Coalesce option:

Click Coalesce.
In the field, enter the number of partitions.

Viewing workspace jobs and job details

You can view a list of jobs for the workspace, and view details for individual jobs.

Job types

Tonic Structural runs the following types of jobs on a workspace:

Sensitivity scans - Analyze the source database to identify sensitive data.
Collection scans - Analyze the source data for a MongoDB workspace to determine the available fields in each collection, the field types, and how prevalent the fields are.
Schema retrieval jobs - Refresh the cached version of the source database schema.
Data generation, data pipeline generation, and containerized generation jobs - Generate the destination data from the source data.
Upsert data generation jobs - Generate the intermediate database from the source database.
Upsert jobs - Use data from the intermediate database to add new rows to and update changed rows in the destination database. If the migration process is enabled, then it is a step in the upsert job.
SDK table statistics jobs - Only run when you use the SDK to generate data in a Spark workspace, and the assigned generators require the statistics.

Job statuses

A job can have one of the following statuses:

Queued - The job is queued to run, but has not yet started. A job is queued for one of the following reasons:
- Another job is currently running on the same workspace. For example, you cannot run a sensitivity scan and a data generation, or multiple data generations, at the same time on the same workspace. This is true regardless of the number of workers on the instance. On Structural Cloud, there is also a limit on the number of concurrent running jobs for each organization. When that maximum is reached, a new job remains queued until a current running job completes.
- There isn't an available worker on the instance to run the job. A Structural instance with one worker can only run one job at a time. If a job from one workspace is currently running, a job from another workspace cannot start until the first job is finished.

Each of these statuses has a corresponding "with warnings" status. For example, Running with warnings, Completed with warnings. A "with warnings" status indicates that the job had at least one warning at the time of the request.

Viewing the list of jobs

Jobs view displays the list of jobs for the workspace.

Displaying Jobs view

To display Jobs view:

On the workspace management view, in the workspace navigation bar, click Jobs.
On Workspaces view, from the dropdown menu in the Name column, select Jobs.

Filtering the jobs by type

On Jobs view, you use the tabs to filter the jobs based on the job type.

The possible tabs are:

All Jobs - Always displayed. Contains all of the workspace jobs across all job types. When you first display Jobs view, All Jobs is selected.
Data Generation - Always displayed. Includes the following types of jobs:
- Data generation

Information in a jobs list

The list is always sorted by the submission date, with the most recent jobs at the top of the list.

For each job, the job list includes the following information:

Job ID - The identifier of the job. To copy the job identifier, click the icon at the left of the row.
Type - The type of job.
Status - The current status of the job, and how long ago the job reached that status. When you hover over the status, a tooltip displays the actual timestamp for the status change, and the length of time that the job ran. For queued jobs, to display a panel with information about why the job is queued, click the status value.

Filtering jobs by job status

To filter the list by the job status:

Click the filter icon in the Status column heading. The status panel displays all of the statuses that are currently in the list. For example, if there are no Queued jobs, then the Queued status is not in the list. By default, all of the statuses are included, and none of the checkboxes are checked.
To only include jobs that have specific statuses, check the checkbox next to each status to include. Checking all of the checkboxes has the same effect as unchecking all of the checkboxes.

Filtering jobs by identifier

To filter the list by the job identifier, in the filter field, provide the full identifier.

Viewing details for a selected job

For jobs other than Queued jobs, you can display details about the workspace and the job progress.

From the Jobs view, to display the details for a job, click the job row.

Workspace information

The left side of the job details view contains the workspace information.

For a sensitivity scan, the workspace information is limited to the owner, database type, and worker version.

For a data generation job, the workspace information also includes:

Whether subsetting, post-job scripts, or webhooks are used.
The number of schemas, tables, and columns in the source database.
The number of schemas, tables, and columns in the destination database.

Job Log

The Job Log tab shows the start date, start time, and duration of the job, followed by the list of job process steps.

Privacy Report

For data generation jobs, the Privacy Report tab displays the number of at-risk, protected, and not sensitive columns in the source database.

At-risk columns contain sensitive data, but still have Passthrough as the assigned generator.

Protected columns have an assigned generator other than Passthrough.

Not sensitive columns have Passthrough as the assigned generator, but do not contain sensitive data.

Copying the job identifier

The job identifier is a unique identifier for the job. To copy the job identifier, either:

From Jobs view, click the copy () icon in the leftmost column.
From the job details view, click Copy Job ID.

Canceling a job

You can cancel Queued or Running jobs.

For jobs with those statuses, the rightmost column in the job list contains a cancel icon.

To cancel the job, click the icon.

Downloading job information

For workspaces that are configured to write destination data to a container repository, the Jobs view also provides access to the generated artifacts. For more information, go to .

Job logs

Required workspace permission: Download job logs

To download diagnostic logs, you must have the Enable diagnostic logging global permission.

For all jobs, the job logs provide detailed information about the job processing. Tonic.ai support might request the job logs to help diagnose issues.

For upsert jobs where the migration process is enabled, and you configured the GET Schema Change Logs endpoint, the upsert job logs include the migration process logs.

Where to download the job logs

You can download the job logs from the Jobs view or the job details view. The download includes up to 1MB of log entries.

On the Jobs view, to download the logs for a job, click the download icon in the rightmost column.

On the job details view, to download the logs for a job, click Reports and Logs, then select Job Logs.

Downloading diagnostic logs

By default, Structural redacts sensitive values from the job logs. To help support troubleshooting, you can configure data connectors or an individual data generation job to create unredacted versions of the log files, referred to as diagnostic logs. For more information, go to .

To access diagnostic log files, you must have the Enable diagnostic logging global permission.

If you do not have the Enable diagnostic logging global permission, then you cannot download the logs for that job. The download option is disabled.

Privacy Report for data generation

Required workspace permission: View and download Privacy Report

From the job details view, you can download a Privacy Report file that provides an overview of the current protection status of the database columns based on the workspace configuration at the time that the job ran.

You can download either:

The Privacy Report .csv file, which provides details about the table columns, the column content, and the current protection configuration.
The Privacy Report PDF file, which provides charts that summarize the privacy ranking scores for the table columns. It also includes the table from the .csv file.

To display the download options, click Reports and Logs. In the menu:

To download the Privacy Report .csv file, click Privacy Report CSV.
To download the Privacy Report PDF file, click Privacy Report PDF.

For more information about the Privacy Report files and their content, go to .

Additional logs for output to a container repository

For a workspace that , the job includes the following additional logs:

Database logs - Logs for the database container that is used as the destination.
Datapacker logs - Logs for creating the OCI artifact and uploading it to an OCI registry.

To download these logs for a data generation job, on the job details view, click Reports and Logs, then select Database Logs or Datapacker Logs.

CloudWatch logs for data generation

For workspaces that are connected to Amazon Redshift or Snowflake on AWS databases, the data generation job requires multiple calls to a Lambda function. For these data generation jobs, the CloudWatch logs monitor the progress of and display errors for these Lambda function calls.

To download the CloudWatch logs for a data generation job, on the job details view, click Reports and Logs, then select CloudWatch Logs.

The CloudWatch Logs option only displays for Amazon Redshift and Snowflake on AWS data generation jobs.

Oracle SQL Loader log files

Required workspace permission: Download SqlLdr Files

For an Oracle data generation, if both of the following are true:

The data generation job ran SQL Loader (sqlldr).
sqlldr either failed or succeeded with errors.

Then to download the sqlldr log files, click Reports and Logs, then select sqlldr Logs.

Sending a log package to Tonic.ai

Required global permission: Enable diagnostic logging and uploading logs directly to Tonic.ai

The job details include an option to send a log package to Tonic.ai.

You would likely send the log package at the request of the Structural support team, to help to troubleshoot a data generation issue.

To send the package, from the Reports and Logs dropdown list, select Send logs to Tonic.ai.

Structural creates the package, then uploads it to an S3 bucket. Packages are removed from the S3 bucket automatically after 30 days.

Transformed files for file connector data generation

For a data generation from a file connector workspace that uses local files, you can download the transformed files for that job.

The download is a .zip file that contains the files for a selected file group.

On the job details view, when files are available to download, the Data available for file groups panel displays.

To download the files for a file group:

Click Download Results.
From the list, select the file group. Use the filter field to filter the list by the file group name.

Performance metrics for data generation

Required workspace permission: Download job logs

For workspaces that use the newer data generation processing, users can configure a data generation job to also . This is usually done for troubleshooting purposes.

On the job details view, to download the performance metrics for the job, click Reports and Logs, then click Performance Metrics.

Viewing a Gantt chart of a data generation job flow

This feature is currently in beta.

From the job details view, you can display a Gantt chart that shows the flow of a data generation job over time. The chart can help you to understand the different steps of a data generation job and how long it takes Structural to complete each step.

Note that this option is only available for data generation jobs that use the newer data generation process. For more information, go to . Data generation jobs that use the older process do not produce the Gantt chart.

To display the chart, click Reports and Logs, then select View Gantt.

The Job Visualization page displays the Gantt chart of the job progress.

Table View

Table View displays source or preview data for a single table. For a file connector workspace, each table corresponds to a file group.

Required workspace permission:

Source data: Preview source data
Destination data: Preview destination data

If you do not have either of these permissions, then you cannot display Table View.

To display Table View:

On the workspace management view, click Table View.
On Workspaces view, from the dropdown menu in the Name column, select Table View.
From Database View, either click the arrow icon for the table, or click a row in the table.

From Table View, you can view and update the table and column configuration.

Selecting and configuring tables

Selecting the table to view

When you display Table View from Database View, it displays the data for the selected table.

When you display Table View from the workspace management view or Workspaces view, it displays the most recently displayed table.

If Table View was never displayed before, then it displays the first table in the workspace.

To change the selected table, from the Table dropdown list, select the table to view.

Selecting the table mode

Required workspace permission: Assign table modes

To change the table mode that is assigned to the table:

Click the current table mode.
On the table mode panel, from the table mode dropdown list, select the new table mode.

When you change the table mode, Tonic Structural updates the preview data as needed. For example, if you change the table mode to Truncate, then the preview data is empty.

For a , the table mode selection panel indicates whether the selected table mode is inherited from the parent workspace.

If the child workspace currently overrides the parent workspace configuration, then to reset the table mode to the table that is assigned in the parent workspace, click Reset.

Viewing the generator configuration summary

The Model section of Table View displays the configured generators for the table columns.

The header for each Model entry is the column name.

Linked columns share an entry. The heading is a comma-separated list of the linked columns.

Each entry contains the following information:

The column and generator, in the format Column Name >> Generator Name. For example, First_Name >> Name indicates that the First_Name column has the Name generator applied. For linked columns, there is a Column Name >> Generator Name entry for each column.
The selected configuration options for the generator.

For a , each Model entry indicates whether the configuration overrides the parent configuration. For configurations that override the parent, to remove the overrides and restore the inheritance, click Reset.

The Model entry also indicates when is enabled for the column.

To remove the generator from a column, click the delete icon.

Changing the column data display

Toggling between source and preview data

The Preview toggle at the top right of Table View allows you to choose whether to display original source data or the transformed data. You can switch back and forth to understand exactly how Structural transforms the data based on the table and column configuration.

By default, the Preview toggle is in the on position, and the displayed data reflects the selected table mode and the assigned generators. For tables that use Truncate mode, the preview data is empty. Truncated tables do not have data in the destination database.

To display the original source data, toggle Preview to the off position.

Note that for , you cannot preview the destination data from Table View. You must preview the data from Document View.

Using a query to filter the source data

You can provide a query to filter the source data. The query is always against the source data, not the preview data, regardless of whether the Preview toggle is off or on.

For example, you configure a first name field to use the Name generator and enable consistency. You can then query the source data for a specific first name value to check that the preview data uses the same destination value for all of those records.

To apply a query to the source data:

Click the query filter icon, located between the table name and the table mode.
On the Table Filter dialog, provide the WHERE clause for the query.
To apply the query, click Apply.

To clear an applied query, on the Table Filter dialog, click Clear.

If no filter is applied, then the query filter icon has a white background.

If a valid filter is applied, then the query filter icon has a gray background.

If the provided WHERE clause is not valid, then the query filter icon has a red background.

Navigating to a specific column

When the width of the table is more than 1.5 times the visible display area, then the Jump to column option displays.

To bring a specific column into view:

At the top right of Table View, click Jump to column. The list of table columns is displayed.
To filter the list, in the filter field, begin to type text that is in the column name.

Click the column. Structural scrolls the columns to make the selected column visible.

Information in the column headings

In addition to the column name, the column heading provides details about the column type and protection status. It also provides access to change the column configuration.

Primary and foreign key indicators

The column heading indicates when a column is either a primary key or a foreign key.

Protection status

The column heading indicates the column protection status:

At risk columns are sensitive and do not have an assigned generator.
Protected columns have an assigned generator.
Not sensitive columns are not sensitive and do not have an assigned generator.

Sensitivity confidence

The sensitivity confidence indicator indicates the confidence in the detection.

For sensitive columns that Structural detected, the confidence level can be high, medium, or low.

For custom sensitivity rule matches or columns that you manually marked as sensitive, the confidence level is full confidence.

For more information about how Structural identifies values and assigns the confidence level, go to .

Column data type

The column heading displays the type of data that the column contains.

Child workspace overrides

Required license: Enterprise

In a , when a column overrides the parent configuration, an Overriding label displays in the column heading.

To filter Table View to only display columns with overrides, toggle Show Overrides Only to the on position.

Configuring a column

Applying or ignoring a recommended generator

Required workspace permission: Configure column generators

For unprotected columns that have a recommended generator, the column heading displays the available recommendation icon.

When you click the dropdown, the column configuration panel includes the following information:

The sensitivity confidence level
The recommended generator
Sample source and destination values based on the recommended generator

From the panel, you can choose whether to assign or ignore the recommended generator for that type.

To assign the recommended generator, click Apply.
To ignore the recommendation, click Ignore. Structural clears the recommendation.

Changing the column generator configuration

Required workspace permission: Configure column generators

To assign a generator to a column that does not have an assigned generator, or to change the current configuration, click the dropdown in the column heading.

On the generator configuration panel, from the generator type dropdown list, select the generator to assign to the column.

Structural displays the available configuration options for the selected generator. For details about the configuration options for each generator, go to the .

To remove the selected generator or generator preset, and reset the generator to Passthrough, click Reset, then click Reset to Passthrough.

For more information about selecting and configuring generators and generator presets, go to .

Indicating whether a column is sensitive

Required workspace permission: Configure column sensitivity

On the column configuration panel, the Sensitive Data toggle indicates whether the column is marked as sensitive. The initial configuration is based on the sensitivity scan.

To mark a column as sensitive, toggle the setting to the on position.
To mark a column as not sensitive, toggle the setting to the off position.

In a , you cannot configure whether a column is sensitive. A child workspace always inherits the sensitivity designation from its parent workspace.

When you copy a workspace, Structural performs a new sensitivity scan on the copy. It does not copy the sensitivity designations from the original workspace.

Enabling Document View for JSON columns

Supported only for the file connector and PostgreSQL.

For a JSON column, instead of assigning a generator, you can enable Document View.

From Document View, you can view the JSON schema structure and assign generators to individual JSON fields. For more information, go to .

When Document View is enabled, the generator dropdown is replaced with the Open in Document View option.

Using Collection View

For MongoDB and Amazon DynamoDB, Collection View replaces Database View and Table View. From Collection View, you can view the fields in a selected collection. You can then assign a collection mode to the collection, and assign generators to fields.

Selecting the collection to view

From the Collection dropdown list, select the collection to view.

Assigning a collection mode to the collection

Collection mode is the term used for table mode. The collection mode determines at the collection level how Structural uses the collection data to generate the destination database.

Available collection modes

By default, the collection mode is De-Identify. In this mode, Structural uses the assigned generators to transform the source database into the destination database.

For and , the only other options are Truncate and Preserve Destination.

Truncate means that only the collection structure is included in the destination database. The collection has no data in the destination database.
Preserve Destination means that Tonic does not change the data that is currently in the destination database.

Assigning the collection mode

Required workspace permission: Assign table modes

To assign the collection mode:

Click the Collection Mode dropdown list.
On the panel, click the current collection mode.
From the drop-down list, select the mode to use.

Selecting the type of view

You can view a collection either as a hybrid document or as single documents. From the View dropdown list, select the view to use.

Hybrid document view

The default view is Hybrid Document. For the hybrid document view, the key list reflects all of the permutations of every field from every document. For example, a field might sometimes be a datetime value and sometimes a string. Hybrid document view lists both types.

Single document view

Single Document view displays a single document at a time. You can then page through up to 100 documents. Single Document view displays the structure for each document.

Information on the field list

For each field, Collection View always displays:

The field name and type.
For fields that you configured as primary or foreign keys, a key icon.
The assigned generator.

For the hybrid document view, there is also a Field Freq column. Field Freq shows the percentage of documents that contain that permutation of field and type.

For example, a field might be Null 33% of the time and contain a numeric value 67% of the time. Or a field value might be an Int32 value 3% of the time and an Int64 value 6% of the time. The percentages apply to the first 100 documents.

Toggling between source and preview data

Required workspace permission:

Source data: Preview source data
Destination data: Preview destination data

The Preview toggle at the top right of Collection View allows you to choose whether to display original source data or the transformed data. You can switch back and forth to determine exactly how Tonic Structural transforms the data based on the collection and field configuration.

By default, the Preview toggle is in the on position, and the displayed data reflects the selected collection mode and the assigned generators. For collections that use Truncate mode, the preview data is empty. Truncated collections do not have data in the destination database.

To display the original source data, toggle Preview to the off position.

Filtering collection fields

In the single document view, you can filter the fields by either the field name or the field value.

In the hybrid document view, you can filter the fields based on either the field name or field properties.

Filtering single document view by field name or value

You can filter single document view to only display fields that have specific text in either the field name or the field value.

To filter by value, toggle Search by Value to the on position.

After you select the filter type, in the search field, type text that is in the field name or value. As you type, Structural filters the list to only include fields that contain the filter text.

Filtering hybrid view by field name

To filter hybrid view by field name, in the search field, begin to type text that is in the field name. As you type, Structural filters the list to only include fields with names that include the filter text.

Filtering hybrid view by field properties

From the hybrid document view, you can filter the fields based on field properties.

To display the Filters panel, click Filters.

Searching for a filter

To search for a filter or a filter value, in the search field, start to type the value. The search looks for text in the individual settings.

Adding a filter

To add a filter, depending on the filter type, either check the checkbox or select a filter option. As you add filters, Structural applies them to the field list.

Above the list, Structural displays tags for the selected filters.

Clearing the selected filters

To clear all of the currently selected filters, click Clear All.

Filters panel filters

The Filters panel in hybrid view includes the following fields.

At-risk fields

An at-risk field:

Is marked as sensitive
Is assigned the Passthrough generator.

To only display at-risk fields, on the Filters panel, check At-Risk Field.

When you check At-Risk Field, Structural adds the following filters under Privacy Settings:

Sets the sensitivity filter to Sensitive.
Sets the protection status filter to Not protected.

Sensitivity

You can filter the fields based on the field sensitivity.

On the Filters panel, under Privacy Settings, the sensitivity filter is by default set to All, which indicates to display both sensitive and non-sensitive fields.

To only display sensitive fields, click Sensitive.
To only display non-sensitive fields, click Not sensitive.

Note that when you check At-risk Field, Structural automatically selects Sensitive.

Protection status

You can filter the fields based on whether they have any generator other than Passthrough assigned.

On the Filters panel, under Privacy Settings, the field protection filter is by default set to All, which indicates to display both protected and not protected fields.

To only display fields that have an assigned generator, click Protected.
To only display fields that do not have an assigned generator, click Not protected.

Note that when you check At-Risk Field, Structural automatically selects Not protected.

Recommended generators

When Structural detects that a field is sensitive, it can also determine a recommended generator.

For example, when it detects a name value, it also recommends the Name generator.

You can filter the fields to display the fields that have recommended generators.

On the Filters panel, under Recommended Generators, check the checkbox next to the recommended generator for which to display the fields that have that recommendation.

Field data type

You can filter the fields by the field data type. For example, you might only display columns that contain either numeric or integer values.

To only display fields that have specific data types, on the Filters panel, under Database Data Types, check the checkbox for each data type to include.

The list of data types only includes data types that are present in the currently displayed fields and that are compatible with other applied filters.

To search for a specific data type, in the Filters search field, begin to type the data type.

Unresolved schema changes

For more information about schema changes, go to .

To only display fields that have unresolved schema changes, on the Filters panel, check Unresolved Schema Changes.

Sensitivity type

For detected sensitive fields, the sensitivity type indicates the type of data that was detected. Examples of sensitivity types include First Name, Address, and Email.

To only display fields that contain specific sensitivity types, on the Filters panel, under Sensitivity Type, check the checkbox for each sensitivity type to include.

The list of sensitivity types only includes sensitivity types that are present in the currently displayed fields.

To search for a specific sensitivity type, in the Filters search field, type the sensitivity type.

Sensitivity confidence

When the Structural sensitivity scan identifies a value as belonging to a sensitivity type, it also determines how confident it is in that determination.

You can filter the columns based on the confidence level.

To only display columns that have a specific confidence level, on the Filters panel, under Sensitivity confidence, check the checkbox next to each confidence level to include.

Primary or foreign keys

You can filter the column list to indicate whether to include:

Columns that are not primary or foreign keys.
Columns that are foreign keys.
Columns that are primary keys.

On the Filters panel, under Field Type:

To display fields that are neither a primary key nor a foreign key, check Non-keyed.
To display fields that are primary keys, check Primary key.
To display fields that are foreign keys, check Foreign key.

Commenting on fields

Required license: Professional or Enterprise

You can add comments to fields. For example, you might use a comment to explain why you selected a particular generator or marked a field as sensitive or not sensitive.

Adding a new comment

If a field does not have any comments, then to add a comment:

Click the comment icon.
In the comment field, type the comment text.
Click Comment.

Replying to an existing comment

When a field has existing comments, the comment icon is green. To add comments:

Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user and timestamp.
In the comment field, type the comment text.
Click Reply.

Indicating whether a field is sensitive

Required workspace permission: Configure column sensitivity

On the field configuration panel, the sensitivity toggle at the top right indicates whether the field is marked as sensitive.

To mark a field as sensitive, toggle the setting to the Sensitive position.

To mark a field as not sensitive, toggle the setting to the Not Sensitive position.

Assigning a generator to a field and type

Required workspace permission: Configure column generators

You can assign a generator to each combination of field and type. For example, depending on the document, the data type for a field might be either string or integer. You can indicate to use the Character Scramble generator when the field type is a string and the Random Integer generator when the field type is integer.

In hybrid document view, the Null type reflects when the column value is Null. You do not assign a generator to it.

To assign a generator:

Click the generator value for the field.
On the configuration panel, from the Generator Type dropdown list, select the generator.
Configure the generator options. For details about the available configuration options for each generator, go to the .

Assigning generators to fields that match JSONPath expressions

In addition to assigning generators to individual fields, you can assign generators to generic paths. The paths use JSONPath syntax.

For more information, go to .

Disabling examples for sparse collections

By default, Structural retrieves 100 documents. It then uses the data in these documents to populate example values in the hybrid document.

For sparsely populated collections, where less common fields are not present in those 100 documents, Structural retrieves extra documents until it has example values for all fields. For very sparsely populated collections, this might cause the collection view to load slowly, because it must retrieve many documents.

To disable examples for sparse collections, set the TONIC_MONGO_DISABLE_EXTRA_EXAMPLES to true. You can add this setting manually to the Environment Settings list on Structural Settings.

Note that this setting applies to both MongoDB and Amazon DynamoDB.

When this setting is true, fields that do not have a retrieved value use a dummy default value that is based on the data type.

Getting started with the Structural free trial

If you are a user who wants to set up an account in an existing Tonic Structural Cloud or self-hosted organization, go to Creating a new account in an existing organization.

About the Structural free trial

The Structural 14-day free trial allows you to explore and experiment in Structural Cloud before you decide whether to purchase Structural.

When you sign up for a free trial, Structural automatically creates a sample workspace for you to use. You can also create a workspace that uses your own database or files.

The free trial provides tools to introduce you to Structural and to guide you through configuring and completing a data generation.

Structural tracks and displays the amount of time remaining in your free trial. You can request a demonstration and contact support.

When the free trial period ends, you can continue to use Structural to configure workspaces. You can no longer generate data or train models. Contact Tonic.ai to discuss purchasing a Structural license, or select the option to .

Signing up for the free trial

To start a new free trial of Structural:

Go to .
Click Create Account.

On the Create your account dialog, to create an account, either:

To use a corporate Google email address to create the account, click Create account using Google.
To create a new Structural account:
1. Enter your email address. You cannot use a public email address for a free trial account.

Structural sends an activation link to your email address.

After you activate your account and log in, Structural next prompts you to select the use case that best matches why you are exploring Structural.

If none of the provided use cases fits, use the Other option to tell us about your use case.

After you select a use case, click Next. The Create Your Workspace panel displays.

Determining whether to use your own data

When you sign up for a free trial, Structural provides access to a sample PostgreSQL workspace that you can use to explore how to configure and run data generation.

You can also choose to create a workspace that uses your own data, either from local files or from a database.

If you do connect to your own data, then you must allowlist the Structural static IP addresses. For more information, go to .

On the Create your workspace panel:

To use the sample workspace, click Use a sample workspace, then click Next. Structural displays , which summarizes the protection status for the source data. It also displays the and the .
To create a workspace that uses local files as the source data, click Upload Files, then click Next. Go to .
To create a new workspace that uses your own data, click Bring your own data

Uploading files

The Upload files option creates a local files workspace. The source data consists of groups of files selected from a local file system. The files in a file group must have the same type and structure. Each file group becomes a "table" in the source data.

For other workspaces that you create during the free trial, you can also create a file connector workspace that uses files from cloud storage ( Amazon S3 or Google Cloud Storage).

After you select Upload files and click Next, you are prompted to provide a name for the workspace.

In the field provided, enter the name to use for the workspace, then click Next.

Structural displays the File Groups view, where you can .

It also displays the with links to resources to help you get started.

After you create at least one file group, you can start to use the other Structural features and functions.

Connecting to a database

If you connect to your own data, then you must allowlist the Structural static IP addresses. For more information, go to .

Provide a name for your workspace

If you choose to create a workspace with your own data, then the first step is to provide a name for the workspace.

In the field provided, enter the name to use for your first workspace, then click Next.

The Invite others to Tonic panel displays.

Invite other users to Structural and your workspace

Under Invite others to Tonic, you can optionally invite other users with the same corporate email domain to start their own Structural free trial. The users that you invite are able to view and edit your workspace.

For example, you might want to invite other users if you don't have access to the connection information for the source data. You can invite a user who does have access. They can then update the workspace configuration to add the connection details.

To continue without inviting other users, click Skip this step.

To invite users:

For each user to invite, enter the email address, then press Enter. The email addresses must have the same corporate email domain as your email address.
After you create the list of users to invite, click Next.

The Add source data connection view displays.

Supported data connectors for free trial workspaces

The final step in the workspace creation is to provide the source data to use for your workspace.

Structural provides data connectors that allow you to connect to an existing database. Each data connector allows you to connect to a specific type of database. Structural supports several types of application databases, data warehouses, and Spark data solutions.

For the first workspace that you create using the free trial wizard, you can choose:

For subsequent workspaces that you create from Workspaces view, you can also choose , , and .

Selecting the data connector

To connect to an existing database, on the Add source data connection panel, click the data connector to use, then click Add connection details.

The panel also includes a Local files option, which creates a local files file connector workspace, the same as the Upload files option.

Use the connection details fields to provide the connection information for your source data. The specific fields depend on the type of data connector that you select.

After you provide the connection details, to test the connection, click Test Connection.

To save your workspace, click Save.

Structural displays , which summarizes the protection status for the source data.

It also displays the with links to resources to help you get started.

Free trial resources

The Structural free trial includes a couple of resources to introduce you to Structural and to guide you through the tasks for your first data generation.

Getting Started Guide panel

The Getting Started Guide panel provides access to Structural information and support resources.

The Getting Started Guide panel displays automatically when you first start the free trial. To display the Getting Started Guide panel manually, in the Structural heading, click Getting Started.

The Getting Started Guide panel provides links to Structural instructional videos and this Structural documentation. It also contains links to request a Structural demo, contact Tonic.ai support, and purchase a Structural Cloud pay-as-you-go subscription.

Quick start checklist

For each free trial workspace, Structural provides access to a workspace checklist.

The checklist displays at the bottom left of the workspace management view. It displays automatically when you display the workspace management view. To hide the checklist, click the minimize icon. To display the checklist again, click the checklist icon.

The checklist provides a basic list of tasks to perform in order to complete a Structural data generation.

Each checklist task is linked to the Structural location where you can complete that task. Structural automatically detects and marks when a task is completed.

The checklist tasks are slightly different based on the type of workspace.

Checklist for database-based workspaces

For workspaces that are connected to a database, including the sample PostgreSQL workspace and workspaces that you connect to your own data, the checklist contains:

Connect a source database - Set the connection to the source database. In most cases, you set the source connection when you create the workspace. When you click this step, Structural navigates to the Source Settings section of the workspace details view.
Connect to destination database - Set the location where Structural writes the transformed data. When you click this step, Structural navigates to the Destination Settings section of the workspace details view.
Apply generators to modify dataset - Configure how Structural transforms at least one column in the source data. When you click this step:

Checklist for local file workspaces

For workspaces that use data from local files, the checklist contains:

Create a file group - Create a file group with files that you upload from a local file system. Each file group becomes a table in the workspace. When you click this step, Structural navigates to the File Groups view for the workspace.
Apply generators to modify dataset - Configure how Structural transforms at least one column in the source files. When you click this step:
- If there are available generator recommendations, then Structural navigates to Privacy Hub

Checklist for cloud storage file workspaces

For workspaces that use data from files in cloud storage (Amazon S3 or Google Cloud Storage), the checklist contains:

Configure output location - Configure the cloud storage location where Structural writes the transformed files. When you click this step, Structural navigates to the Output location section of the workspace details view.
Create a file group - Create a file group that contains files selected from cloud storage. When you click this step, Structural navigates to the File Groups view for the workspace.
Apply generators to modify dataset - Configure how Structural transforms at least one column in the source data. When you click this step:

Next step hints

In addition to the workspace checklists, Structural uses next step hints to help guide you through the workspace configuration and data generation.

When a next step hint is available, it displays as an animated marker next to the suggested next action.

When you hover over the highlighted action, Structural displays a help text popup that explains the recommended action.

When you click the highlighted action, the hint is removed, and the next hint is displayed.

Creating a file group

For a file connector workspace, to identify the source data, you create file groups. A file group is a set of files of the same type and with the same structure. Each file group becomes a table in the workspace. For CSV files, each column becomes a table column. For XML and JSON file groups, the table contains a single XML or JSON column.

On the File Groups view, click Create File Group.

Uploading local files

For a file connector workspace that uses local files, you can either drag and drop files from your local file system to the file group, or you can search for and select files to add. For more information, go to .

Selecting files from cloud storage

For a file connector workspace that uses cloud storage, you select the files to include in the file group. For more information, go to .

Configuring file delimiters and settings

For files that contain CSV content, you configure the delimiters and other file settings. For more information, go to .

Assigning a generator

To get value out of the data generation process, you assign generators to the data columns.

A generator indicates how to transform the data in a column. For example, for a column that contains a name value, you might assign the Name generator, which indicates how to generate a replacement name in the generation output.

Applying all recommendations

For sensitive columns that Structural detects, Structural can also provide a recommended generator configuration.

When there are recommendations available, Privacy Hub displays a link to review all of the recommendations.

The Recommended Generators by Sensitivity Type panel displays a list of sensitive columns that Structural detected, along with the suggested generators to apply.

After reviewing, to apply all of the suggested generators, click Apply All. For more information about using this panel, go to .

Selecting a generator

You can also choose to apply an individual generator manually. You can do this from , , or .

To display Database View, on the workspace management view, click Database View.

On Database View, in the column list, the Applied Generator column lists the currently assigned generator for each column. For a new workspace, the columns are all assigned the Passthrough generator. The Passthrough generator simply passes the source value through to the destination data without masking it.

Click a column that is marked as Passthrough, and that is not marked as sensitive. For example, in the sample workspace, the customers.Last_Transaction column. The column configuration panel displays. To select a generator, click the generator dropdown. The list contains generators that can be assigned to the column based on the column data type. For customers.Last_Transaction, the Timestamp Shift generator is a good option.

Assigning a recommended generator

For Passthrough columns that Structural identified as containing sensitive data, the Applied Generator column displays an icon to indicate that there is a recommended generator.

In Database View, click one of those columns. For example, in the sample workspace, the customers.email column is marked as containing an email address.

For customers.Email, click the generator dropdown. Instead of the column configuration panel, there is a panel that indicates the recommended generator. For customers.Email, the recommended generator is Email. To assign the Email generator, click Apply. The column configuration panel displays with the generator assigned.

Configuring the destination location

To run a data generation, Structural must have a destination for the transformed data.

For a local files workspace, Structural saves the transformed files to the application database.

For workspaces that use data from a database, and for workspaces that use cloud storage files, you configure where Structural writes the output data.

Available output options

The destination location for data generation output can be one of the following:

For database-based data connectors, you can write the transformed data to a destination database.
For some Structural data connectors, Structural can .
For file connector workspaces that transform files from cloud storage (Amazon S3 or Google Cloud Storage), you .

Displaying the current destination configuration

To display the destination configuration for the workspace:

Click the Workspace Settings tab.
Scroll to the Destination Settings section or, for a file connector workspace that uses cloud storage files, scroll to the Output location section.

Confirming or changing the destination configuration

Destination database

To write the data to a destination database, click Database Server. Structural displays the configuration fields for the destination database.

For information on how to configure the destination information for a specific data connector, go to the workspace configuration information for that data connector. The contains a list of the available data connectors, and provides a link to the documentation for each data connector.

Container repository

To write the data to a data volume in a container repository, click Container Repository. Structural displays the configuration fields to select a base image and provide the details about the repository.

For more information, go to .

Cloud storage files output location

For a file connector workspace that uses files from cloud storage (Amazon S3 or Google Cloud Storage), you configure the cloud storage output location where Structural writes the transformed files. The configuration includes the required credentials to use.

For more information, go to .

Running data generation

After you complete the workspace and generator configuration, you can run your first data generation.

The data generation process uses the assigned generators to transform the source data. It writes the transformed data to the configured destination location.

For a local files workspace, it writes the files to the Structural application database.

Starting the generation

The Generate Data option is at the top right of the Tonic heading.

When you click Generate Data, Structural displays the Confirm Generation panel.

The Confirm Generation panel provides access to the current destination configuration, along with other advanced generation options such as subsetting and upsert.

It also indicates if there are any issues that prevent you from starting the data generation. For example, if the workspace does not have a configured destination, then Structural cannot run the data generation.

To start the data generation, click Run Generation. For more information about running data generation, go to .

Viewing the job details

To view the job status and details:

Click Jobs.
In the list, click the data generation job.

Next steps for free trial users

The first time that you complete all of the steps in a checklist, Structural displays a panel with options to chat with our sales team, schedule a demo, or purchase a subscription.

You can also continue to get to know Structural and experiment with other Structural features such as or using to mask more complex values such as JSON or XML.

If your free trial has expired, to get an extension, you can reach out to us using either the in-app chat or an email message.

Privacy Hub

About Privacy Hub

Privacy Hub tracks the current protection status of source data columns based on:

Column sensitivity, either from the most recent sensitivity scan or from manual assignments
Assigned
Assigned

To display Privacy Hub, either:

On the workspace management view, in the workspace navigation bar, click Privacy Hub.
On Workspaces view, click the workspace name.

From Privacy Hub, you can:

Review and apply the recommended generators for all detected sensitive columns
View the current protection status of columns
Manually mark columns as sensitive or not sensitive

You can also track the history of changes to column sensitivity and the assigned column generators. For more information, go to .

Viewing the count of detected sensitive columns that are not protected

The sensitivity scan detects specific types of sensitive data.

If your workspace contains any columns that the sensitivity scan identified, and for which you have not either:

Assigned a generator
Marked as not sensitive

Then Tonic Structural displays a Sensitivity Recommendations banner that contains a count of those columns.

The count only includes sensitive columns that the sensitivity scan detects. If you manually mark a column as sensitive, it is not included in the list.

On the banner, the Review Recommendations option allows you to review the detected columns and the recommended generators for each detected sensitive data type.

You can then apply the recommended generators or ignore the recommendations. When you ignore a recommendation, you either:

Indicate to remove the generator recommendation for the column.
Indicate that the column data is not sensitive.

For more information, go to .

Viewing the protection status for each column

The protection status panels at the top of Privacy Hub provide an overview of the current protection status of the columns in the source data.

Each panel displays:

The number of columns that are in that category.
The estimated percentage of columns that are in that category.

Note that for a , the protection status displays a separate box for each combination of JSON path and data type.

From each panel, you can .

The column counts do not include columns that do not have data in the destination database. For example, if a table is assigned Truncate table mode, then Privacy Hub ignores the columns in that table.

The information on these panels updates automatically as you change whether columns are sensitive and assign generators to columns.

At-Risk Columns

The At-Risk Columns panel reflects columns that:

Are populated in the destination database.
Are marked as sensitive.
Have the generator set to Passthrough, which indicates that Structural does not perform any transformation on the data.

For each column, the At-Risk Columns panel also indicates the sensitivity confidence, from full confidence (completely red) to low confidence (a small percentage of red).

The goal is to have 0 at-risk columns.

When you click Open in Database View, you navigate to . The column list is filtered to show columns that are at risk.

Protected Columns

The Protected Columns panel reflects columns that:

Are populated in the destination database.
Are assigned a generator other than Passthrough.

It includes both sensitive and non-sensitive columns.

Note that a column is considered protected based solely on the assigned generator. Some more complex generators, such as JSON Mask or Conditional, allow you to apply different generators to specific portions of a value or based on a specific condition. However, the protection status does not reflect these sub-generators. An applied sub-generator could be Passthrough.

When you click Open in Database View, you navigate to . The column list is filtered to show all included columns that are protected.

Not Sensitive Columns

The Not Sensitive Columns panel reflects columns that:

Are populated in the destination database.
Are marked as not sensitive.
Have the generator set to Passthrough.

When you click Open in Database View, you navigate to . The column list is filtered to show included columns that are not sensitive and are not protected.

Viewing the protection status for each table

The Database Tables list shows the protection status for each table in the source database. You can view the number of columns that have each protection status, and update the column configuration.

The list does not include tables where the table mode is Truncate or Preserve Destination. Truncated tables are not populated in the destination database. For Preserve Destination tables, the existing data in the destination database does not change.

Information in the list

For each table, Database Tables provides the following information:

Name - The table name. For a workspace, each table corresponds to a file group. Each is also in a separate row. For JSON columns, the Name column displays both the table name and the column name.
Not Sensitive - The number of not sensitive columns in the table. Not sensitive columns are not marked as sensitive and have Passthrough as the generator. When you click the value, you navigate to , filtered to display the not sensitive columns for the table.

Filtering the list

You can filter the Database Tables list either by the table name or by the schema.

Filtering by table name

To filter the list by table name, in the filter field, begin to type text that is in the table name. As you type, Structural updates the list to only display matching tables.

Filtering by schema

To filter the list to only include tables that belong to a specific schema:

Click Filter by Schema.
From the schema dropdown list, select the schema.

When you select a schema, Structural adds it to the filter field.

Sorting the list

You can sort the Database Tables list by any column except for the Privacy Status column.

To sort by a column, click the column heading. To reverse the sort order, click the heading again.

Managing columns from the table list

The Privacy Status column in the Database Tables list indicates the protection status of the columns in the table.

This column provides the same as the protection status panels at the top of Privacy Hub, but is limited to the columns in a specific table.

Viewing and configuring columns

Navigating through columns and viewing column details

Each protection status panel displays a series of boxes to represent the columns that apply to that status. For example, if the source data contains four columns that are at-risk, then the At-Risk Columns panel displays four boxes, one for each column.

The Privacy Status column in the Database Tables list displays the same set of boxes for the columns in an individual table.

If the number of columns is too large to fit, then the last box shows the number of additional columns that apply. For example, if there are 15 columns that don't fit, then the last box is labeled +15.

When you hover over a box, the column name displays in a tooltip.

When you click a box, the details panel for that column displays.

When you click the box for remaining columns, the details panel for the first column in the remaining columns displays.

You can use the next and previous icons at the bottom right of the details panel to display the details for the next or previous column.

The column details panel opens to the settings view. The settings view contains the following information:

The table and column name.
Whether the column is flagged as sensitive.
The type of sensitive data that the column contains.

Indicating whether a column is sensitive

Required workspace permission: Configure column sensitivity

From the settings view of the column details, you can configure the column sensitivity.

You cannot change the sensitivity of columns in a child workspace. A child workspace always inherits the sensitivity from its parent workspace. For more information, go to .

As you change the column sensitivity, Structural updates the protection status panels.

To change whether the column is sensitive, toggle the Sensitive option. The column is moved if needed to reflect its new status. However, you remain on the current panel.

For example, from the At-Risk Columns panel, you change a column to be not sensitive. The column is moved to the Not Sensitive Columns panel. When you click the next or previous icons, you view the details for the next or previous column on the At-Risk Columns panel.

Selecting and configuring a generator for the column

Required workspace permission: Configure column generators

From the column details, you can assign and configure the column generator.

When you change the column generator, Structural updates the protection status panels.

If the column generator was previously Passthrough, then the column is moved to the Protected Columns panel. However, you remain on the current panel. For example, you assign a generator to a column that is on the At-Risk Columns panel. The column is moved to the Protected Columns panel, but when you click the next or previous icons, you view the details for the next or previous column on the At-Risk Columns panel.

Selecting the generator

For sensitive columns that are not protected, Structural displays the recommended generator as a button.

For self-hosted instances that have an Enterprise license, the recommended generator is the built-in generator preset.

To assign the recommended generator to the column, click the button.

Otherwise, select the generator from the Generator Type dropdown list.

For more information about selecting a generator, go to .

Configuring the generator

If the selected generator requires additional configuration, then below the Generator Type dropdown list is an Edit Generator Options link.

To display the configuration fields for the generator, click Edit Generator Options.

For information about configuring a selected generator or generator preset, go to .

After you configure the generator, to return to the settings view, click Back.

Displaying sample data for a column

Required workspace permission:

Source data: Preview source data

From the column details, you can display sample data for the column. The sample data allows you to compare the source and destination versions of the column values.

To display the sample data, click the view sample (magnifying glass) icon.

On the sample data view of the column details:

The Original Data tab shows the values in the source data.
The Protected Output tab shows the values that the generator produced.

Enabling Document View for JSON columns

Supported only for the file connector and PostgreSQL.

For a JSON column, instead of assigning a generator, you can enable Document View.

From Document View, you can view the JSON schema structure and assign generators to individual JSON fields. For more information, go to .

To enable Document View, on the column details panel, toggle Use Document View to the on position. When Document View is enabled, the generator dropdown is replaced with the Open in Document View option.

Commenting on a column

Required license: Professional or Enterprise

From the column details, you can view and add comments on the column. You might use a comment to explain why you selected a particular generator or marked a column as sensitive or not sensitive.

From the column details, to display the comments for the column, click the comment icon.

The comments view displays any existing comments on the column. The most recent comment is at the bottom of the list. Each comment includes the name of the user who made the comment.

To add the first comment to a column, type the comment into the comment text area, then click Comment.

To add an additional comment, type the comment into the comment text area, then click Reply.

Downloading a preview Privacy Report

Required license: Enterprise

The Privacy Report files that you download from Privacy Hub or the workspace download menu provide an overview of the current protection status based on the current configuration.

This is different from the Privacy Report files that you download from the data generation job details, which show the protection status for the data produced by that data generation.

You can download either:

The Privacy Report .csv file, which provides details about the table columns, the column content, and the current protection configuration.
The Privacy Report PDF file, which provides charts that summarize the privacy ranking scores for the table columns. It also includes the table from the .csv file.

For more information about the Privacy Report files and their content, go to .

From workspace management view

To download the report from the workspace management view, click the download icon. In the download menu:

To download the Privacy Report PDF file, click Download Privacy Report PDF.
To download the Privacy Report .csv file, click Download Privacy Report CSV.

From Privacy Hub

To download the report from Privacy Hub, click Reports and Logs, then:

To download the Privacy Report .csv file, click Privacy Report CSV.
To download the Privacy Report PDF file, click Privacy Report PDF.

Running a new sensitivity scan on the data

Required workspace permission: Run sensitivity scan

Privacy Hub provides an option to manually start a new . For example, you might want to run a new sensitivity scan when:

You add columns to the source database. The new scan identifies whether the new columns contain sensitive data.
The data in a column changes significantly, and a column that Structural originally marked as not sensitive might now contain sensitive data.

You cannot run a sensitivity scan on a . Child workspaces always inherit the sensitivity results from their parent workspace.

To run a new sensitivity scan, click Run Sensitivity Scan.

When Structural runs a new sensitivity scan:

Structural analyzes and determines the sensitivity of any new columns.
It does not change the sensitivity of existing columns that you marked as sensitive or not sensitive.
For existing columns that you did not change the sensitivity of:

The protection status panels are updated to reflect the results of the new scan.

Structural license plans

Tonic Structural provides different license plans to accommodate organizations that are of different sizes and that have more or less complex data architectures.

Basic license

The Basic license is designed for very small organizations that have a very simple data architecture. It provides access to Structural's core de-identification and data generation features.

Users

The Basic license allows access for a single user, with an option to purchase an additional two users.

There is no access to .

Data connectors

With a Basic license, you can create workspaces for one data connector type. The data connector type must be one of the following:

Concurrent jobs

With a Basic license, your Structural instance can have only one Structural worker. This means that only one sensitivity scan or data generation job can run at a time.

Structural features

With a Basic license, you can , and for those workspaces.

You can use to view the current sensitivity status based on the current workspace configuration.

The Basic license does NOT provide access to the following features:

Structural API

With a Basic license, you only have access to the basic version of the Structural API.

You cannot use the basic Structural API to perform the following API tasks, which require the advanced API:

Professional license

The Professional license is designed for larger organizations that have more complex data architectures. The organization might have a larger team that supports multiple databases.

The Professional license is also granted to .

The Professional license provides access to a larger set of Structural features than the Basic license.

Users

The Professional license allows up to 10 users. You can purchase access for unlimited users as an add-on.

You can use to manage your Structural users.

Data connectors

With a Professional license, you can create workspaces for up to two types of data connectors. You can purchase one additional data connector type as an add-on.

Those data connectors can be of any type except for and .

Concurrent jobs

With a Professional license, your Structural instance can have more than one Structural worker.

This means that you can run multiple jobs from different workspaces at the same time. You can never run multiple jobs from the same workspace at the same time.

Structural features

With a Professional license, you can do the following:

, and for those workspaces.
Use to view the current sensitivity status for your workspace configuration.
. The Professional license does not allow you to assign the built-in Viewer and Auditor permission sets.

The Professional license does NOT provide access to the following features:

Structural API

With a Professional license, you only have access to the basic version of the Structural API.

You cannot use the basic Structural API to perform the following API tasks, which require the advanced API:

Enterprise license

The Enterprise license is ideal for very large organizations that have multiple teams that support very large and complex data structures, and that might have more requirements related to scale and compliance.

It provides full access to all Structural features.

Users

An Enterprise instance does not limit the number of users.

Data connectors

You can use any number of any of the available data connectors.

The Enterprise license provides exclusive access to the and data connectors.

Structural features

The following features are exclusive to the Enterprise license:

Structural API

The Enterprise license provides exclusive access to the advanced API.

The advanced Structural API provides access to all of the available API tasks, including the following tasks that are not available in the basic API:

Feature comparison across Structural license plans

The following table compares the available features for the Structural license plans.

Feature

Basic

Professional

Enterprise

Tonic Structural

Tonic Structural User Guide

hashtagConnect to your data

hashtagConfigure and generate transformed data

hashtagManage a self-hosted Tonic Structural instance

About Tonic Structural

Structural data generation workflow

Logging into Structural for the first time

Creating and managing workspaces

Workspace configuration settings

Workspace identification and connection type

hashtagFields to identify the workspace

hashtagConnection type

Using secrets managers for authentication

Assigning tags to a workspace

hashtagManaging tags from workspace settings

hashtagManaging tags from Workspaces view

hashtagAssigning tags

hashtagEditing the assigned tags

Managing access to workspaces

Transferring ownership of a workspace

Managing the workspace schema cache

hashtagViewing the schema cache status

hashtagRefreshing the schema cache

Configuring data generation

Database View

Displaying sample data for a column

Identifying similar columns

Commenting on columns

hashtagCreating a new comment

hashtagResponding to existing comments

Working with document-based data

Identifying sensitive data

Manually indicating whether a column is sensitive

Algebraic

hashtagCharacteristics

hashtagHow to configure

Alphanumeric String Key

hashtagCharacteristics

hashtagHow to configure

Array Character Scramble

Character Substitution

Continuous

hashtagCharacteristics

File Name

Find and Replace

hashtagCharacteristics

hashtagHow to configure

FNR

hashtag

Hostname

hashtagCharacteristics

About Tonic Structural

Structural data generation workflow

Tonic Structural User Guide

hashtagConnect to your data

hashtagConfigure and generate transformed data

hashtagManage a self-hosted Tonic Structural instance

Displaying sample data for a column

Identifying similar columns

Logging into Structural for the first time

Workspace configuration settings

Workspace identification and connection type

hashtagFields to identify the workspace

hashtagConnection type

Using secrets managers for authentication

Assigning tags to a workspace

hashtagManaging tags from workspace settings

hashtagManaging tags from Workspaces view

hashtagAssigning tags

hashtagEditing the assigned tags

Managing the workspace schema cache

hashtagViewing the schema cache status

hashtagRefreshing the schema cache

Commenting on columns

hashtagCreating a new comment

hashtagResponding to existing comments

Working with document-based data

Manually indicating whether a column is sensitive

Algebraic

Connect to your data

Configure and generate transformed data

Manage a self-hosted Tonic Structural instance

Fields to identify the workspace

Connection type

Managing tags from workspace settings

Managing tags from Workspaces view

Assigning tags

Editing the assigned tags

Viewing the schema cache status

Refreshing the schema cache

Creating a new comment

Responding to existing comments

Characteristics

How to configure

Characteristics

How to configure

Characteristics

Characteristics

How to configure

Characteristics

Connect to your data

Configure and generate transformed data

Manage a self-hosted Tonic Structural instance

Fields to identify the workspace

Connection type

Managing tags from workspace settings

Managing tags from Workspaces view

Assigning tags

Editing the assigned tags

Viewing the schema cache status

Refreshing the schema cache

Creating a new comment

Responding to existing comments

Characteristics

How to configure

Characteristics

How to configure

Characteristics

How to configure

Characteristics

Characteristics

About workspace ownership

About ownership transfer

Completing the ownership transfer

View table and column information

Configure and comment on columns

Characteristics

How to configure

Characteristics

How to configure

How to configure

How to configure

How to configure

How to configure

Self-hosted Tonic Structural instance

Structural Cloud

Infrastructure engineers

Database administrators

Structural users

Data consumers

Security and compliance

Configuring the overrides

Enabling and setting an override

Removing an override

Available overrides

Workspace statistics seed for cross-run consistency

Data generation performance settings

Data encryption and decryption keys

Destination database schema creation

Overwrite handling for Databricks

Displaying the bulk edit option

Marking the columns as sensitive or not sensitive

Changing the assigned generator

Assigning the recommended generator to the columns

Restoring the baseline configuration for the columns

Creating a sensitivity rule

Characteristics

How to configure

Adding a sub-generator