A Tonic Structural implementation can involve the following roles - from those who set up the Structural environment to the consumers of the data that Structural processes.

Note that these roles are not related to role-based access (RBAC) within Structural, which is managed using permission sets.

Infrastructure engineers

For self-hosted instances of Structural.

Infrastructure engineers set up the Structural application and its relevant dependencies. They are typically DevOps, Site Reliability Engineering (SRE), or Kubernetes cluster administrators.

Infrastructure engineers perform the following Structural-related tasks:

• Ensure that the proper infrastructure is ready for Structural installation based on the deployment checklist.

• Follow the installation instructions. Works with Tonic.ai support as needed.

• Perform routine maintenance of Structural and the Structural environment. Updates Structural and its dependencies as needed.

• Create Structural-processed data pipelines for development and testing workflows.

Database administrators

For both self-hosted instances of Structural and Structural Cloud.

Database administrators integrate Structural into your data architecture to support Structural data connectors.

They ensure that source databases are available to Structural, and that Structural can write to destination databases.

Database administrators perform the following Structural-related tasks:

• Set up the required Structural access to source databases.

• Set up destination databases for Structural to write transformed data to.

Structural users

Structural users are the actual users of the Structural application.

Depending on the use case, Structural users might be compliance analysts, DevOps, or data engineers.

Tonic users perform the following Structural-related tasks:

• Use the Structural data generation workflow to configure the logic used to transform the source data and to generate the transformed data.

• Work with data consumers to produce usable data.

Data consumers

Data consumers are the end users of transformed destination data.

They are typically QA testers, developers, or analysts.

Data consumers perform the following Structural-related tasks:

• Validate the usability of the destination data.

• Provide guidance on application-specific requirements for data.

Security and compliance

Security and compliance specialists ensure and validate that the data that Structural produces meets expectations, and that Structural is compliant with other security-related processes.

Security and compliance specialists perform the following Structural-related tasks:

• Provide guidance on what data is sensitive.

• Sign off on proposed approaches to mask sensitive data.

• Approve data access and permissions.

Self-hosted Tonic Structural instance

You can deploy a self-hosted, on-premises instance of Tonic Structural.

For a self-hosted instance, Structural provides administrator tools that allow you to monitor Structural services and manage Structural users.

You can configure Structural environment settings to customize your instance.

On a self-hosted instance, based on your license plan, you have access to the full set of supported data connectors.

Structural Cloud

Structural Cloud is our secure hosted environment. On Structural Cloud, Tonic.ai handles monitoring Structural services and updating Structural.

For single sign-on (SSO), Structural Cloud only supports Okta.

Structural Cloud does not include:

• Custom permission sets

• Environment setting configuration. Structural Cloud uses a single configuration.

• Structural data encryption

• Access to the following data connectors:

  • Amazon EMR

  • Amazon Redshift

  • Db2 for LUW

  • Spark SDK

Structural Cloud also supports a pay-as-you-go plan, where free trial users can move on to set up a monthly subscription. For more information, go to Setting up and managing a Structural Cloud pay-as-you-go subscription.

Each Structural Cloud user belongs to a Structural Cloud organization, which is determined either by the user's email domain or by a workspace invitation. Structural Cloud users do not have any access to workspaces or users from other organizations.

Each free trial user is in a separate organization, along with any users that they invite to have access to a free trial workspace.

For information about Structural Cloud organizations, go to Structural organizations.

The Account Admin permission set allows a Structural Cloud user to manage organization users and workspaces. For information about granting access to the Account Admin permission set, go to Granting Account Admin access for a Structural Cloud organization.

The Tonic Structural platform creates safe, realistic datasets to use in staging environments or for local development. The Structural web application and API can be used by engineers, data analysts, or security experts.

Structural connects to source databases that contain sensitive data such as personally identifiable information (PII) or protected health information (PHI). To protect that data, Structural transforms the sensitive values and then writes the transformed data to a destination location.

New to Structural? Review the Tonic Structural workflow overview. For information on how to create a Structural account and start a Structural free trial, go to Getting started with the Structural free trial.

Want to know what's in the latest Structural releases? Go to the Tonic Structural release notes.

The Structural application heading includes a feature updates icon, which displays a summary of the newest features, and includes a link to the Structural release notes.

Connect to your data

Configure and generate transformed data

Manage a self-hosted Tonic Structural instance

Need help with Structural? Contact [email protected].

Tonic Structural data generation combines sensitive data detection and data transformation to create safe, secure, and compliant datasets.

The Structural data generation workflow involves the following steps:

You can also view this video overview of the Structural data generation workflow.

1. To get started, you create a workspace. When you create a workspace, you identify the type of source data, such as PostgreSQL or MySQL, and establish the connections to the source database and the destination location. The source database contains the original data that you want to synthesize. The destination location is where Structural stores the synthesized data. It might be a database, a storage location, a container repository, or an Ephemeral database snapshhot.

2. Next, you analyze the results of the initial sensitivity scan. The sensitivity scan identifies columns that contain sensitive data. These columns need to be protected by a generator.

3. Based on the sensitivity scan results, you configure the data generation. The configuration includes:

   • Assigning table modes to tables. The table mode controls the number of rows and columns that are copied to the destination database.

   • Indicating column sensitivity. You can make adjustments to the initial sensitivity assignments. For example, you can mark additional columns as sensitive that the initial scan did not identify as sensitive.

   • Assigning and configuring column generators. To protect the data in a column, especially a sensitive column, you assign a generator to it. The generator replaces the source value with a different value in the destination database. For example, the generator might scramble the characters or assign a random value of the same type.

4. After you complete the configuration, you run the data generation job. The data generation job uses the configured table modes and generators to transform the data from the source database and write the transformed data to the destination location. You can track the job progress and view the job results.

The Tonic Structural synthetic data platform combines sensitive data detection and data transformation to allow users to create safe, secure, and compliant datasets.

Common Structural use cases include creating staging and development environments, and trying out a new cloud provider without complex data agreements.

Structural allows you to reduce bug counts, shorten testing life cycles, and share data with partners, all while helping to ensure security and compliance with the latest regulations, from GDPR to CCPA.

You can use the Structural API to integrate with CI/CD pipelines or to create automated processes that ensure that the generated data is available on demand.

You can associate custom tags with each workspace. Tags can help to organize and provide a quick glance into the workspace configuration.

Tags are accessible to every user that has access to the workspace.

Tags are stored in the workspace JSON, and are included in the workspace export. You can also use the API to get access to tags.

Managing tags from workspace settings

You can add and edit tags in the Tags field on the New Workspace and Workspace Settings views.

• To add tags, enter a comma-separated list of the tags to add.

• To remove a tag, click its delete icon.

Managing tags from Workspaces view

You can also manage tags directly from Workspaces view.

Assigning tags

To add tags to a workspace that does not currently have tags:

1. Hover over the Tags column for the workspace.

2. Click Add Tags.

3. In the tag input field, type a comma-separated list of tags to apply.

4. Press Enter.

Editing the assigned tags

To edit the assigned tags:

1. Click the Tags column for the workspace.

2. In the tag input field, to remove tag, click its delete icon.

3. To add tags, type a comma-separated list of the tags to add.

4. To save the tag changes, press Enter.

For each column on Database View, you can display a sample list of the column values.

For columns that have an assigned generator, the sample shows both the current values and the possible values after the generator is applied.

To display the sample values, in the Column column, click the magnifying glass icon.

If the generator is Passthrough, then the sample data panel contains only Original Data.

If a different generator is assigned, then the sample data panel contains both Original Data and Protected Output.

During sensitivity scans and schema change scans, Tonic Structural identifies groups of similar columns.

To identify similar columns, Structural uses a text embedding model to calculate the semantic similarity between any two column names in the database. When a column name's semantic similarity to the name of a given column is above a specified threshold, then the column is similar to the given column.

If a column has similar columns, then the Applied Generator column contains an icon that includes the count of similar columns.

By default, the similar columns icon is hidden. To display the similar columns icon, hover over the column row.

When you assign a generator to a column, the similar columns icon for that column remains visible during your current session.

When you click the similar columns icon, Structural displays a panel with an option to filter the list to display the current column and its similar columns. To apply the filter, click Filter.

The similar columns filter is applied, and other column filters are removed. Table filters remain in place.

You can also manually indicate that a column is sensitive or not sensitive.

For example, the sensitivity scan might incorrectly identify a column as sensitive. Or a column might contain data that you consider sensitive but that does not match a detected sensitivity type.

When you manually change a column from not sensitive to sensitive, Structural marks the sensitivity detection as full confidence.

For information on how to change whether a column is sensitive:

• For Privacy Hub, go to .

• For Database View, go to:

  • For a single column,

  • For multiple selected columns,

• For Table View, go to .

The Structural API also provides endpoints to designate columns as sensitive or not sensitive.

When you go to Tonic Structural for the first time, you create an account. How you create an account depends on the type of user you are.

A new Structural user can be one of the following:

• Free trial users use Structural Cloud to explore and experiment with Structural before they decide whether to purchase it.

• Self-hosted instances are installed on-premises. The customer administers the Structural users.

• New users are added to existing organizations based on their email domain.

The workspace settings for a new workspace (New Workspace view) or edited workspace (Workspace Settings tab) provide information about the workspace and its data.

Every workspace includes the following settings to identify the workspace and to select the type of data connector.

Fields to identify the workspace

All workspaces have the following fields that identify the workspace:

1. In the Workspace name field, enter the name of the workspace.

2. In the Workspace description field, provide a brief description of the workspace. The description can contain up to 200 characters.

3. In the Tags field, provide a comma-separated list of tags to assign to the workspace. For more information on managing tags, go to .

Connection type

Under Connection Type, select the type of data connector to use for the workspace data. You cannot change the connection type on a .

The Basic and Professional licenses limit the number and type of data connectors you can use.

• A Basic instance can only use one data connector type, which can be either PostgreSQL or MySQL. After you create your first workspace, any subsequent workspaces must use the same data connector type.

• A Professional instance can use up two different data connector types, which can be any type other than Oracle or Db2 for LUW. After you create workspaces that use two different data connector types, any subsequent workspaces must use one of those data connector types.

If the database that you want to connect to isn't in the list, or you want to have different database types for your source and destination database, contact [email protected].

When you select a connector type, Structural updates the view to display the connection fields used for that connector type. The specific fields vary based on the .

Blocking data generation for all schema changes

Most workspaces that connect to a database have a Block data generation if schema changes detected toggle. The setting is usually in the Source Settings section.

By default, the option is turned off. When the option is off:

• Structural blocks data generation when there are sensitive schema changes. A sensitive schema change is a change that might result in data leakage. For example, a new column is added. Until you assign a generator to the column, the data is passed through as is to the output data. Before you can generate data with the new column, you must resolve the schema change.

• Structural does not block data generation when there are only notification schema changes. A notification schema change is a change that Structural can ignore during data generation. For example, a column is removed. Structural can ignore that column during data generation, until you are able to permanently resolve the schema change.

If this option is turned on, then if Structural detects any changes at all to the schema, then until you resolve the schema changes, data generation is blocked.

For more information, go to .

For self-hosted instances, Structural provides to configure features that include:

• Consistency across runs and databases

• Data generation performance

The Advanced Workspace Overrides section of the workspace details view allows you to override those environment settings for an individual workspace.

For example, the environment setting TONIC_TABLE_PARALLELISM determines the number of tables that Structural processes simultaneously. You can then override that value within individual workspaces.

The workspace overrides are available on both self-hosted instances and on Structural Cloud.

Configuring the overrides

To display the available override settings, expand Advanced Workspace Overrides.

Enabling and setting an override

For information on how to configure the statistics seed, go to .

For other settings, to enable the override and set the override value:

1. Toggle the setting to the on position.

1. Set the value.

Removing an override

To remove the override, toggle the setting to the off position.

Available overrides

Workspace statistics seed for cross-run consistency

For generators where is enabled, a statistics seed enables consistency across data generation runs. The Structural-wide statistics seed value ensures consistency across both data generation runs and workspaces.

You use the Override Statistics Seed setting to override the Structural-wide statistics seed value.

You can either disable consistency across data generations, or provide a seed value for the workspace. The workspace seed value ensures consistency across data generation runs for that workspace, and across other workspaces that have the same seed value.

For details about using seed values to ensure consistency across data generation runs and databases, go to .

Data generation performance settings

Structural provides environment settings to manage . For example, these settings include configuration for parallel processing.

From Advanced Workspace Overrides, you can override some of these data generation performance settings for an individual workspace.

Data encryption and decryption keys

To use Structural data encryption, you must .

You use the Override Data Decryption Key and Override Data Encryption Key settings to override the Structural-wide keys that are provided in the environment settings.

Destination database schema creation

Some data connectors allow you to configure whether you provide the schema for the destination database. For more information, go to related information for , , , , , and .

From Advanced Workspace Overrides, you can override the instance-wide configuration.

You can export a workspace configuration to a JSON file, and import configuration from a workspace configuration JSON file.

For example, you might want to preserve a version of the workspace configuration before you test other changes. You can then use the exported file to restore the original configuration.

Or you might want to use a script to make changes to an exported configuration file. You can then import the updated file to update the workspace configuration.

Information in the exported file

The workspace JSON configuration file includes the following information:

• Sensitivity designations that you assigned to columns

• Assigned table modes

• Assigned column generators

• Subsetting configuration

• Post-job script configuration

Exporting the workspace configuration

To export the workspace configuration, either:

• On the workspace management view, from the download menu, select Export Workspace.

• On Workspaces view, click the actions menu for the workspace, then select Export.

When you export a child workspace, the exported workspace does not retain any of the inheritance information. The exported information is the same for all exported workspaces.

Importing a workspace configuration file

To import a workspace configuration file:

1. Select the import option. Either:

   • On the workspace management view, from the download menu, select Import Workspace.

   • On Workspaces view, click the actions menu for the workspace, then select Import.

2. On the Import Workspace dialog, to select the file to import, click Browse.

3. After you select the file, click Import.

When you import a workspace configuration into a child workspace, Tonic Structural only updates the configuration that can be overridden. If a configuration must be inherited from the parent workspace, then it is not affected by the imported configuration. For more information, go to .

When you create a workspace, you become the owner of the workspace, and by default are assigned the built-in Manager workspace permission set for the workspace. The Manager permission set provides full access to the workspace configuration, data, and results.

With a Professional or Enterprise license, you can also assign workspace permission sets to other users and to SSO groups. You can also transfer a workspace to a different owner.

If you are granted access to any workspace permission set for a workspace, then you have access to all of the workspace management views for that workspace. However, you can only perform tasks that you have permission for in that workspace.

Workspace access is managed from the Workspaces view. You cannot assign workspace permission sets from Structural Settings view.

You can also view an .

From Database View, you can add comments to columns. For example, you might use a comment to explain why you selected a particular generator or marked a column as sensitive or not sensitive.

Creating a new comment

If a column does not have any comments, then to add a comment:

1. In the Applied Generator column, click the comment icon.

2. In the comment field, type the comment text.

3. Click Comment.

Responding to existing comments

When a column has existing comments, the comment icon is green. To add comments:

1. Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user.

2. In the comment field, type the comment text.

3. Click Reply.

For document-based data connectors - currently and - Database View and Table View are replaced by Collection View.

"Collection" is the term that Structural uses to refer to MongoDB collections and DynamoDB tables.

Structural also allows you to run collection scans to identify the data structure.

Tonic Structural uses its sensitivity scan to identify source data columns that contain sensitive information. The scan ignores truncated tables.

The sensitivity scan identifies Structural's built-in sensitivity types. It also looks for custom types that you define.

You can also manually mark a column as sensitive or not sensitive.

The algebraic generator identifies the algebraic relationship between three or more numeric values and generates new values to match. At least one of the values must be a non-integer.

If a relationship cannot be found, then the generator defaults to the generator.

This generator can be linked with other Algebraic generators.

Characteristics

How to configure

To configure the generator, from the Link To dropdown list, select the columns to link this column to. You can select other columns that are assigned the Algebraic generator.

You must select at least three columns.

The column values must be numeric. At least one of the columns must contain a value other than an integer.

If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates unique alphanumeric strings based on any printable ASCII characters. The length of the source string is not preserved. You can choose to exclude lowercase letters from the generated values.

Characteristics

How to configure

To configure the generator:

1. To exclude lowercase letters from the generated values, toggle Exclude Lowercase Alphabet to the on position.

2. Toggle the Consistency setting to indicate whether to make the generator consistent. By default, the generator is not consistent.

3. If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

This generator replaces all instances of the find string with the replace string.

For example, you can indicate to replace all instances of abc with 123.

Characteristics

How to configure

To configure the generator:

1. In the Find field, type the string to look for in the source column value. To use a regular expression to identify the source value, check the Use Regex checkbox. If you use a regular expression, use backslash ( \ ) as the escape character.

2. In the Replace field, type the string to replace the matching string with.

3. If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a valid Finnish Personal Identity Code (PIC) that would have been issued during a specific date range.

Characteristics

How to configure

To configure the generator:

1. Under Date Range, set the start and date for the date range to generate the PICs for.

2. Toggle the Consistency setting to indicate whether to make the generator self-consistent. By default, the generator is not consistent.

3. If Structural data encryption is enabled, then to use it for this column, toggle Use data encryption process to the on position.

Every workspace has an owner. The owner is always a user.

The user who creates the workspace is automatically the owner of the workspace.

By default, the workspace owner is assigned the built-in Manager workspace permission set. On Enterprise instances, you can choose a different workspace permission set to assign to all workspace owners.

You cannot remove that permission set from the workspace owner.

You can transfer a workspace to a different owner. The new owner is assigned the owner permission set. If the previous owner does not otherwise have access to the owner permission set, then that permission set is removed.

To transfer workspace ownership:

1. To transfer ownership of a single workspace, from the workspace actions menu, select Transfer Ownership.

2. To transfer ownership of multiple workspaces:

   1. Check the checkbox for each workspace to grant access to.

   2. From the Actions menu, select Transfer Ownership.

3. On the transfer ownership panel, from the User dropdown list, select the new owner.

4. If you are the current owner of the workspace, then to grant yourself non-owner access after you transfer the ownership:

   1. Toggle Receive access to workspace to the on position.

   2. Select the workspace permission set to assign to yourself.

5. Click Transfer Ownership.

Database View provides a complete view of your source database structure and configuration.

To display Database View, either:

• On the workspace management view, in the workspace navigation bar, click Database View.

• On Workspaces view, from the dropdown menu in the Name column, select Database View.

Database View consists of:

• On the left, the list of tables in the source database.

• On the right, the list of columns in those tables.

View table and column information

Configure and comment on columns

Generators transform the data in a source database column. You assign the generators to use. Tonic Structural offers a variety of generators to transform different types of data.

For details about how to assign and configure generators, and manage generator presets, go to Generator assignment and configuration.

You can also view this video overview of generators and how they work.

About the available generators

Generator characteristics and types

Generates unique alphanumeric strings of the same length as the input.

For example, for the origin value ABC123, the output value is a six-character alphanumeric string such as D24N05.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

By default, the generator is not consistent.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

A version of the Character Scramble generator that can be used for array values.

This generator replaces letters with random other letters, and numbers with random other numbers. Punctuation and whitespace are preserved.

For example, for the following array value:

["ABC.123", 3, "last week"]

The output might be something like:

["KFR.860", 7, "sdrw mwoc"]

This generator securely masks letters and numbers. There is no way to recover the original data.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

By default, the generator is not consistent.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a random company name-like string.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator consistent.

By default, the generator is not consistent.

If consistency is enabled, then by default it is self-consistent. To make the generator consistent with another column, from the Consistent to dropdown list, select the column.

When the generator is consistent with itself, then a given source value is always mapped to the same destination value. For example, My Business is always mapped to New Business.

When the generator is consistent with another column, then a given source value in that other column always results in the same destination value for the company name column. For example, if the company name column is consistent with a name column, then every instance of John Smith in the name column in the source database has the same company name in the destination database.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

The Categorical generator shuffles the existing values within a field while maintaining the overall frequency of the values. It disassociates the values from other pieces of data. Note that NULL is considered a separate value.

For example, a column contains the values Small, Medium, and Large. Small appears 3 times, Medium appears 4 times, and Large appears 5 times. In the output data, each value still appears the same number of times, but the values are shuffled to different rows.

This generator is optimized for categories with fewer than 10,000 unique values. If your underlying data has more unique values (for example, your field is populated by freeform text entry), we recommend that you use the Character Scramble or Custom Categorical generator.

Characteristics

How to configure

To configure the generator:

1. From the Link To dropdown, select the columns to link to the current column. You can select from other columns that use the Categorical generator.

2. Toggle the Differential Privacy setting to indicate whether to make the output data differentially private. By default, differential privacy is disabled.

3. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

This generator replaces letters with random other letters and numbers with random other numbers. Punctuation, whitespace, and mathematical symbols are preserved.

For example, for the following input string:

ABC.123 123-456-789 Go!

The output would be something like:

PRX.804 296-915-378 Ab!

This generator securely masks letters and numbers. There is no way to recover the original data.

Character Scramble is similar to Character Substitution, with a couple of key differences.

While you can enable consistency for the entire value, Character Scramble does not always replace the same source character with the same destination character. Because there is no guarantee of unique values, you cannot use Character Scramble on unique columns.

Character Substitution, however, does always map the same source character to the same destination character. Character Substitution is always consistent, which makes it less secure than Character Scramble. You can use Character Substitution on unique columns.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

By default, the generator is not consistent.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Performs a random character replacement that preserves formatting (spaces, capitalization, and punctuation).

Characters are replaced with other characters from within the same Unicode Block. A given source character is always mapped to the same destination character. For example, M might always map to V.

For example, for the following input string:

Miami Store #162

The output would be something like:

Vgkjg Gmlvf #681

Note that for a numeric column, when a generated number starts with a 0, the starting 0 is removed. This could result in matching output values in different columns. For example, one column is changed to 113 and the other to 0113, which also becomes 113.

Character Substitution is similar to Character Scramble, with a couple of key differences. Because Character Substitution always maps the same source character to the same destination character, it is always consistent. It also can be used for unique columns.

In Character Scramble, the character mapping is random, which makes Character Scramble slightly more secure. However, Character Scramble cannot be used for unique columns.

Characteristics

How to configure

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a random company name-like string.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator consistent.

By default, the generator is not consistent.

If consistency is enabled, then by default it is self-consistent. To make the generator consistent with another column, from the Consistent to dropdown list, select the column.

When the generator is consistent with itself, then a given source value is always mapped to the same destination value. For example, My Company is always mapped to New Company.

When the generator is consistent with another column, then a given source value in that other column always results in the same destination value for the company name column. For example, if the company name column is consistent with a name column, then every instance of John Smith in the name column in the source database has the same company name in the destination database.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Uses a single value to mask all of the values in the column.

For example, you can replace every value in a string column with the value String1. Or you can replace every value in a numeric column with the value 12345.

Characteristics

How to configure

To configure the generator, in the Constant Value field, provide the value to use.

The value must be compatible with the field type. For example, you cannot provide a string value for an integer column.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a continuous distribution to fit the underlying data.

This generator can be linked to other Continuous generators to create multivariate distributions and can be partitioned by other columns.

Characteristics

How to configure

To configure the generator:

1. From the Link To drop-down list, select the other Continuous generator columns to link to. The linking creates a multivariate distribution.

2. From the Partition By drop-down list, select one or more columns to use to partition the data. The selected columns must have the generator set to either Passthrough or Categorical. For more information about partitioning and how it works, go to Partitioning a column.

3. Toggle the Differential Privacy setting to indicate whether to make the output data differentially private. By default, the generator is not differentially private.

4. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

A version of the Categorical generator that selects from values that you provide instead of shuffling the original values.

Characteristics

How to configure

To configure the generator:

1. From the Link To dropdown list, select the columns to link this column to. You can only select other columns that use the Custom Categorical generator.

2. In the Custom Categories text area, enter the list of values that the generator can choose from. Put each value on a separate line. To add a NULL value to the list, use the keyword {NULL}.

3. Toggle the Consistency setting to indicate whether to make the column consistent. By default, consistency is disabled.

4. If you enable consistency, then by default the generator is self-consistent. To make the generator consistent with another column, from the Consistent to dropdown list, select the column. When a generator is self-consistent, then a given value in the source database is always mapped to the same value in the destination database. When a generator is consistent with another column, then a given source value in that column always results in the same value for the current column in the destination database. For example, a department column is consistent with a username column. For each instance of User1 in the source database, the value in the department column is the same.

5. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates timestamps that fit an event distribution. The source timestamp must include a date. It cannot be a time-only value.

Link columns to create a sequence of events across multiple columns. This generator can be partitioned by other columns.

Characteristics

How to configure

To configure the generator:

1. From the Link To dropdown list, select the other Event Timestamps generator columns to link this column to. Linking creates a sequence across multiple columns.

2. From the Partition drop-down list, select one or more columns to use to partition the data. The selected columns must have their generator set to either Passthrough or Categorical. For more information about partitioning and how it works, go to Partitioning a column.

3. The Options list displays the current column and linked columns. Use the Up and Down buttons to configure the column sequence.

4. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

This generator scrambles characters, but preserves formatting and keeps the file extension intact.

For example, for the following input value:

DataSummary1.pdf

The output value would look something like:

RsnoPwcsrtv5.pdf

This generator securely masks letters and numbers. There is no way to recover the original data.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

By default, the generator is not consistent.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

The FNR generator transforms Norwegian national identity numbers. In Norwegian, the term for national identity number abbreviates to FNR.

The first six digits of an FNR reflects the person's birthdate. You can choose to preserve the birthdates from the source values in the destination values. If you do not preserve the source values, the destination values are still within the same date range as the source values.

Another digit in an FNR indicates whether the person is male or female. You can specify whether to preserve in the generated value the gender indicated in the source value.

The last digits in an FNR are a checksum value. The last digits in the destination value are not a checksum - the values are random.

Characteristics

How to configure

To configure the generator:

1. To preserve the gender from the source value in the destination value, toggle Preserve Gender to the on position.

2. To preserve the birthdate from the source value in the destination value, toggle Preserve Birthdate to the on position.

3. Toggle the Consistency setting to indicate whether to make the generator consistent. By default, consistency is disabled.

4. If you enable consistency, then by default the generator is self-consistent. To make the generator consistent with another column, from the Consistent to dropdown list, select the column. When a generator is self-consistent, then a given value in the source database is always mapped to the same value in the destination database. When a generator is consistent with another column, then a given value for that other column in the source database results in the same value in the destination database. For example, if the FNR column is consistent with a Name column, then every instance of John Smith in the source database results in the same FNR in the destination database.

5. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

This generator can be used to mask columns of latitude and longitude.

The Geo generator divides the globe into grids that are approximately 4.9 x 4.9 km. It then counts the number of points within each grid.

During data generation, each (latitude, longitude) pair is mapped to its grid.

• If the grid contains a sufficient number of points to preserve privacy, then the generator returns a randomly chosen point in that grid.

• If the grid does not contain enough points to preserve privacy, then the generator returns a random coordinate from the nearest grid that contains enough points.

Characteristics

How to configure

To configure the generator:

1. From the Link To dropdown list, select the column to link to this one. You typically assign the Geo generator to both the latitude and longitude column, then link those columns.

2. From the value type dropdown, select whether this column contains a latitude value or a longitude value.

3. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates random host names, based on the English language.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator consistent.

By default, the generator is not consistent.

If you enable consistency, then by default the generator is self-consistent. To make the generator consistent with another column, from Consistent to, select the column.

When the generator is consistent with itself, then a given value in the source database is mapped to the same value in the destination database. For example, Host123 in the source database always produces MyHostABC in the destination database.

When the generator is consistent with another column, then a given source value in the other column results in the same host name value in the destination database. For example, a host name column is consistent with a department column. Every instance of Sales in the source data is given the same host name in the destination database.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates unique integer values. By default, the generated values are within the range of the column’s data type.

You can also specify a range for the generated values. The source values must be within that range.

This generator cannot be used to transform negative numbers.

Characteristics

How to configure

To configure the generator:

1. In the Minimum field, enter the minimum value to use for an output value. The minimum value cannot be larger than any of the values in the source data.

2. In the Maximum field, enter the maximum value to use for an output value. The maximum value cannot be smaller than any of the values in the source data.

3. Toggle the Consistency setting to indicate whether to make the column self-consistent. By default, consistency is disabled.

4. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates unique object identifiers.

Can be assigned to text columns that contain MongoDB ObjectId values. The column value must be 12 bytes long.

Characteristics

How to configure

To configure the generator:

1. A MongoID object identifier consists of an epoch timestamp, a random value, and an incremented counter. To only change the random value portion of the identifier, but keep the timestamp and counter portions, toggle Preserve Timestamp and Incremental Counter to the on position.

2. Toggle the Consistency setting to indicate whether to make the generator self-consistent. By default, the generator is not consistent.

3. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a random boolean value.

Characteristics

How to configure

To configure the generator, in the Percent True field, enter the percentage of values to set to True in the output.

For example, if you set this to 60, then 60% of the output values are True, and 40% of the output values are False.

If you set this to 100, then all of the output values are True.

If you set this to 0, then all of the output values are False.

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates NULL values to fill the rows of the specified column.

Characteristics

How to configure

The Null generator has no configuration options.

Passthrough is the default option.

It passes through the value from the source database to the destination database without masking it.

Characteristics

How to configure

Passthrough has no configuration options.

After you select the connector type, you configure:

• Where to find the source data

• Where to write the data generation output

Source database connection

For data connectors that connect to a database, the Source Settings section provides connection information for the source database.

You cannot change the source data configuration for a .

For information about the source connection fields for a specific data connector, go to the workspace configuration topic for that .

Upsert configuration

For data connectors that support upsert, the workspace configuration includes an Upsert section to allow you to enable and configure upsert. Upsert adds and updates rows in the destination database, but keeps all other existing rows intact.

If you enable upsert, then you cannot write output to an Ephemeral database or to a container repository. You must write the output to a destination database.

For more information, go to .

Destination data location

For data connectors that connect to a database, the Destination Settings section provides information about where and how Structural writes the output data from data generation.

Depending on the data connector type, you might be able to write to either:

• Destination database - Writes the output data to a destination database on a database server.

• Ephemeral snapshot - Writes the output data to a Tonic Ephemeral user snapshot.

• Container repository - Writes the output data to a data volume in a container repository.

Destination database

When you write the output to a destination database, the destination database must be of the same type as the source database.

Structural does not create the destination database. It must exist before you generate data.

In Destination Settings, you provide the connection information for the destination database. For information about the destination database connection fields for a specific data connector, go to the workspace configuration topic for that .

If available, the Copy Settings from Source allows you to copy the source connection details to the destination database, if both databases are in the same location. Structural does not copy the connection password.

Tonic Ephemeral snapshot

Tonic Ephemeral is a separate Tonic.ai product that allows you to create temporary databases to use for testing and demos. For more information about Ephemeral, go to the .

If Ephemeral supports your workspace database type, then you can write the destination data to a snapshot in Ephemeral. For data larger than 10 GB, this option is recommended instead of writing to a container repository.

From Ephemeral, you can use the snapshot to start new Ephemeral databases.

For more information, go to .

Container repository

Some data connectors allow you to write the transformed data to a data volume in a container repository instead of to a database server.

You can use the resulting data volume to create a database in Tonic Ephemeral. If you do plan to use the data to start an Ephemeral database, and the size of the data is larger than 10 GB, then the recommendation is to write the data to an Ephemeral user snapshot instead.

For more information, go to .

Testing database connections

When you provide connection details for a database server, Structural provides a Test Connection button to test the connection, and verify that Structural can use the connection details to connect to the database. Structural uses the connection details to try to reach the database, and indicates whether it succeeded or failed. We strongly recommend that you test the connections.

The TONIC_TEST_CONNECTION_TIMEOUT_IN_SECONDS determines the number of seconds before a connection test times out. You can configure this setting from the Environment Settings tab on Structural Settings. By default, the connection test times out after 15 seconds.

File connector source and destination data

A workspace uses files as its source data and produces transformed versions of those files as its output.

For file connector workspaces, the File Location section indicates where the source files are obtained from - either a local file system or a cloud storage solution (Amazon S3 or Google Cloud Storage).

When the files come from cloud storage, the Output Location section indicates where to write the transformed files. You must also provide the cloud storage connection credentials.

For more information, go to .

Tonic Structural uses workspace permission sets for role-based access (RBAC) of each workspace.

A workspace permission set is a set of . Each permission provides access to a specific workspace feature or function.

Structural provides . Enterprise instances can also .

To share workspace access, you assign workspace permission sets to users and, if you use SSO to manage Structural users, to SSO groups. Before you assign a workspace permission set to an SSO group, make sure that you are aware of who is in the group. The permissions that are granted to an SSO group automatically are granted to all of the users in the group. For information on how to configure Structural to filter the allowed SSO groups, go to .

You cannot remove the owner workspace permission set from the workspace owner. By default, the owner permission set is the built-in Manager permission set.

To change the current access to the workspace:

1. To manage access to a single workspace, either:

   • On the workspace management view, in the heading, click the share icon.

   • On Workspaces view, click the actions menu for the workspace, then select Share.

2. To manage access for multiple workspaces:

   1. Check the checkbox for each workspace to grant access to.

   2. From the Actions menu, select Share Workspaces.

3. The workspace access panel contains the current list of users and groups that have access to the workspace. To add a user or group to the list of users and groups, begin to type the user email address or group name. From the list of matching users or groups, select the user or group to add. Free trial users can invite other users to start their own free trial. Provide the email addresses of the users to invite. The email addresses must have the same corporate email domain as your email address. When the invited users sign up for the free trial, they are added to the Structural organization for the free trial user that invited them and have access to the workspace.

4. For a user or group, to change the assigned workspace permission sets:

   1. Click Access. The dropdown list is populated with the list of custom and built-in workspace permission sets. If you selected multiple workspaces, then on the initial display of the workspace sharing panel, for each permission set that a user or group currently has access to, the list shows the number of workspaces for which the user or group has that permission set. For example, you select three workspaces. A user currently has Editor access for one workspace and Viewer access for the other two. The Editor permission set has 1 next to it, and the Viewer permission set has 2 next to it.

   2. Under Custom Permission Sets, check the checkbox next to each workspace permission set to assign to the user or group. Uncheck the checkbox next to each workspace permission set to remove from the user or group.

   3. Under Built-In Permission Sets, check the workspace permission set to assign to the user or group. You can only assign one built-in permission set. By default, for an added user or group, the Editor permission set is selected. To select a built-in workspace permission set that is lower in access than the currently selected permission set, you must first uncheck the selected permission set. For example, if Editor is currently checked, then to change the selection to Viewer, you must first uncheck Editor.

5. To remove all access for a user or group, and remove the user or group from the list, click Access, then click Revoke.

6. To save the new access, click Save.

Structural identifies the following types of sensitive values. These include some information types that are considered by many privacy standards and frameworks such as HIPAA, GDPR, CCPA, and PCI.

For more information about the HIPAA and Safe Harbor information types that Structural detects, go to the Tonic.ai guide .

Names

• First

• Last

• Full

Organization

Location

• Street address

• ZIP

• PO Box

• City

• State and two-letter abbreviation

• Country

• Postal code

• GPS coordinates

Contact information

• Email address

• Telephone number

User credentials

• Username

• Password

Financial information

• Credit card number

• International bank account number (IBAN)

• SWIFT code for bank transfers

• Money amount

• BTC (Bitcoin) address

Identification

• Social Security Number

• Passport number

• Driver's license number

• Birth date

• Gender

• Biometric identifier, such as a fingerprint or voiceprint

• Full face photographic images and similar images

Medical information

• ICD-9 and ICD-10 codes (Used to identify diseases)

• Medical record number

• Health plan beneficiary number

• Admission date

• Discharge date

• Date of death

Other personal information

• Marital status

Accounts and licenses

• Account number

• Certificate or license number

Network and web location

• IP address

• IPv6 address

• MAC address

• Web URL

International Mobile Equipment Identity (IMEI)

Vehicle information

• Vehicle identification number (VIN)

• License plate number

This is a .

A version of the generator that can be used for array values.

Uses regular expressions to parse strings and replace specified substrings with the output of specified generators. The parts of the string to replace are specified inside unnamed top-level capture groups.

Characteristics

How to configure

Adding a regular expression

To add a regular expression:

1. Click Add Regex. On the configuration panel, Cell Value shows a sample value from the source database. You can use the previous and next options to navigate through the values.

2. By default, Replace all matches is enabled. To only match the first occurrence of a pattern, toggle Replace all matches to the off position.

3. In the Pattern field, enter a regular expression. If the expression is valid, then Structural displays the capture groups for the expression.

4. For each capture group, to select and configure the generator to apply, click the selected generator. You cannot select another composite generator.

5. To save the configuration and immediately add a generator for another path expression, click Save and Add Another. To save the configuration and close the add generator panel, click Save.

Managing the regular expressions list

From the Regexes list:

• To edit a regular expression, click the edit icon.

• To remove a regular expression, click the delete icon.

Links columns in two tables. This column value is the sum of the values in a column in another table.

This generator does not provide a preview. The sums are not computed until the other table is generated.

For example, a Customers table contains a Total_Sales column. The Transactions table uses a foreign key Customer_ID column to identify the customer who made the transaction, and an Amount column that contains the amount of the sale. The Customer_ID value in the Transactions table is a value from the ID primary key column in the Customers table.

You assign the Cross Table Sum generator to the Total_Sales column. In the generator configuration, you indicate that the value is the sum of the Amount values for the Customer_ID value that matches the primary key ID value for the current row.

For the Customers row for ID 123, the Total_Sales column contains the sum of the Amount column for Transactions rows where Customer_ID is 123.

Characteristics

How to configure

To configure the generator:

1. From the Foreign Table dropdown list, select the table that contains the column for which to sum the values.

2. From the Foreign Key dropdown list, select the foreign key. The foreign key identifies the row from the current table that is referred to in the foreign table.

3. From the Sum Over dropdown list, select the column for which to sum the values.

4. From the Primary Key dropdown list, select the primary key for the current table.

5. If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

This is a .

Runs selected generators on specified key values in an HStore column in a PostgreSQL database. HStore columns contain a set of key-value pairs.

Characteristics

How to configure

Adding a sub-generator

To assign a generator to a key:

1. Under Sub-generators, click Add Generator. On the sub-generator configuration panel, the Cell HStore field contains a sample value from the source database. You can use the previous and next icons to page through different values.

2. Under Enter a key, enter the name of a key from the column value. For example, for the column value: "pages"=>"446", "title"=>"The Iliad", "category"=>"mythology" To apply a generator to the title, you would enter title as the key. Matched HStore Values shows the result from the value in Cell HStore.

3. From the Generator Configuration dropdown list, select the generator to apply to the key value. You cannot select another composite generator.

4. Configure the selected generator. You cannot configure the selected generator to be consistent with another column.

5. To save the configuration and immediately add a generator for another key, click Save and Add Another. To save the configuration and close the add generator panel, click Save.

Managing the sub-generators list

From the Sub-Generators list:

• To edit a generator assignment, click the edit icon.

• To remove a generator assignment, click the delete icon.

• To move a generator assignment up or down in the list, click the up or down arrow.

Generates an address-like string to replace either:

• For a Canadian postal address, the street name or postal code.

• For a United Kingdom (UK) mailing address, the postal code.

To replace a Canadian postal code:

• The generator selects a real postal code that starts with the same three digits - has the same Forward Sortation Area (FSA) - as the original postal code, but that has a different Local Delivery Unit (LDU).

• For a postal code whose FSA is not on the list that the generator uses, you can provide a fallback value to use.

To replace a UK postal code, the generator selects a real postal code.

Characteristics

How to configure

To configure the generator:

1. From the Generator Type dropdown list, select International Address.

2. From the Country dropdown list, select the country (Canada or United Kingdom).

3. From the Address Component dropdown list, select the address component that this column contains. For Canada, the available options are:

   • Street Name

   • Postal Code

   For the UK, the only option is to generate a postal code.

4. For a Canadian postal code, in the Fallback Value field, type the FSA to use if the value in the data does not exist. For example, the FSA in the data might be new and not yet in the list that Structural uses, or the FSA might be invalid. By default, the fallback value is NULL, meaning that in the destination data, the postal code value is the string literal "NULL".

5. Toggle the Consistency setting to indicate whether to make the column self-consistent. By default, consistency is disabled.

6. If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a random telephone number that matches the country or region of the input telephone number, but preserves the format. For example, (123) 456-7890 or 123-456-7890.

If the input is not a valid telephone number, the generator randomly replaces numeric characters. You can also replace invalid numbers with valid numbers.

By default, the numbers are United States telephone numbers.

If the input is a valid telephone number, or if you replace invalid numbers, then the generated numbers pass Google's verification.

Characteristics

How to configure

To configure the generator:

1. Toggle the Replace invalid numbers setting to indicate whether to replace invalid input values with a valid output value. By default, the generator does not replace invalid values. It randomly replaces numeric characters.

2. Toggle the Consistency setting to indicate whether to make the generator self-consistent. By default, consistency is disabled.

3. If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a random double number between the specified minimum (inclusive) and maximum (exclusive).

Characteristics

How to configure

To configure the generator:

1. In the Minimum field, type the minimum value to use in the output values. The minimum value is inclusive. The output values can be that value or higher.

2. In the Maximum field, type the maximum value to use in the output values. The maximum value is exclusive. The output values are lower than that value.

3. If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates a random hash string.

Characteristics

How to configure

If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

Generates unique numeric strings of the same length as the input value.

For example, for the input value 123456, the output value would be something like 832957.

You can apply this generator only to columns that contain numeric strings.

Characteristics

How to configure

To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

By default, the generator is not consistent.

If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

If you have multiple workspaces, then it is likely that many of the workspace components and configurations are the same or similar. It can be difficult to maintain that consistency across separate, independent workspaces.

When you copy a workspace, the new workspace is completely independent of the original workspace. There is no visibility into or inheritance of changes from the original workspace.

Workspace inheritance allows you to create workspaces that are children of a selected workspace. Unlike a copy of a workspace, a child workspace remains tied to its parent workspace.

By default, a child workspace's configuration is synchronized with the configuration of the parent. In other words, any changes to the parent workspace are copied to its child workspaces. Child workspaces can also override some of the parent configuration. From the parent workspace, you can track the child workspaces and how they are customized .

For example, you might want separate workspaces for different development teams. Each team can make adjustments to suit their specific projects - such as different subsets - but inherit everything else.

What does a child workspace inherit?

By default, a child workspace inherits all of the configuration from the parent workspace, except for the following:

• Workspace name - A child workspace has its own name.

• Workspace description - A child workspace has its own description.

• Tags - A child workspace has its own tags.

• Destination database - A child workspace writes output data to its own destination database. You can copy the destination database from the parent workspace.

• Intermediate database - For upsert, a child workspace does not inherit the intermediate database.

• Webhooks - A child workspace has its own webhooks.

How parent workspace changes affect child workspaces

When you change the configuration of a parent workspace, the configuration is also updated in the child workspaces.

The exception is when a child workspace overrides the configuration. If the configuration is overridden, then the child workspace does not inherit the change.

Tonic Structural indicates on both the parent and child workspaces when the configuration is overridden.

What can a child workspace override?

A child workspace can override the following configuration items.

• Table modes - A child workspace can override the table mode for individual tables. The other tables continue to inherit the table mode that is configured in the parent workspace.

• Column generators - A child workspace can override the generator for individual columns. The other columns continue to inherit the generator that is configured in the parent workspace. For linked columns, a change to any of the linked columns overrides the inheritance for all of the columns.

• Subsetting - A child workspace can override the subsetting configuration from the parent workspace. Any change in the child workspace means that the child workspace no longer inherits any changes to the subsetting configuration from the parent workspace. For example, if you change the percentage setting on a single target table from 5 to 6, that eliminates the subsetting inheritance. The child workspace keeps the subsetting configuration that it already has, but it is not updated when the parent workspace is updated.

• Post-job scripts - A child workspace can override the post-job scripts. Any change to the post-job scripts in the child workspace means that the child workspace no longer inherits any changes to the post-job scripts configuration.

• Statistics seed - A child workspace can override the statistics seed configuration.

From each view, you can eliminate the overrides and restore the inheritance.

What must a child workspace inherit?

A child workspace cannot override the following configuration items:

• Data connector type and source database - A child workspace always uses the same source data as the parent workspace.

• Foreign keys - A child workspace always uses the same foreign key configuration as the parent workspace.

• Sensitivity designation for a column - A child workspace cannot change whether a column is marked as sensitive.

How schema changes are resolved in parent and child workspaces

For removed tables and columns, when a child workspace overrides the parent workspace configuration for the table or column, you must resolve the change in the child workspace.

If there is a conflicting change for the removed table or column in the parent workspace configuration, then regardless of whether the configuration is inherited, you must resolve that change in the parent workspace before the change is resolved for the child workspace.

For changes to column nullability or data type, you resolve the change separately in the child and parent workspaces.

You also dismiss notifications (new tables and columns) separately in the parent and child workspaces.

The Structural sensitivity scan identifies sensitive columns in source data. The scan ignores truncated tables.

When sensitivity scans run

Structural runs sensitivity scans automatically based on specific events. You can also run manual sensitivity scans on demand.

On a self-hosted instance, sensitivity scans can also run automatically at the same time each day.

Event-based sensitivity scans

Structural automatically runs a sensitivity scan when you:

• Create a completely new workspace and connect a data source.

• Change the data connection details for the source database.

• Add a file group to a file connector workspace.

A child workspace always inherits the sensitivity designations from its parent workspace.

When you copy a workspace, Structural runs a new sensitivity scan on the copy to identify sensitive columns. However, it keeps the sensitivity designation for columns that you specifically marked as sensitive or not sensitive.

Manual sensitivity scans

In addition to the automatic scans, from Privacy Hub, you can start a sensitivity scan manually.

Scheduling daily sensitivity scans

On self-hosted instances, Structural can also run scheduled daily sensitivity scans in the background.

The daily scans only run on the 10 workspaces that had the most recent activity. Activity includes:

• User-initiated updates that are included in the Protection Audit Trail.

• Data generation jobs.

By default, Structural runs the sensitivity scans each day at midnight.

To enable and configure the daily sensitivity scans, use the following environment settings. You can add these settings to the Environment Settings list on Structural Settings.

• TONIC_ENABLE_SCHEDULED_SENSITIVITY_SCAN - Boolean to indicate whether to enable the scheduled daily sensitivity scans. The default value is true. To disable the scheduled daily scan, set this to false.

• TONIC_SENSITIVITY_SCAN_HOUR - When scheduled scans are enabled, the hour at which to run the scans. The setting uses the local time zone. The value is an integer between 0 and 23, where 0 is midnight and 23 is 11:00 PM. For example, a value of 14 indicates to run the job at 2:00 PM. The default value is 0.

• TONIC_PII_SCAN_MAX_TIMEOUT_IN_MINUTES_IF_AUTOMATIC - The number of minutes after which a scheduled scan times out. By default, the scan times out after 3 minutes.

Configuring parallel processing for sensitivity scans

For improved performance, sensitivity scans can use parallel processing.

For relational databases such as PostgreSQL and SQL Server, to configure parallel processing, you use the environment setting TONIC_PII_SCAN_PARALLELISM_RDBMS. The default value is 4.

For document-based databases such as MongoDB, you use the environment setting TONIC_PII_SCAN_PARALLELISM_DOCUMENTDB. The default value is 1.

How Structural identifies sensitive values

The Structural sensitivity scan uses the following rules and processes to:

• Identify sensitive columns.

• Recommend generators for those columns. For information about applying recommended generators to columns, go to Reviewing and applying recommended generators.

• Indicate its confidence that an identified column is sensitive and is of the detected sensitivity type.

Note that this process cannot guarantee perfect precision and recall. We strongly recommend that a human reviews the sensitivity scan results and the broader dataset to ensure that nothing sensitive was missed.

Rule-based data type, column name, and value analysis - High, medium, or low confidence

To identify that a column contains sensitive information for a built-in sensitivity type, Structural looks at the data type, column name, and column values.

This part of the sensitivity scan uses regular expression matching and dictionary lookups. It produces high, medium, or low confidence detections.

When this part of the sensitivity scan determines that a column contains sensitivity data, it:

• Marks the column as sensitive

• Assigns the sensitivity type to the column

• Recommends the generator configuration for the identified sensitivity type. Note that if the recommended generator is not compatible with the column, then Structural discards the recommendation.

• Marks the sensitivity detection as high, medium, or low confidence. The confidence level is based on a calculation of how well the column matched the applicable rules.

Custom sensitivity rules - Full confidence

The sensitivity scan also looks for any columns that match custom sensitivity types that you define in your custom sensitivity rules.

Custom sensitivity rules are based on the column data type and column name. For more information about custom sensitivity rules, go to Creating and managing custom sensitivity rules.

Custom sensitivity rules always produce full confidence detections.

When a column matches a custom sensitivity rule, Structural:

• Marks the column as sensitive.

• Assigns the sensitivity rule name as the sensitivity type.

• Recommends the generator preset from the sensitivity rule.

• Marks the sensitivity detection as full confidence.

Model-based analysis - Medium and low confidence

To identify additional sensitive columns that might not be captured by the other parts of the scan, the sensitivity scan uses an artificial intelligence (AI) model. Note that the model is pre-trained. Structural does not use customer data to train the model, and it does not send any customer data externally.

This part of the scan produces medium or low confidence detections for built-in entity types.

The model considers the table and column name. If the combination of table and column name is similar in meaning to a sensitivity type that Structural has a recommended generator for, then Structural:

• Marks the column as sensitive.

• Assigns the sensitivity type to the column.

• Recommends the generator configuration for that sensitivity type.

• Uses AI to compare the table name and column name combination to the sensitivity type, and produces a semantic similarity score.

• Based on the semantic similarity score, marks the sensitivity detection as either medium or low confidence.

Downloading the sensitivity scan log

To download the log of the most recent sensitivity scan, either:

• On the workspace management view, from the download menu, select Download Sensitivity Scan Log.

• On Privacy Hub, click Reports and Logs, then select Scan Log.

The log tracks the progress of the scan.

A Tonic Structural workspace provides a context within which to configure and generate transformed data.

A workspace represents a path between the source data and the transformed output data. For example, postgres-prod-copy to postgres-staging.

A workspace includes:

• Where to find the source data to transform during data generation

• Where to write the transformed data

• The rules for the transformation

Manage workspaces

Workspace details and tools

You use the workspace management view to configure and run data generation for an individual workspace.

When you log in to Tonic Structural, it displays the workspace management view for the workspace that was selected when you logged out.

Components of the workspace management view

The workspace management view includes the following components.

Workspace information

The top left of the workspace management view provides information about the workspace, including:

• The workspace name

• When the workspace was last updated

• The user who last updated the workspace

• Whether the workspace is a child workspace

Workspace options

The top right of the workspace management view provides general options for working with the workspace, including:

• Undo and redo options for configuration changes

• The workspace share icon, to grant workspace access to other users and groups

• The workspace download menu to:

  • Download sensitivity scan and privacy reports

  • Export and import workspace configuration

• The workspace actions menu

• The Generate Data button, to start a data generation job

Workspace navigation bar

The workspace navigation bar provides access to workspace configuration options.

Displaying the workspace management view

To display the workspace management view for a workspace:

• On Workspaces view, in the Name column either:

  • Click the workspace name. The workspace management view opens to Privacy Hub.

  • Click the dropdown icon, then select a workspace management option.

• Click the search field at the top. A list of available type the name of the workspace. As you type, Tonic displays a list of matching workspaces. In the list, click the workspace name.

Collapsing and expanding the workspace heading

To reduce the amount of vertical space used by the heading of the workspace management view, you can collapse it.

To collapse the heading, click the collapse icon in the Structural heading.

When you collapse the workspace management heading:

• The workspace information is hidden. The workspace name is displayed in the search field.

• The workspace options are moved up into the Structural heading.

The workspace navigation bar remains visible.

When you collapse the heading, the collapse icon changes to an expand icon. To restore the full heading, click the expand icon.

The table list at the left of Database View contains the list of tables in the source database. You can filter the table list and assign tables modes to the tables.

Information in the table list

The table list is grouped by schema. You can expand and collapse the list of tables in each schema. This does not affect the displayed columns.

For a file connector workspace, each table corresponds to a file group.

For each table, the table list includes the following information:

• The name of the table.

• The number of columns that have an assigned generator (a generator other than Passthrough). The number does not display if none of the table columns has an assigned generator.

• The assigned table mode. The table list only shows the first letter of the table mode:

  • D = De-identify

  • S = Scale

  • T = Truncate

  • P = Preserve Destination

  • I = Incremental

For a child workspace, if the selected table mode overrides the parent workspace configuration, then the override icon displays.

To display Table View for a table, click the arrow icon to the right of the table entry.

Filtering the table list

You can filter the table list by name and by the assigned table mode. You can also filter the tables based on whether any of the columns have assigned generators.

As you filter the table list, the column list also is filtered to only include the columns for the filtered tables.

Filtering by table name

To filter the table list by name, in the filter field, begin to type text that is in the table name.

As you type, Tonic Structural filters the list to only display tables with names that contain the filter text.

Filtering by the assigned table mode

To filter the table list based on the assigned table mode:

1. Click Filters.

2. On the filter panel, check the checkbox next to each table mode to include. By default, the list includes all of the table modes. As you check and uncheck the table mode checkboxes, Structural adds and removes the associated tables from the list.

Filtering to exclude tables that have assigned generators

You can filter the table list to only display tables that have no assigned generators:

1. Click Filters.

2. On the filter panel, to only show tables that do not have assigned generators, check the No Generators Applied checkbox.

Assigning table modes to tables

The table mode determines the number of rows and columns in the destination database. For details about the available table modes and how they work, go to Table modes.

Updating a single table

To change the assigned table mode for a single table:

1. Click the table mode dropdown next to the table name.

2. From the table mode dropdown list, select the table mode.

3. For a child workspace, the table mode selection panel indicates whether the selected table mode is inherited from the parent workspace. If the child workspace currently overrides the parent workspace configuration, then to reset the table mode to the table mode that is assigned in the parent workspace, click Reset.

Updating multiple tables

To change the assigned table mode for multiple tables:

1. Check the checkbox for each table to change the table mode for. To select a continuous range of tables, click the first table in the range, then Shift-click the last table in the range. To select all of the tables in a schema, click the schema name.

2. Click Bulk Edit.

3. On the panel, click the radio button for the table mode to assign to the selected tables.

The bulk edit option on Database View allows you to configure multiple columns at the same time. From the bulk editing panel, you can:

• Mark the selected columns as sensitive or not sensitive.

• Assign a generator to the selected columns.

• Apply the recommended generator to the selected columns.

• Reset the generator configuration to the baseline. This option requires that all of the selected columns are assigned the same preset.

Depending on the column selection, you can also create a new sensitivity rule.

Displaying the bulk edit option

To select the columns and display the bulk edit option:

1. Check the checkbox next to each column to update.

2. Click Bulk Edit.

Marking the columns as sensitive or not sensitive

On the Bulk Edit panel, under Sensitivity:

• To mark the selected columns as sensitive, click Sensitive.

• To mark the selected columns as not sensitive, click Not Sensitive.

Changing the assigned generator

On the Bulk Edit panel, under Bulk Edit Applied Generator, select and configure the generator to assign to the selected columns.

Assigning the recommended generator to the columns

If any of the selected columns have a recommended generator, then on the Bulk Edit panel, the Generator recommendations found panel displays. The panel indicates the number of selected columns that have a recommendation.

To assign the recommended generators to those columns, click Apply.

Restoring the baseline configuration for the columns

For a generator preset, the baseline configuration is the configuration that is saved for that preset. The baseline configuration determines the default configuration to use when you assign the preset to a column. After you select the preset, you can override the baseline configuration.

If all of the selected columns are assigned the same preset, then to restore the baseline configuration for all of the columns, click Reset to Baseline.

Creating a sensitivity rule

You might bulk edit columns that could benefit from a custom sensitivity rule.

For example, in your data, the Widget column is in multiple tables and contains sensitive data that Structural cannot identify. You select all of the Widget columns so that you can mark them as sensitive and apply the Character Scramble generator to them.

However, a custom sensitivity rule would ensure that in the future, Widget columns are always marked as sensitive and have the Character Scramble generator recommended.

On the Bulk Edit panel, when all of the selected columns:

• Have the same data type.

• Do not have a generator assigned.

• Do not have a recommended generator.

Then Structural displays the Create a Sensitivity Rule panel, which contains the option to create a new sensitivity rule.

To create a sensitivity rule:

1. Click Create Custom Rule.

2. On the Create Custom Rule view, configure the new sensitivity rule. Structural automatically selects a data type based on the selected columns. The current workspace is used as the testing workspace to verify the columns that match the rule configuration. For details about the sensitivity rule configuration, go to .

3. When you finish configuring the new rule:

   • To both save the rule and apply the generator preset to all workspace columns that match the rule, click Save and Apply. On the confirmation panel, click Confirm Auto Apply.

   • To save the rule, but not apply the configured generator preset to matching columns, click Save.

Structural closes the sensitivity rule configuration view and returns you to Database View. It maintains the previous column selection.

If you did not apply the generator preset, then the sensitivity rule is included in the next sensitivity scan.

When you first connect to a MongoDB or Amazon DynamoDB database, Tonic Structural performs a scan to determine the available fields in each collection, the field types, and how prevalent the fields are. It performs this scan at the same time as the initial sensitivity scan.

For each collection, Structural creates a hybrid document, which is a superset of all of the fields contained in the collection documents.

Configuring the collection scan

By default, for each collection:

• The scan includes all of the documents in the collection, and continues until the scan is finished.

• Every unique path (field+data type) in the collection is added to the hybrid document.

You can change the default scan behavior. To change the scan configuration, use the following environment settings. You can add these settings manually to the Environment Settings list on Structural Settings.

Note that these settings, including settings that include MONGO in the name, apply to both MongoDB and Amazon DynamoDB.

Configuring how schemas are scanned

The following options control the number of documents that Structural scans in a collection.

These options allow you to limit the number of scanned documents when the additional documents do not add fields to the hybrid document.

For large homogenous collections, where all or most documents have the same structure, configuring these options can improve performance.

If you set both options, then the scan completes when it reaches either limit. For example, if the maximum document count is 10 and the maximum scan time is 360 seconds, then the scan completes either after 10 documents or after 360 seconds, whichever comes first.

Configuring how fields are collapsed in the hybrid document

Typically, the number of unique fields in a collection is small relative to the number of documents. However, in some cases the number of fields is similar to or greater than the number of documents. This most commonly occurs when documents have "data as keys", such as keys that are ObjectIds, UUIDs, or incrementing integers.

In these cases, adding every unique field to the hybrid document can result in a large hybrid document that has an undesirable structure.

Structural offers configuration options to "collapse" fields within the hybrid document. This shrinks the size of the hybrid document. It also allows you to assign a generator to the collapsed group instead of to each unique key.

By default, Structural does not collapse fields.

Collapsing fields when the key is an ObjectId

To enable this, set the environment setting TONIC_MONGO_OBJECT_ID_COLLAPSE_THRESHOLD to the number of ObjectId keys that an object can contain before Structural collapses the object schema into a single key.

For example, if this is 10, then any object that has 10 or more ObjectId keys is collapsed into a single key.

A negative value indicates to not collapse the keys.

The default value is -1.

Collapsing fields when the key matches a custom pattern

To enable Structural to collapse fields, you provide a regular expression to identify the fields that can be collapsed into the same field. You then configure the number of matches that must exist before Structural collapses the fields.

To configure how the fields are collapsed, use the following environment settings:

For example:

• To collapse keys that are integer values, use the regular expression [0-9]+ or \d+

• To collapse keys that are UUIDs, use the regular expression [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}

Viewing the most recent scans for each collection

On Privacy Hub, the Latest Collection Scan table shows the most recent scans on each scanned collection.

The Build Schema option runs a new scan on the collection.

Starting a collection scan

When the source database has a new collection, then on Collection View, you are prompted to run a scan either on that collection or on all collections.

This is a composite generator.

A version of the JSON Mask generator that can be used for array values.

Runs a selected generator on values that match a user-specified JSONPath.

Characteristics

How to configure

Adding a sub-generator

To assign a generator to a path expression:

1. Under Sub-generators, click Add Generator. On the sub-generator configuration panel, the Cell JSON field contains a sample value from the source database. You can use the previous and next icons to page through different values.

2. In the Path Expression field, type the JSONPath expression to identify the value to apply the generator to. To populate a path expression, you can also click a value in the Cell JSON field. Matched JSON Values shows the result from the value in Cell JSON.

3. By default, the selected generator is applied to any value that matches the expression. To limit the types of values to apply the generator to, from the Type Filter, specify the applicable types. You can select Any, or you can select any combination of String, Number, and Null.

4. From the Generator Configuration dropdown list, select the generator to apply to the path expression. You cannot select another composite generator.

5. Configure the selected generator. You cannot configure the selected generator to be consistent with another column.

6. To save the configuration and immediately add a generator for another path expression, click Save and Add Another. To save the configuration and close the add generator panel, click Save.

Managing the sub-generator list

From the Sub-Generators list:

• To edit a generator assignment, click the edit icon.

• To remove a generator assignment, click the delete icon.

• To move a generator assignment up or down in the list, click the up or down arrow.

Enabling data encryption

If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

This is a composite generator.

Applies different generators to the value conditionally based on any value in the table.

For example, a Users table contains Name, Username, and Role columns. For the Username column, you can use a conditional generator to indicate that if the value of Role is something other than Test, then use the Character Scramble generator for the Username value. For Test users, the name is not masked.

Characteristics