Only this pageAll pages
Powered by GitBook
Couldn't generate the PDF for 466 pages, generation stopped at 100.
Extend with 50 more pages.
1 of 100

Tonic Structural

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Structural AI and Agent

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Workspace management

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Configuring data generation

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Trusting the server certificate

For data connectors that support it, to indicate that Tonic Structural should trust the server certificate, toggle Trust Server Certificate to the on position.

Ensuring encryption of database authentication

For data connectors that support it, the Enable SSL/TLS setting indicates whether to encrypt the database authentication.

By default, it is in the on position. We strongly recommend that you do not turn off this setting.

Data connection settings

For each workspace, you select the connector type.

After you select the connector type, you configure:

  • Where to find the source data

  • Where to write the data generation output

  • How to make the connections

Not all connectors support all of the connection options. The workspace configuration information for each connector type lists the supported settings.

Assigning tags to a workspace

Required workspace permission: Configure workspace settings

You can associate custom tags with each workspace. Tags can help to organize and provide a quick glance into the workspace configuration.

Tags are accessible to every user that has access to the workspace.

Tags are stored in the workspace JSON, and are included in the workspace export. You can also use the API to get access to tags.

Managing tags from workspace settings

You can add and edit tags in the Tags field on the New Workspace and Workspace Settings views.

  • To add tags, enter a comma-separated list of the tags to add.

  • To remove a tag, click its delete icon.

You can also manage tags directly from Workspaces view.

To add tags to a workspace that does not currently have tags:

  1. Hover over the Tags column for the workspace.

  2. Click Add Tags.

  3. In the tag input field, type a comma-separated list of tags to apply.

  4. Press Enter

To edit the assigned tags:

  1. Click the Tags column for the workspace.

  2. In the tag input field, to remove tag, click its delete icon.

  3. To add tags, type a comma-separated list of the tags to add.

  4. To save the tag changes, press Enter

Managing workspaces that do not have an owner

Required team permission: Manage user access within the team

On the Team Settings page, the Workspaces Without an Owner tab displays the list of workspaces that do not currently have an owner.

This typically happens when the owner is removed from Structural.

These workspaces do not display on the Workspaces view.

Displaying the list

To display the list:

  1. In the user menu, click Team Settings.

  2. On the Team Settings view, click Workspaces Without an Owner.

For each workspace, the list includes:

  • The workspace name.

  • The email address of the previous owner.

You can filter the list based on the workspace name or the previous owner email address.

To filter the list, begin typing text in either the workspace name or the owner email address.

As you type, Structural updates the list to only include matching workspaces.

You can transfer one or more workspaces to a different owner.

  1. To transfer a single workspace, click its Transfer option. To transfer multiple workspaces:

    1. Check the checkbox for each workspace to transfer.

    2. Click Actions, then click Transfer.

  1. From the New owner dropdown list, select the new owner.

  2. Click Transfer.

The new owner is assigned the owner permission set for the transferred workspaces.

You can also delete workspaces.

To delete a single workspace, click its Delete option.

To delete multiple workspaces:

  1. Check the checkbox for each workspace to delete.

  2. Click Actions, then click Delete.

Creating and managing workspaces

A Tonic Structural workspace provides a context within which to configure and generate transformed data.

A workspace represents a path between the source data and the transformed output data. For example, postgres-prod-copy to postgres-staging.

A workspace includes:

  • Where to find the source data to transform during data generation

Exporting and importing the workspace configuration

You can export a workspace configuration to a JSON file, and import configuration from a workspace configuration JSON file.

For example, you might want to preserve a version of the workspace configuration before you test other changes. You can then use the exported file to restore the original configuration.

Or you might want to use a script to make changes to an exported configuration file. You can then import the updated file to update the workspace configuration.

The workspace JSON configuration file includes the following information:

  • Sensitivity designations that you assigned to columns

Viewing the list of jobs

Jobs view displays the list of jobs for the workspace.

To display Jobs view:

  • On the workspace management view, in the workspace navigation bar, click Jobs.

  • On Workspaces view, from the dropdown menu in the Name column, select Jobs.

On Jobs

Canceling a job

You can cancel Queued or Running jobs.

For jobs with those statuses, the rightmost column in the job list contains a cancel icon.

To cancel the job, click the icon.

Using AI to troubleshoot failed jobs

For the the following types of jobs:

  • Sensitivity scans

  • Data generation jobs

  • Upsert jobs

.
.

Managing tags from Workspaces view

Assigning tags

Editing the assigned tags

Collection scans

You can ask for AI help to troubleshoot the failure.

On the job details, to start the AI troubleshooting, click Ask AI.

Structural sends the job log information to the LLM, which returns an analysis of the failure and suggested next steps.

The diagnosis includes an Open in Agent chat option to allow you to use the Structural Agent to continue the conversation.

Required workspace permission: Download job logs

Connection type Select the data connector to use for the workspace data.

Data locations Configure the data source and destination information.

Authentication encryption Use SSL/TLS to encrypt authentication.

Trust the server certificate Indicate whether to trust the server certificate for authentication.

Provide custom certificates Provide your own client and root certificates.

Use connection tunneling Use SSH or Tailscale tunneling for database connection.

Connection testing Test your data connections.

Filtering the list

Transferring workspaces to a different owner

Deleting workspaces

Workspaces Without an Owner list on the Team Settings view
Actions menu to select an action to perform on selected workspaces
Transfer Workspaces panel to select the user to transfer the workspaces to
Actions menu to select an action to perform on selected workspaces
Job list with a Running job that can be canceled
Where to write the transformed data
  • The rules for the transformation

  • Assigned table modes
  • Assigned column generators

  • Subsetting configuration

  • Post-job script configuration

  • To export the workspace configuration, either:

    • On the workspace management view, from the download menu, select Export Workspace.

    Download menu for a workspace
    • On Workspaces view, click the actions menu for the workspace, then select Export.

    Workspace options column and dropdown list

    When you export a child workspace, the exported workspace does not retain any of the inheritance information. The exported information is the same for all exported workspaces.

    To import a workspace configuration file:

    1. Select the import option. Either:

      • On the workspace management view, from the download menu, select Import Workspace.

      • On Workspaces view, click the actions menu for the workspace, then select Import.

    2. On the Import Workspace dialog, to select the file to import, click Browse.

    3. After you select the file, click Import.

    When you import a workspace configuration into a child workspace, Tonic Structural only updates the configuration that can be overridden. If a configuration must be inherited from the parent workspace, then it is not affected by the imported configuration. For more information, go to About workspace inheritance.

    Required workspace permission: Export and import workspace

    Information in the exported file

    Exporting the workspace configuration

    Importing a workspace configuration file

    view, you use the tabs to filter the jobs based on the job type.

    The possible tabs are:

    • All Jobs - Always displayed. Contains all of the workspace jobs across all job types. When you first display Jobs view, All Jobs is selected.

    • Data Generation - Always displayed. Includes the following types of jobs:

      • Data generation

      • Data pipeline generation

      • Containerized data generation

      • Upsert data generation

      • Upsert

    • Sensitivity Scan - Always displayed. Lists the sensitivity scans for the workspace.

    • Collection Scan - Displays for MongoDB and Amazon DynamoDB workspaces. Lists the collection scans on the source data.

    • Schema - Lists the schema retrieval jobs for the workspace.

    • Statistics - Displays for workspaces that use Spark-based data. Lists the SDK table statistics jobs.

    The list is always sorted by the submission date, with the most recent jobs at the top of the list.

    For each job, the job list includes the following information:

    • Job ID - The identifier of the job. To copy the job identifier, click the icon at the left of the row.

    • Type - The type of job.

    • Status - The current status of the job, and how long ago the job reached that status. When you hover over the status, a tooltip displays the actual timestamp for the status change, and the length of time that the job ran. For queued jobs, to display a panel with information about why the job is queued, click the status value.

    • Submitted - The date and time when the job was submitted.

    • Completed - The date and time when the job finished running.

    To filter the list by the job status:

    Job status filter options for Jobs view
    1. Click the filter icon in the Status column heading. The status panel displays all of the statuses that are currently in the list. For example, if there are no Queued jobs, then the Queued status is not in the list. By default, all of the statuses are included, and none of the checkboxes are checked.

    2. To only include jobs that have specific statuses, check the checkbox next to each status to include. Checking all of the checkboxes has the same effect as unchecking all of the checkboxes.

    To filter the list by the job identifier, in the filter field, provide the full identifier.

    Displaying Jobs view

    Filtering the jobs by type

    Jobs view

    Information in a jobs list

    Filtering jobs by job status

    Filtering jobs by identifier

    Data flow and privacy in Structural AI

    Overall interaction flows with an LLM

    The following diagrams show the general interaction between Structural and an LLM based on the Structural deployment.

    Structural Cloud

    On Structural Cloud, all users use the same LLM, which is managed by Tonic.ai.

    Structural Cloud interaction with the LLM

    Self-hosted instances that use the hosted LLM

    Self-hosted instances can choose to use the LLM that Tonic.ai maintains.

    For each customer, requests to the LLM are routed through a SaaS proxy to the hosted LLM in the appropriate region.

    Self-hosted Structural interaction with the hosted LLM

    Self-hosted instances that use their own LLM connection

    Self-hosted instances can also establish their own connection to an LLM.

    Self-hosted Structural interaction with a custom LLM connection

    Flows for AI features

    The following flows show the request and results for specific Structural AI features.

    LLM-based sensitivity detection

    Here is a high-level flow of LLM-based sensitivity detection.

    High-level flow for LLM-based sensitivity detection

    To complete the analysis, Structural sends the following to the LLM to complete the analysis:

    • Database schema.

    • By default, sample source data values.

    • LLM-based sensitivity rules

    To manage LLM-based sensitivity detection:

    • You can exclude the sample data, and only send the schema information.

    • You can disable it entirely.

    For more information go to the information on configuring AI-based functionality for and on .

    Here is a high-level overview of a Structural Agent interaction:

    When it sends your prompts to the LLM, to provide context for tasks such as recommending generators, Structural also sends:

    • The database schema.

    • By default, representative data samples.

    You can configure Structural to never send any sample data to the LLM. This configuration would also prevent data from being sent as part of the LLM-based sensitivity detection.

    For more information go to the information on configuring AI-based functionality for and on .

    The and generators both include a prompt field to use to configure the generator.

    Structural sends the prompt to the LLM. The LLM returns the list of values or the template.

    When a data connection test fails, Structural provides an option to use the LLM to troubleshoot the issue.

    Structural sends to the LLM the connection details and the error. The LLM analyzes the information and returns suggestions for how to resolve the connection issue.

    Structural stores your Structural Agent conversations on your computer, in your local browser storage.

    On Structural Cloud and on our hosted LLM, our usage of information specifically follows the usage policy for .

    For all LLM-based interactions, input prompts and generated responses are only used for workspace data analysis and configuration.

    Prompts and responses are never used to train the underlying models.

    However, as a best practice, we recommend that you do not include highly sensitive values in your manual prompts.

    Job types and statuses

    Job types

    Tonic Structural runs the following types of jobs on a workspace:

    • Sensitivity scans - Analyze the source database to identify sensitive data.

    • Collection scans - Analyze the source data for a MongoDB workspace to determine the available fields in each collection, the field types, and how prevalent the fields are.

    • Schema retrieval jobs - Refresh the cached version of the source database schema.

    • Data generation, data pipeline generation, and containerized generation jobs - Generate the destination data from the source data.

    • Upsert data generation jobs - Generate the intermediate database from the source database.

    • Upsert jobs - Use data from the intermediate database to add new rows to and update changed rows in the destination database. If the migration process is enabled, then it is a step in the upsert job.

    • SDK table statistics jobs - Only run when you use the SDK to generate data in a Spark workspace, and the assigned generators require the statistics.

    A job can have one of the following statuses:

    • Queued - The job is queued to run, but has not yet started. A job is queued for one of the following reasons:

      • Another job is currently running on the same workspace. For example, you cannot run a sensitivity scan and a data generation, or multiple data generations, at the same time on the same workspace. This is true regardless of the number of workers on the instance. On Structural Cloud, there is also a limit on the number of concurrent running jobs for each organization. When that maximum is reached, a new job remains queued until a current running job completes.

      • There isn't an available worker on the instance to run the job. A Structural instance with one worker can only run one job at a time. If a job from one workspace is currently running, a job from another workspace cannot start until the first job is finished.

    Each of these statuses has a corresponding "with warnings" status. For example, Running with warnings, Completed with warnings. A "with warnings" status indicates that the job had at least one warning at the time of the request.

    Viewing workspace jobs and job details

    You can view a list of jobs for the workspace, and view details for individual jobs. From the job details, you can download job information.

    You can also use the Structural Agent to get information about jobs and diagnose failed jobs. For example:

    How many data generation jobs have run for this workspace?

    How many columns were de-identified during the most recent successful data generation?

    When was the most recent sensitivity scan for this workspace? How many sensitive columns did it identify?

    Why did the most recent data generation job fail? What's the solution to enable the job to complete successfully?

    Required workspace permission: Download job logs

    To download diagnostic logs, you must have the Enable diagnostic logging team permission.

    Viewing details for a selected job

    For jobs other than Queued jobs, you can display details about the workspace and the job progress.

    Displaying the job details view

    From the Jobs view, to display the details for a job, click the job row.

    Job details page for a data generation job

    Information on the job details view

    Workspace information

    The left side of the job details view contains the workspace information.

    For a sensitivity scan, the workspace information is limited to the owner, database type, and worker version.

    For a data generation job, the workspace information also includes:

    • Whether subsetting, post-job scripts, or webhooks are used.

    • The number of schemas, tables, and columns in the source database.

    • The number of schemas, tables, and columns in the destination database.

    The Job Log tab shows the start date, start time, and duration of the job, followed by the list of job process steps.

    For data generation jobs, the Privacy Report tab displays the number of at-risk, protected, and not sensitive columns in the source database.

    At-risk columns contain sensitive data, but still have Passthrough as the assigned generator.

    Protected columns have an assigned generator other than Passthrough.

    Not sensitive columns have Passthrough as the assigned generator, but do not contain sensitive data.

    The job identifier is a unique identifier for the job. To copy the job identifier, either:

    • From Jobs view, click the copy () icon in the leftmost column.

    • From the job details view, click Copy Job ID.

    Viewing a Gantt chart of a data generation job flow

    This feature is currently in beta.

    From the job details view, you can display a Gantt chart that shows the flow of a data generation job over time. The chart can help you to understand the different steps of a data generation job and how long it takes Structural to complete each step.

    Note that this option is only available for data generation jobs that use the newer data generation process. For more information, go to . Data generation jobs that use the older process do not produce the Gantt chart.

    To display the chart, click Reports and Logs, then select View Gantt.

    The Job Visualization page displays the Gantt chart of the job progress.

    Job Visualization page with a Gantt chart of the job flow

    About workspace inheritance

    If you have multiple workspaces, then it is likely that many of the workspace components and configurations are the same or similar. It can be difficult to maintain that consistency across separate, independent workspaces.

    When you copy a workspace, the new workspace is completely independent of the original workspace. There is no visibility into or inheritance of changes from the original workspace.

    Workspace inheritance allows you to create workspaces that are children of a selected workspace. Unlike a copy of a workspace, a child workspace remains tied to its parent workspace.

    By default, a child workspace's configuration is synchronized with the configuration of the parent. In other words, any changes to the parent workspace are copied to its child workspaces. Child workspaces can also override some of the parent configuration. From the parent workspace, you can track the child workspaces and how they are customized .

    For example, you might want separate workspaces for different development teams. Each team can make adjustments to suit their specific projects - such as different subsets - but inherit everything else.

    By default, a child workspace inherits all of the configuration from the parent workspace, except for the following:

    About the workspace management view

    You use the workspace management view to configure and run data generation for an individual workspace.

    When you log in to Tonic Structural, it displays the workspace management view for the workspace that was selected when you logged out.

    The workspace management view includes the following components.

    The top left of the workspace management view provides information about the workspace, including:

    • The workspace name

    Workspaces view View, filter, and sort the list of workspaces that you have access to.

    Create, edit, and delete workspaces

    Workspace management view Provides access to workspace configuration and generation tools.

    Workspace inheritance Create child workspaces that inherit source data and configuration from their parent workspace.

    Assign workspace tags Use tags to help organize and identify workspaces.

    Export and import workspace configuration

    Manage workspaces without an owner Transfer workspaces to a different owner, or delete the workspaces.

    To view information about why a job is queued, click the status value.
  • Running - The job is in progress.

  • Canceled - The job is canceled.

  • Completed - The job completed successfully.

  • Failed - The job failed to complete.

  • Job statuses

    Job types and statuses Lists the types of jobs and the possible job statuses.

    View the job list Display the Jobs view and filter the job list by type and status.

    View job details Display the details for a job.

    Use AI for job troubleshooting Use an LLM to analyze failed jobs and provide suggested next steps to resolve the issues.

    Cancel a job Cancel a queued or running job.

    Download job information Download logs, the Privacy Report, and other job information.

    View a data generation Gantt flow Display a Gantt chart of the flow for a data generation job.

    Structural Agent conversations

    Generator configuration

    Data connection troubleshooting

    How Structural stores Agent conversations

    How the LLM uses information

    Structural Cloud
    self-hosted instances
    Structural Cloud
    self-hosted instances
    Custom Categorical
    Text Composition
    Anthropic on Amazon Bedrock
    High-level flow for Structural Agent interactions
    Flow for LLM-assisted generator configuration
    Flow for LLM-assisted data connection troubleshooting

    Job Log

    Privacy Report

    Copying the job identifier

    Privacy Report tab on the job details page

    Workspace name - A child workspace has its own name.

  • Workspace description - A child workspace has its own description.

  • Tags - A child workspace has its own tags.

  • Destination database - A child workspace writes output data to its own destination database. You can copy the destination database from the parent workspace.

  • Intermediate database - For upsert, a child workspace does not inherit the intermediate database.

  • Webhooks - A child workspace has its own webhooks.

  • When you change the configuration of a parent workspace, the configuration is also updated in the child workspaces.

    The exception is when a child workspace overrides the configuration. If the configuration is overridden, then the child workspace does not inherit the change.

    Tonic Structural indicates on both the parent and child workspaces when the configuration is overridden.

    A child workspace can override the following configuration items.

    • Schema management settings - A child workspace can override the settings to determine how to respond to schema changes and whether to cache the source database schema.

    • Table modes - A child workspace can override the table mode for individual tables. The other tables continue to inherit the table mode that is configured in the parent workspace.

    • Column generators - A child workspace can override the generator for individual columns. The other columns continue to inherit the generator that is configured in the parent workspace. For linked columns, a change to any of the linked columns overrides the inheritance for all of the columns.

    • Subsetting - A child workspace can override the subsetting configuration from the parent workspace. Any change in the child workspace means that the child workspace no longer inherits any changes to the subsetting configuration from the parent workspace. For example, if you change the percentage setting on a single target table from 5 to 6, that eliminates the subsetting inheritance. The child workspace keeps the subsetting configuration that it already has, but it is not updated when the parent workspace is updated.

    • Post-job scripts - A child workspace can override the post-job scripts. Any change to the post-job scripts in the child workspace means that the child workspace no longer inherits any changes to the post-job scripts configuration.

    • Statistics seed - A child workspace can override the .

    From each view, you can eliminate the overrides and restore the inheritance.

    A child workspace cannot override the following configuration items:

    • Data connector type and source database - A child workspace always uses the same source data as the parent workspace.

    • Advanced workspace overrides - A child workspace cannot add, remove, or change any advanced workspace overrides. It always uses the same overrides as the parent workspace.

    • Foreign keys - A child workspace always uses the same foreign key configuration as the parent workspace.

    • Sensitivity designation for a column - A child workspace cannot change whether a column is marked as sensitive.

    For removed tables and columns, when a child workspace overrides the parent workspace configuration for the table or column, you must resolve the change in the child workspace.

    If there is a conflicting change for the removed table or column in the parent workspace configuration, then regardless of whether the configuration is inherited, you must resolve that change in the parent workspace before the change is resolved for the child workspace.

    For changes to column nullability or data type, you resolve the change separately in the child and parent workspaces.

    You also dismiss notifications (new tables and columns) separately in the parent and child workspaces.

    Required license: Enterprise

    What does a child workspace inherit?

    How parent workspace changes affect child workspaces

    What can a child workspace override?

    What must a child workspace inherit?

    How schema changes are resolved in parent and child workspaces

    When the workspace was last updated

  • The user who last updated the workspace

  • Whether the workspace is a child workspace

  • The top right of the workspace management view provides general options for working with the workspace, including:

    Workspace options
    • The workspace share icon, to grant workspace access to other users and groups

    • The workspace download menu to:

      • Download sensitivity scan and privacy reports

    • The history icon, to

    • The workspace actions menu

    • The Generate Data button, to

    The workspace navigation bar provides access to workspace configuration options.

    Workspace navigation bar

    To display the workspace management view for a workspace:

    • On Workspaces view, in the Name column either:

      • Click the workspace name. The workspace management view opens to Privacy Hub.

      • Click the dropdown icon, then select a workspace management option.

    Workspace tools menu
    • Click the search field at the top. A list of available type the name of the workspace. As you type, Tonic displays a list of matching workspaces. In the list, click the workspace name.

    Workspace search

    To reduce the amount of vertical space used by the heading of the workspace management view, you can collapse it.

    To collapse the heading, click the collapse icon in the Structural heading.

    Workspace heading with collapse option highlighted

    When you collapse the workspace management heading:

    • The workspace information is hidden. The workspace name is displayed in the search field.

    • The workspace options are moved up into the Structural heading.

    The workspace navigation bar remains visible.

    When you collapse the heading, the collapse icon changes to an expand icon. To restore the full heading, click the expand icon.

    Collapsed workspace heading with expand option highlighted

    Components of the workspace management view

    Workspace information

    Workspace management view for a workspace
    Workspace information

    Workspace options

    Workspace navigation bar

    Displaying the workspace management view

    Collapsing and expanding the workspace heading

    Managing the Structural Agent and Agent chat conversations

    How chat conversations are stored

    Tonic Structural stores chat conversations in browser storage.

    Using the AI Agent tab

    The AI Agent tab provides a full-page view of the Structural Agent chat.

    AI Agent tab on the workspace management tab

    Using the AI Agent chat panel

    In addition to the AI Agent tab, you can use the AI Agent chat panel, which allows you to interact with the Structural Agent while you view Structural pages.

    Available display types

    The chat panel supports the following display options.

    Side-by-side

    With the Side-by-side option, the agent chat panel displays next to the Structural page.

    To accommodate the chat panel, the Structural page becomes narrower.

    You can display the chat panel on the left or right side of the screen.

    Structural Agent using the Side-by-side display

    Overlay

    With the Overlay option, the agent chat panel displays on top of the Structural page.

    You can display the chat panel on the left or right side of the screen.

    From the chat panel, to select the display type:

    1. Click the chat panel options menu.

    2. In the menu, click the display type.

    To also select the Agent chat panel location, click the location option under the display type.

    For example, if you select the Move to right side option under Side-by-side, then the chat panel both changes to the side-by-side view and displays at the right.

    If you don't select a specific location, then the chat panel uses the most recently selected location.

    To change the display location:

    1. Click the chat panel options menu.

    2. For the selected display option:

      • To move the chat panel to the left, click Move to left side.

      • To move the chat panel to the right, click Move to right side.

    To hide the chat panel, either:

    • Click the hide icon at the top right.

    • In the options menu, click Hide.

    To display the chat panel, click the chat panel icon. The icon displays at the bottom right of each Structural page.

    To adjust the chat panel width, click and drag the panel border.

    The Structural Agent chat remains in place across the entire Structural application.

    You can read information about data from any workspace that you have access to. For example, you can get a summary list of workspaces organized by data connector, identify unprotected columns in any workspace, or find the workspace with the most recent failed job.

    However, you can only perform configuration or generation actions based on the current context. For example, you can only change the configuration for a workspace if you are currently in the workspace management view for that workspace. The Structural Agent can navigate to the workspace that you want to work with.

    As you navigate to different workspaces or to views outside of a workspace, the Structural Agent chat notes the change in context, and provides links to allow you to navigate to each context.

    You can create multiple conversations within the Structural Agent chat. For example, you might start a new conversation when you change to a different workspace.

    From the Structural Agent chat, to start a new conversation, click the conversation dropdown, then select New Conversation.

    Structural automatically assigns a name to each new conversation, based on the current context.

    To assign a new name to the current conversation:

    1. Click the conversation dropdown, then click Rename.

    2. In the name field, provide the new name.

    3. Click Save.

    To change to a different conversation, from the conversation dropdown, select the conversation to change to.

    To delete the current conversation:

    1. Click the conversation dropdown, then click Delete.

    2. On the confirmation panel, click Delete.

    Structural deletes the current conversation and switches you to the first available conversation.

    When you delete a conversation, Structural also removes it from the browser storage.

    While the Agent is working, to stop the response and cancel the action, in the chat prompt, click the stop response icon.

    Before it performs an action to modify the workspace, the Agent takes snapshots of the current workspace configuration.

    To undo an action, or to revert to a previous point from the current conversation, prompt the Agent.

    For example:

    • Undo that action.

    • Revert the generator assignments for the datetime columns.

    • Go back to before we assigned generators to the name columns.

    If needed, the Agent can ask you to select a snapshot to revert to.

    To redo an action, cick the redo icon.

    A response from the Agent might be useful elsewhere. For example, you can prompt the Agent to generate Structural API calls, or the Agent might return a useful summary of the data and its status.

    To copy the most recent Agent response, click the copy icon.

    You can provide basic positive or negative feedback to indicate the helpfulness of a response. The Agent then uses that feedback to adjust its future responses in the current chat.

    To provide positive feedback, click the thumbs up icon.

    To provide negative feedback, click the thumbs down icon.

    Creating, editing, or deleting a workspace

    Creating a workspace

    When you create a new workspace, you can either:

    • Create a completely new workspace.

    • Create a copy of an existing workspace. The copy initially uses the configuration from the original workspace. After the copy is created, it is completely independent from the original workspace.

    • Create a child of an existing workspace. Child workspaces inherit configuration from the parent workspace. They continue to be updated automatically when the parent workspace is updated. For more information, go to .

    You can also view this .

    When you create a workspace, a new chat starts for the workspace.

    To create a completely new workspace, on Workspaces view, click Create Workspace > New Workspace.

    For information about the configuration settings for a workspace, go to .

    On the new workspace view, the carousel at the bottom left includes an option to create a workspace that uses sample data that we provide. On the carousel, click the right arrow until the sample dataset option is selected.

    To create a workspace based on an existing workspace, either:

    • On the workspace management view of the workspace to copy, from the workspace actions menu, select Duplicate Workspace.

    • On Workspaces view, click the actions menu for the workspace, then select Duplicate Workspace.

    When you create a copy of a workspace, the copy initially inherits the following workspace configuration:

    • Source and destination database connections

    • Sensitivity designations, including manual designations that override the sensitivity scan results

    • Table mode assignments

    • Generator configuration

    You can create a workspace that is a child of an existing workspace. You cannot create a child workspace of another child workspace.

    The parent workspace must have a source database configured. You cannot create a child workspace from a workspace that uses the Databricks, self-managed Spark cluster, or MongoDB data connector.

    To create a child workspace from within a specific workspace, either:

    • On Workspaces view, click the actions menu for the parent workspace, then select Create Child Workspace.

    • On the workspace management view, from the workspace actions menu, select Create Child Workspace.

    On the new child workspace view, Parent Workspace identifies the parent workspace.

    To create a child workspace from outside of a specific workspace:

    • On Workspaces view, click Create Workspace > Child Workspace.

    • On the new workspace view, in the carousel at the bottom left, click to navigate to the Create a child workspace view, then click Create child workspace.

    When you use one of these options to create a child workspace, then Parent Workspace is not populated.

    From the Parent Workspace dropdown list, select the parent workspace for the new child workspace.

    To edit the configuration for an existing workspace, either:

    • On the workspace management view:

      • On the workspace navigation bar, click Workspace Settings.

      • From the workspace actions menu, select Workspace Settings.

    You can delete workspaces that you no longer need.

    You cannot delete a parent workspace. You must first delete all of its child workspaces.

    To delete a workspace:

    • On the workspace management view, from the workspace actions menu, select Delete Workspace.

    • On the Workspaces view, click the actions menu for the workspace, then select Delete.

    • On the Workspace Settings view, click Delete Workspace.

    Viewing and configuring tables

    The table list at the left of Database View contains the list of tables in the source database. You can filter the table list and assign tables modes to the tables.

    You can use the Structural Agent to filter Database View to a specific table. For example:

    • Display the products table on Database View.

    • Show me the stores and vendors tables on Database View.

    Information in the table list

    The table list is grouped by schema. You can expand and collapse the list of tables in each schema. This does not affect the displayed columns.

    For a workspace, each table corresponds to a file group.

    For each table, the table list includes the following information:

    • The name of the table.

    • The number of columns that have an assigned generator (a generator other than Passthrough). The number does not display if none of the table columns has an assigned generator.

    • The . The table list only shows the first letter of the table mode:

      • D = De-identify

    For a child workspace, if the selected table mode overrides the parent workspace configuration, then the override icon displays.

    To display for a table, click the arrow icon to the right of the table entry.

    You can filter the table list and . You can also filter the tables based on .

    As you filter the table list, the column list also is filtered to only include the columns for the filtered tables.

    To filter the table list by name, in the filter field, begin to type text that is in the table name.

    As you type, Tonic Structural filters the list to only display tables with names that contain the filter text.

    To filter the table list based on the assigned table mode:

    1. Click Filters.

    2. On the filter panel, check the checkbox next to each table mode to include. By default, the list includes all of the table modes. As you check and uncheck the table mode checkboxes, Structural adds and removes the associated tables from the list.

    You can filter the table list to only display tables that have no assigned generators:

    1. Click Filters.

    2. On the filter panel, to only show tables that do not have assigned generators, check the No Generators Applied checkbox.

    The table mode determines the number of rows and columns in the destination database. For details about the available table modes and how they work, go to .

    To change the assigned table mode for a single table:

    1. Click the table mode dropdown next to the table name.

    2. From the table mode dropdown list, select the table mode.

    3. For a child workspace, the table mode selection panel indicates whether the selected table mode is inherited from the parent workspace. If the child workspace currently overrides the parent workspace configuration, then to reset the table mode to the table mode that is assigned in the parent workspace, click Reset.

    To change the assigned table mode for multiple tables:

    1. Check the checkbox for each table to change the table mode for. To select a continuous range of tables, click the first table in the range, then Shift-click the last table in the range. To select all of the tables in a schema, click the schema name.

    2. Click Bulk Edit.

    3. On the panel, click the radio button for the table mode to assign to the selected tables.

    Configuring multiple columns

    The bulk edit option on Database View allows you to configure multiple columns at the same time. From the bulk editing panel, you can:

    • Mark the selected columns as sensitive or not sensitive.

    • Assign a generator to the selected columns.

    • Apply the recommended generator to the selected columns.

    • Reset the generator configuration to the baseline. This option requires that all of the selected columns are assigned the same preset.

    Depending on the column selection, you can also create a new sensitivity rule.

    To select the columns and display the bulk edit option:

    1. Check the checkbox next to each column to update.

    2. Click Bulk Edit.

    On the Bulk Edit panel, under Sensitivity:

    • To mark the selected columns as sensitive, click Sensitive.

    • To mark the selected columns as not sensitive, click Not Sensitive.

    On the Bulk Edit panel, under Bulk Edit Applied Generator, select and configure the generator to assign to the selected columns.

    If any of the selected columns have a recommended generator, then on the Bulk Edit panel, the Generator recommendations found panel displays. The panel indicates the number of selected columns that have a recommendation.

    To assign the recommended generators to those columns, click Apply.

    For a generator preset, the baseline configuration is the configuration that is saved for that preset. The baseline configuration determines the default configuration to use when you assign the preset to a column. After you select the preset, you can override the baseline configuration.

    If all of the selected columns are assigned the same preset, then to restore the baseline configuration for all of the columns, click Reset, then select Reset to Generator Preset.

    You might bulk edit columns that could benefit from a custom sensitivity rule.

    For example, in your data, the Widget column is in multiple tables and contains sensitive data that Structural cannot identify. You select all of the Widget columns so that you can mark them as sensitive and apply the Character Scramble generator to them.

    However, a custom sensitivity rule would ensure that in the future, Widget columns are always marked as sensitive and have the Character Scramble generator recommended.

    On the Bulk Edit panel, when all of the selected columns:

    • Have the same data type.

    • Do not have a generator assigned.

    • Do not have a recommended generator.

    Then Structural displays the Create a Sensitivity Rule panel, which contains the option to create a new sensitivity rule.

    To create a sensitivity rule:

    1. Click Create Custom Rule.

    2. On the Create Custom Rule view, configure the new sensitivity rule. Structural automatically selects a data type based on the selected columns. The current workspace is used as the testing workspace to verify the columns that match the rule configuration. For details about the sensitivity rule configuration, go to .

    3. When you finish configuring the new rule:

    Structural closes the sensitivity rule configuration view and returns you to Database View. It maintains the previous column selection.

    If you did not apply the generator preset, then the sensitivity rule is included in the next sensitivity scan.

    Performing scans on collections

    When you first connect to a database, Tonic Structural performs a scan to determine the available fields in each collection, the field types, and how prevalent the fields are. It performs this scan at the same time as the initial sensitivity scan.

    For each collection, Structural creates a hybrid document, which is a superset of all of the fields contained in the collection documents.

    By default, for each collection:

    • The scan includes all of the documents in the collection, and continues until the scan is finished.

    • Every unique path (field+data type) in the collection is added to the hybrid document.

    Downloading job information

    For workspaces that are configured to write destination data to a container repository, the Jobs view also provides access to the generated artifacts. For more information, go to .

    For all jobs, the job logs provide detailed information about the job processing. Tonic.ai support might request the job logs to help diagnose issues.

    For upsert jobs where the migration process is enabled, and you configured the GET Schema Change Logs endpoint, the upsert job logs include the migration process logs.

    You can download the job logs from the Jobs view or the job details view. The download includes up to 1MB of log entries.

    On the Jobs view, to download the logs for a job, click the download icon in the rightmost column.

    On the job details view, to download the logs for a job, click Reports and Logs, then select Job Logs.

    Business Name

    Generates a random company name-like string.

    Array JSON Mask

    This is a .

    A version of the generator that can be used for array values.

    Runs a selected generator on values that match a user-specified .

    Array Regex Mask

    This is a .

    A version of the generator that can be used for array values.

    Uses regular expressions to parse strings and replace specified substrings with the output of specified generators. The parts of the string to replace are specified inside unnamed top-level capture groups.

    statistics seed configuration

    Subsetting configuration

  • Post-job scripts

  • On Workspaces view, click the actions menu for the workspace, then select Workspace Settings.

    Creating a completely new workspace

    Required team permission: Create workspaces

    Creating a copy of a workspace

    Required workspace permission: Copy workspace (in the workspace to copy)

    Or

    Required team permission: Copy any workspace

    Creating a child workspace

    Required license: Enterprise

    Required workspace permission: Create child workspaces (in the parent workspace)

    From within a specific workspace

    From the Workspaces list or New Workspace view

    Editing a workspace

    Required workspace permission: Configure workspace settings

    Deleting a workspace

    Required workspace permission: Delete workspace

    About workspace inheritance
    video overview of how to create a workspace
    Structural Agent
    Workspace configuration settings
    New Workspace option to create an example workspace that uses sample data
    Workspace actions menu
    Workspace options column and dropdown list
    New Child Workspace view for a child workspace created directly from a
    Create Child Workspace option on the New Workspace view
    New Child Workspace view for a new child workspace without a parent selected

    T = Truncate

  • P = Preserve Destination

  • I = Incremental

  • Filtering the table list

    Filtering by table name

    Filtering by the assigned table mode

    Filtering to exclude tables that have assigned generators

    Assigning table modes to tables

    Required workspace permission: Assign table modes

    Updating a single table

    Updating multiple tables

    file connector
    assigned table mode
    Table View
    by name
    by the assigned table mode
    whether any of the columns have assigned generators
    Table modes
    Table list on Database View
    Table that overrides the configuration from the parent workspace
    Filtering the table list by table name
    Filtering the table list by table mode
    Assigning a table mode to a single table
    Assigning a table mode to multiple tables
    Export and import workspace configuration
    display the Version History
    start a data generation job

    Selecting the chat panel display type

    Selecting the chat panel display location

    Hiding and displaying the chat panel

    Adjusting the chat panel width

    Identifying the chat context

    Starting and managing chat conversations

    Starting a new conversation

    Renaming the current conversation

    Switching to a different conversation

    Deleting the current conversation

    Stopping an action

    Undoing and redoing an action

    Copying the most recent response from the Agent

    Providing feedback to the Agent

    Structural Agent using the Overlay display
    Agent options menu with the display types
    Icon to show the chat panel
    Conversation menu for the Structural Agent
    Stop response icon in the Structural Agent chat prompt
    Redo icon for the Structural Agent
    Copy response icon for the Structural Agent
    To both save the rule and apply the generator preset to all workspace columns that match the rule, click Save and Apply. On the confirmation panel, click Confirm Auto Apply.
  • To save the rule, but not apply the configured generator preset to matching columns, click Save.

  • Displaying the bulk edit option

    Marking the columns as sensitive or not sensitive

    Required workspace permission: Configure column sensitivity

    Changing the assigned generator

    Required workspace permission: Configure column generators

    Assigning the recommended generator to the columns

    Required workspace permission: Configure column generators

    Restoring the baseline configuration for the columns

    Required workspace permission: Configure column generators

    Creating a sensitivity rule

    Required license: Enterprise

    Required global permission: Create and manage sensitivity rules

    Bulk Edit panel to update multiple columns
    Bulk Edit panel with the option to create a sensitivity rule
    You can change the default scan behavior. To change the scan configuration, use the following environment settings. You can add these settings manually to the Environment Settings list on Structural Settings.

    The following options control the number of documents that Structural scans in a collection.

    These options allow you to limit the number of scanned documents when the additional documents do not add fields to the hybrid document.

    For large homogenous collections, where all or most documents have the same structure, configuring these options can improve performance.

    TONIC_DOCUMENT_SCAN_MAX_DOCS_COUNT

    The maximum number of documents to scan for each schema in a collection. For example, if this is 10, then Structural scans up to 10 documents, and ignores the remaining documents. When this value is empty, Structural scans all of the documents.

    TONIC_DOCUMENT_SCAN_MAX_TIME_SECONDS

    The maximum amount of time in seconds to scan a schema. For example, if this is 360, then Structural scans a schema for up to 360 seconds. When this value is empty, Structural continues the scan until it is complete. You can override this setting in an individual workspace.

    If you set both options, then the scan completes when it reaches either limit. For example, if the maximum document count is 10 and the maximum scan time is 360 seconds, then the scan completes either after 10 documents or after 360 seconds, whichever comes first.

    Typically, the number of unique fields in a collection is small relative to the number of documents. However, in some cases the number of fields is similar to or greater than the number of documents. This most commonly occurs when documents have "data as keys", such as keys that are ObjectIds, UUIDs, or incrementing integers.

    In these cases, adding every unique field to the hybrid document can result in a large hybrid document that has an undesirable structure.

    Structural offers configuration options to "collapse" fields within the hybrid document. This shrinks the size of the hybrid document. It also allows you to assign a generator to the collapsed group instead of to each unique key.

    By default, Structural does not collapse fields.

    To enable this, set the environment setting TONIC_MONGO_OBJECT_ID_COLLAPSE_THRESHOLD to the number of ObjectId keys that an object can contain before Structural collapses the object schema into a single key.

    For example, if this is 10, then any object that has 10 or more ObjectId keys is collapsed into a single key.

    A negative value indicates to not collapse the keys.

    The default value is -1.

    To enable Structural to collapse fields, you provide a regular expression to identify the fields that can be collapsed into the same field. You then configure the number of matches that must exist before Structural collapses the fields.

    To configure how the fields are collapsed, use the following environment settings:

    TONIC_DOCUMENT_COLLAPSE_FIELDS_REGEX

    The regular expression that identifies the fields that can be collapsed into a single field. By default, this value is empty.

    TONIC_DOCUMENT_COLLAPSE_FIELDS_REGEX_THRESHOLD

    The number of fields that match the regular expression before Structural collapses the fields into a single field. For example, if this is 5, then after Structural finds 5 fields that match the regular expression, it collapses all of the matching fields into a single field. A negative value indicates to not collapse the fields. The default value is -1.

    For example:

    • To collapse keys that are integer values, use the regular expression [0-9]+ or \d+

    • To collapse keys that are UUIDs, use the regular expression [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}

    On Privacy Hub, the Latest Collection Scan table shows the most recent scans on each scanned collection.

    The Build Schema option runs a new scan on the collection.

    Latest Collection Scan on Privacy Hub

    When the source database has a new collection, then on Collection View, you are prompted to run a scan either on that collection or on all collections.

    Collection scan prompt on Collection View

    If a collection scan fails, you can ask for AI assistance to troubleshoot the failure.

    Required workspace permission: Run collection scan

    Configuring the collection scan

    MongoDB

    Configuring how schemas are scanned

    Configuring how fields are collapsed in the hybrid document

    Collapsing fields when the key is an ObjectId

    Collapsing fields when the key matches a custom pattern

    Viewing the most recent scans for each collection

    Starting a collection scan

    By default, Structural redacts sensitive values from the job logs. To help support troubleshooting, you can configure data connectors or an individual data generation job to create unredacted versions of the log files, referred to as diagnostic logs. For more information, go to
    .

    To access diagnostic log files, you must have the Enable diagnostic logging global permission.

    If you do not have the Enable diagnostic logging global permission, then you cannot download the logs for that job. The download option is disabled.

    For a workspace that writes the output to a container repository, the job includes the following additional logs:

    • Database logs - Logs for the database container that is used as the destination.

    • Datapacker logs - Logs for creating the OCI artifact and uploading it to an OCI registry.

    To download these logs for a data generation job, on the job details view, click Reports and Logs, then select Database Logs or Datapacker Logs.

    For workspaces that are connected to Amazon Redshift or Snowflake on AWS databases, the data generation job requires multiple calls to a Lambda function. For these data generation jobs, the CloudWatch logs monitor the progress of and display errors for these Lambda function calls.

    To download the CloudWatch logs for a data generation job, on the job details view, click Reports and Logs, then select CloudWatch Logs.

    The CloudWatch Logs option only displays for Amazon Redshift and Snowflake on AWS data generation jobs.

    For an Oracle data generation, if both of the following are true:

    • The data generation job ran SQL Loader (sqlldr).

    • sqlldr either failed or succeeded with errors.

    Then to download the sqlldr log files, click Reports and Logs, then select sqlldr Logs.

    The job details include an option to send a log package to Tonic.ai.

    You would likely send the log package at the request of the Structural support team, to help to troubleshoot a data generation issue.

    To send the package, from the Reports and Logs dropdown list, select Send logs to Tonic.ai.

    Structural creates the package, then uploads it to an S3 bucket. Packages are removed from the S3 bucket automatically after 30 days.

    From the job details view, you can download a Privacy Report file that provides an overview of the current protection status of the database columns based on the workspace configuration at the time that the job ran.

    You can download either:

    • The Privacy Report .csv file, which provides details about the table columns, the column content, and the current protection configuration.

    • The Privacy Report PDF file, which provides charts that summarize the privacy ranking scores for the table columns. It also includes the table from the .csv file.

    To display the download options, click Reports and Logs. In the menu:

    Reports and Logs menu for a data generation job
    • To download the Privacy Report .csv file, click Privacy Report CSV.

    • To download the Privacy Report PDF file, click Privacy Report PDF.

    For more information about the Privacy Report files and their content, go to Using the Privacy Report to verify data protection.

    For a data generation from a file connector workspace that uses local files, you can download the transformed files for that job.

    The download is a .zip file that contains the files for a selected file group.

    On the job details view, when files are available to download, the Data available for file groups panel displays.

    To download the files for a file group:

    1. Click Download Results.

    2. From the list, select the file group. Use the filter field to filter the list by the file group name.

    Job details option to download transformed file connector files

    For workspaces that use the newer data generation processing, users can configure a data generation job to also generate performance metrics. This is usually done for troubleshooting purposes.

    On the job details view, to download the performance metrics for the job, click Reports and Logs, then click Performance Metrics.

    Job logs

    Required workspace permission: Download job logs

    To download diagnostic logs, you must have the Enable diagnostic logging team permission.

    Where to download the job logs

    Downloading diagnostic logs

    Viewing and downloading container artifacts

    Other logs for specific data connectors and data generation

    Additional logs for output to a container repository

    CloudWatch logs for data generation

    Oracle SQL Loader log files

    Required workspace permission: Download SqlLdr Files

    Sending a log package to Tonic.ai

    Required global permission: Enable diagnostic logging and uploading logs directly to Tonic.ai

    Privacy Report for data generation

    Required workspace permission: View and download Privacy Report

    Transformed files for file connector data generation

    Performance metrics for data generation

    Required workspace permission: Download job logs

    Redacted and diagnostic (unredacted) logs

    Differential privacy

    Yes, if consistency is not enabled.

    Data-free

    Yes, if consistency is not enabled.

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • 1 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    To configure the generator, toggle the Consistency setting to indicate whether to make the generator consistent.

    By default, the generator is not consistent.

    If consistency is enabled, then by default it is self-consistent. To make the generator consistent with another column, from the Consistent to dropdown list, select the column.

    • When the generator is consistent with itself, then a given source value is always mapped to the same destination value. For example, My Business is always mapped to New Business.

    • When the generator is consistent with another column, then a given source value in that other column always results in the same destination value for the company name column. For example, if the company name column is consistent with a name column, then every instance of John Smith in the name column in the source database has the same company name in the destination database.

    When consistency is enabled, use the Case-sensitive toggle to indicate whether the consistency is case-sensitive.

    By default, it is case-sensitive. For example, the values Business1 and BUSINESS1 are considered different values. Business1 might always be replaced with Example Co, and BUSINESS1 might be replaced with MyCo.

    To make the consistency case-insensitive, toggle Case-sensitive to the off position. When the consistency is case-insensitive, Business1 and BUSINESS1 are considered the same value and have the same replacement.

    If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Consistency

    Yes, can be made self-consistent or consistent with another column.

    Linking

    No, cannot be linked.

    Characteristics

    How to configure

    Enabling and disabling consistency

    Setting the type of consistency

    Setting case sensitivity for consistency

    Enabling Structural data encryption

    Determined by the specified sub-generators.

    Differential privacy

    Determined by the specified sub-generators.

    Data-free

    Determined by the specified sub-generators.

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    5

    Generator ID (for the API)

    To assign a generator to a path expression:

    1. Under Sub-generators, click Add Generator. On the sub-generator configuration panel, the Cell JSON field contains a sample value from the source database. You can use the previous and next icons to page through different values.

    2. In the Path Expression field, type the JSONPath expression to identify the value to apply the generator to. To populate a path expression, you can also click a value in the Cell JSON field. Matched JSON Values shows the result from the value in Cell JSON.

    3. By default, the selected generator is applied to any value that matches the expression. To limit the types of values to apply the generator to, from the Type Filter, specify the applicable types. You can select Any, or you can select any combination of String, Number, and Null.

    4. From the Generator Configuration dropdown list, select the generator to apply to the path expression. You cannot select another composite generator.

    5. Configure the selected generator. You cannot configure the selected generator to be consistent with another column.

    6. To save the configuration and immediately add a generator for another path expression, click Save and Add Another. To save the configuration and close the add generator panel, click Save.

    From the Sub-Generators list:

    • To edit a generator assignment, click the edit icon.

    • To remove a generator assignment, click the delete icon.

    • To move a generator assignment up or down in the list, click the up or down arrow.

    If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Consistency

    Determined by the specified sub-generators.

    Characteristics

    composite generator
    JSON Mask
    JSONPath

    Linking

    How to configure

    Adding a sub-generator

    Managing the sub-generator list

    Enabling data encryption

    Linking

    Determined by the selected sub-generators.

    Differential privacy

    Determined by the selected sub-generators.

    Data-free

    Determined by the selected sub-generators.

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    5

    Generator ID (for the API)

    To add a regular expression:

    1. Click Add Regex. On the configuration panel, Cell Value shows a sample value from the source database. You can use the previous and next options to navigate through the values.

    2. By default, Replace all matches is enabled. To only match the first occurrence of a pattern, toggle Replace all matches to the off position.

    3. In the Pattern field, enter a regular expression. If the expression is valid, then Structural displays the capture groups for the expression.

    4. For each capture group, to select and configure the generator to apply, click the selected generator. You cannot select another composite generator.

    5. To save the configuration and immediately add a generator for another path expression, click Save and Add Another. To save the configuration and close the add generator panel, click Save.

    From the Regexes list:

    • To edit a regular expression, click the edit icon.

    • To remove a regular expression, click the delete icon.

    Consistency

    Determined by the selected sub-generators.

    Characteristics

    composite generator
    Regex Mask

    How to configure

    Adding a regular expression

    Managing the regular expressions list

    Address

    Generates a random mailing address-like string.

    You can indicate which part of an address string that the column contains. For example, the column might contain only the street address or the city, or it might contain the full address.

    Configuring an individual column

    For an individual column in Database View, you can configure the assigned generator and determine the column sensitivity.

    From the column list, to display the generator configuration panel, in the Applied Generator column, click the generator name tag.

    The Structural sensitivity scan provides an initial indication of whether a column is sensitive and, if it is sensitive:

    • The type of sensitive data that is in the column.

    • The confidence level of the sensitivity detection.

    BusinessNameGenerator
    ArrayJsonMaskGenerator
    ArrayRegexMaskGenerator

    Differential privacy

    Yes, if consistency is not enabled.

    Data-free

    Yes, if consistency is not enabled.

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • 1 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    From the Link To dropdown list, select the columns to link this column to.

    You can link columns that use the Address generator to mask one of the following address components:

    • City

    • City State

    • Country

    • Country Code

    • State

    • State Abbreviation

    • Zip Code

    • Latitude

    • Longitude

    Note that when linked to another address column, a country or country code is always the United States.

    City, state, and zip code are always real values and match across linked columns.

    From the address component dropdown list, select the address component that this column contains.

    For most data connectors, the available address components are:

    • Building Number

    • Cardinal Direction (North, South, East, West)

    • City

    • City Prefix (Examples: North, South, East, West, Port, New)

    • City Suffix (Examples: land, ville, furt, town)

    • City with State (Example: Spokane, Washington)

    • City with State Abbr (Example: Houston, TX)

    • Country (Examples: Spain, Canada)

    • Country Code (Uses the 2-character country code. Examples: ES, CA)

    • County

    • Direction (Examples: North, Northeast, Southwest, East)

    • Full Address

    • Latitude (Examples: 33.51, 41.32)

    • Longitude (Examples: -84.05, -74.21)

    • Ordinal Direction (Examples: Northeast, Southwest)

    • Secondary Address (Examples: Apt 123, Suite 530)

    • State (Examples: Alabama, Wisconsin)

    • State Abbr (Examples: AL, WI)

    • Street Address (Example: 123 Main Street)

    • Street Name (Examples: Broad, Elm)

    • Street Suffix (Examples: Way, Hill, Drive)

    • US Address

    • US Address with Country

    • Zip Code (Example: 12345)

    Note that street values are always generated and do not reflect a real address. To generate addresses with real streets, use the Valid Address generator.

    Databricks and Spark SDK workspaces only support the following address components:

    • Building Number

    • City

    • Country

    • Country Code

    • Full Address

    • Latitude

    • Longitude

    • State

    • State Abbr

    • Street Address

    • Street Name

    • Street Suffix

    • US Address

    • US Address with Country

    • Zip Code

    Toggle the Consistency setting to indicate whether to make the column consistent.

    By default, the consistency is disabled.

    If consistency is enabled, then by default, the generator is self-consistent.

    To make the generator consistent with another column, from the Consistent to dropdown list, select the column.

    When the Address generator is consistent with itself, then the same value in the source database is always mapped to the same destination value. For example, for a column that contains a state name, Alabama is always mapped to Illinois.

    When the Address generator is consistent with another column, then the same value in the other column always results in the same destination value for the address column. For example, if the address column is consistent with a name column, then every instance of John Smith in the name column in the source database has the same address value in the destination database.

    When consistency is enabled, use the Case-sensitive toggle to indicate whether the consistency is case-sensitive.

    By default, consistency is case-sensitive. For example, the values Anytown and ANYTOWN are considered different values. Anytown might always be replaced with Chicago, and ANYTOWN might be replaced with Atlanta.

    To make the consistency case-insensitive, toggle Case-sensitive to the off position. When the consistency is case-insensitive, Anytown and ANYTOWN are considered the same value and have the same replacement.

    If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Consistency

    Yes, can be made self-consistent or consistent with another column.

    Linking

    Characteristics

    Yes, can be linked.

    How to configure

    Linking to other columns

    Identifying the address component

    Supported address components (connectors other than Databricks and Spark SDK)

    Supported address components (Databricks and Spark SDK)

    Enabling and disabling consistency

    Setting the type of consistency

    Setting case sensitivity for consistency

    Enabling Structural data encryption

    For more information, go to Identifying sensitive data.

    In a child workspace, you cannot configure whether a column is sensitive. A child workspace always inherits the sensitivity designations from its parent workspace.

    From the Status column, to confirm or change the column sensitivity, click the Status value.

    The status panel indicates whether the column is sensitive. It identifies the sensitivity type, and indicates how the sensitivity was determined - by a sensitivity scan or by a user.

    For a column that matches a built-in sensitivity type, the first time that you display the panel, the Sensitive data? setting displays Yes and No options for you to confirm or change the sensitivity.

    Column status panel with confirmation options
    • To indicate that the column is sensitive, click Yes.

    • To indicate that the column is not sensitive, click No.

    When you click Yes or No, the Yes and No options change to a simple toggle. When you click Yes, the sensitivity confidence level changes to full.

    Status panel after you select No or Yes to indicate the sensitivity

    After that:

    • To indicate that the column is sensitive, toggle Sensitive data? to the on position.

    • To indicate that the column is not sensitive, toggle Sensitive data? to the off position.

    When a column matches a sensitivity rule, the sensitivity panel indicates that the column matched a sensitivity rule.

    Sensitivity panel for a column that matched a sensitivity rule

    You use the Sensitive data? toggle to indicate whether the column is actually sensitive.

    When a column does not match a built-in sensitivity type or a custom sensitivity rule, the sensitivity panel indicates that column is not sensitive.

    Sensitivity panel for a not sensitive column

    The Sensitive data? setting displays Yes and No options for you to confirm or change the sensitivity.

    • To indicate that the column is sensitive, click Yes.

    • To confirm that the column is not sensitive, click No.

    When you click Yes or No, the Yes and No options change to a simple toggle.

    If you click Yes:

    • The panel updates to indicate that a user confirmed that the column is sensitive.

    • The sensitivity confidence level is set to full confidence.

    Sensitivity panel after you change the sensitivity on a not sensitive column

    After that:

    • To indicate that the column is sensitive, toggle Sensitive data? to the on position.

    • To indicate that the column is not sensitive, toggle Sensitive data? to the off position.

    To configure the sensitivity, you can also use the Sensitive Data toggle on the column configuration panel.

    Sensitivity toggle on the column configuration panel
    • To indicate that a column is sensitive, toggle the sensitivity setting to the on position.

    • To indicate that the column is not sensitive, toggle the sensitivity setting to the off position.

    When you change the sensitivity from the generator configuration panel, the Sensitive data? setting on the sensitivity panel also changes from the Yes and No options to the toggle.

    When a sensitivity scan identifies a column, Structural recommends a generator for the column. For example, when the sensitivity scan identifies a column as a first name, Structural recommends the Name generator configured to generate a first name value.

    In the Assigned Generator column on Database View, columns that do not have an assigned generator, and that have a recommended generator, display the available recommendation icon.

    Column with the available recommendation icon

    When you click the generator dropdown, the column configuration panel includes the following information:

    • The sensitivity confidence level.

    • The recommended generator.

    • Sample source and destination values based on the recommended generator.

    Recommended generator panel for a column

    From the panel, you choose whether to assign or ignore the recommended generator for that type.

    • To assign the recommended generator, click Apply.

    • To ignore the recommendation, click Ignore. Structural clears the recommendation.

    To change the generator that is assigned to a selected column:

    1. Click the generator name tag for the column.

    2. To assign a different generator to the column, from the Generator Type dropdown list, select the generator.

    3. Configure the generator options.

    To reset an assigned generator to Passthrough, which indicates to not transform the data, click Reset, then click Reset to Passthrough.

    For details about the configuration options for each generator, go to the Generator reference.

    For more information about selecting and configuring generators and generator presets, go to Assigning and configuring generators.

    For a JSON column, instead of assigning a generator, you can enable Document View.

    From Document View, you can view the JSON schema structure and assign generators to individual JSON fields. For more information, go to Using Document View for JSON columns.

    To enable Document View, on the column configuration panel, toggle Use Document View to the on position. Note that if you have custom value processors, or enabled Structural data encryption, then the Use Document View toggle is in the advanced options.

    When Document View is enabled, the generator dropdown is replaced with the Open in Document View option.

    Displaying the generator configuration panel

    Indicating whether a column is sensitive

    Required workspace permission: Configure column sensitivity

    Generator configuration panel

    Status column

    Built-in sensitivity type

    Sensitivity rule match

    No built-in sensitivity type or sensitivity rule match

    Column configuration panel

    Assigning or ignoring the recommended generator

    Required workspace permission: Configure column generators

    Changing the column generator configuration

    Required workspace permission: Configure column generators

    Enabling Document View for JSON columns

    Supported only for the file connector and PostgreSQL.

    Configuring the available secrets managers

    Structural currently supports:

    • AWS Secrets Manager

    • HashiCorp Vault

    • Cyberark Central Credential Provider

    AddressGenerator
    Sensitivity rule configuration
    Azure Key Vault

    To display the list of secrets managers:

    1. In the user menu, click Team Settings.

    2. On Team Settings view, click Secrets Managers.

    Secrets Managers tab on Team Settings

    To create a secrets manager:

    1. On the Secrets Managers tab, click Add Secrets Manager.

    2. On the Create Secrets Manager panel, in the Name field, provide a name to use to identify the secrets manager. Secrets manager names must be unique. The name is used in the secrets manager dropdown list on the workspace settings view.

    3. From the Type dropdown list, select the secrets manager product.

    4. Configure the credentials to use to connect to the secrets manager.

    5. Click Save.

    For an existing secrets manager, you can change the name and the credentials configuration.

    You cannot change the type.

    To edit an existing secrets manager:

    1. In the secrets manager list, click the edit icon for the secrets manager.

    2. On the Edit Secrets Manager panel, update the configuration.

    Edit Secrets Manager panel
    1. Click Save.

    When you delete a secrets manager, it is removed from the workspace database connections that use it. Structural is no longer able to connect to those databases.

    To delete a secrets manager:

    1. In the secrets manager list, click the delete icon for the secrets manager.

    2. On the confirmation panel, click Delete.

    The AWS Secrets Manager credentials that you provide must have the following permissions:

    • On each secret to use, secretsmanager:GetSecretValue

    • On the encryption key for secrets that are encrypted with a customer managed key (CMK), kms:Decrypt

    Here is an example policy that grants the required Secrets Manager permissions:

    For AWS Secrets Manager, from the Authentication dropdown list, select the source of the credentials:

    Authentication options for AWS Secrets Manager
    • Environment - Only available on self-hosted instances. Indicates to use either:

      • The credentials for the AWS Identity and Access Management (IAM) role on the host machine.

      • The credentials set in the following environment settings:

        • TONIC_AWS_ACCESS_KEY_ID - An AWS access key that is associated with an IAM user or role

        • TONIC_AWS_SECRET_ACCESS_KEY - The secret key that is associated with the access key

        • TONIC_AWS_REGION - The AWS Region to send the authentication request to

    • Assume Role - Indicates to use the specified assumed role.

    • User Credentials - Indicates to use the provided user credentials.

    To provide an assumed role, select Assume Role, then:

    Configuration fields for the Assume Role option for AWS Secrets Manager credentials
    1. In the Role ARN field, provide the Amazon Resource Name (ARN) for the role.

    2. In the Session Name field, provide the role session name. If you do not provide a session name, then Structural automatically generates a default unique value. The generated value begins with TonicStructural.

    3. In the Duration (in seconds) field, provide the maximum length in seconds of the session. The default is 3600, indicating that the session can be active for up to 1 hour. The provided value must be less than the maximum session duration that is allowed for the role.

    4. From the AWS Region dropdown list, select the AWS Region to send the authentication request to.

    Structural generates the external ID that is used in the assume role request. Your role’s trust policy must be configured to condition on your unique external ID.

    Here is an example trust policy:

    To provide the credentials, select User Credentials, then:

    Configuration fields for the User Credentials option for AWS Secrets Manager credentials
    1. In the AWS Access Key field, enter the AWS access key that is associated with an IAM user or role.

    2. In the AWS Secret Key field, enter the secret key that is associated with the access key.

    3. Optional. In the AWS Session Token field, provide the session token to use.

    4. From the AWS Region dropdown list, select the AWS Region to send the authentication request to.

    For a HashiCorp Vault secrets manager:

    Configuration panel for a HashiCorp Vault secrets manager
    1. In the Vault Server field, provide the server name or IP address where the secrets manager is located.

    2. From the Secrets Engine dropdown list, select the version of the secrets engine used by the secrets manager.

    3. From the Authentication Method dropdown list, select how to authenticate to the secrets manager. The options are:

      • AppRole

      • Token

      • LDAP

    4. For app role authentication:

      1. In the Role ID field, provide the identifier of the application role.

      2. In the Secret ID field, provide the secret identifier of the application role.

      On a self-hosted instance, if you do not provide a role identifier and secret identifier, then Structural uses the values of the TONIC_SECRET_MANAGERS_HASHICORP_VAULT_APPROLE_ROLE_ID and TONIC_SECRET_MANAGERS_HASHICORP_VAULT_APPROLE_SECRET_ID

    5. For token authentication, in the Token field, provide the authentication token to use. On a self-hosted instance, if you do not provide a token, Structural uses the value of the TONIC_SECRET_MANAGERS_HASHICORP_VAULT_TOKEN to authenticate.

    6. For LDAP authentication:

      1. In the Username field, provide the LDAP username.

      2. In the Password field, provide the password for the LDAP user.

      On a self-hosted instance, if you do not provide a username and password, then Structural uses the values of the TONIC_SECRET_MANAGERS_HASHICORP_VAULT_LDAP_USERNAME and TONIC_SECRET_MANAGERS_HASHICORP_VAULT_LDAP_PASSWORD

    7. If the authentication method is enabled in a specific namespace, then in the Namespace field, provide the namespace.

    8. If the selected authentication method does not use the default mount path, then in the Mount path field, provide the mount path.

    For a CyberArk secrets manager:

    Configuration panel for a CyberArk Credential Provider secrets manager
    1. In the CyberArk Server field, provide the server name or IP address where the secrets manager is located.

    2. In the CyberArk Port field, provide the port to use to connect to the secrets manager.

    3. In the CyberArk Application ID field, provide the identifier of the CyberArk application that contains the secrets manager.

    4. In the CyberArk Safe field, provide the name of the CyberArk safe that contains the secrets.

    5. Optional. In the CyberArk Folder field, provide the name of the folder within the safe that contains the secrets. If you do not specify a folder here or in the workspace configuration, the folder defaults to Root. To specify a folder path, use Root followed by the rest of the path, with each path component separated by backslashes. For example: Root\OS\Linux.

    By default, Structural uses the application identifier to authenticate to CyberArk.

    You can instead use a CyberArk authentication certificate.

    When you use a certificate, then by default, during authentication, Structural validates the certificate that the server sends. You might need to install a root certificate on the Structural web server and workers. Instead, to bypass the validation, set the environment setting TONIC_SECRET_MANAGERS_CYBER_ARK_CCP_VERIFY_SERVER_CERTIFICATE to false.

    To configure the CyberArk authentication certificate:

    1. Click Certificate.

    Configuration for a CyberArk authentication certificate
    1. Under CyberArk Authentication Certificate, to search for and select the certificate file, click Browse. The certificate must be either .pfx or .p12.

    2. In the CyberArk Certificate Password field, provide the password that is associated with the certificate.

    For an Azure Key Vault secrets manager:

    Configuration panel for an Azure Key Vault secrets manager
    1. In the Vault Name field, provide the name of the key vault.

    2. In the Tenant ID field, provide your Azure tenant identifier to use to authenticate to the vault.

    3. In the Client ID field, provide your Azure client identifier to use to authenticate to the vault.

    4. In the Client Secret field, provide your Azure client secret to use to authentiate to the vault.

    From the configuration panel, you can test whether Structural can use the configured credentials to connect to the secrets manager and retrieve a specific secret.

    To test a secrets manager configuration:

    1. Click Test Secrets Manager.

    2. On the Test Secrets Manager panel, in the Secret Name field, provide the name of the secret.

    3. If needed for the secrets manager type, provide any other information that is needed for the connection.

    4. Click Run Test.

    For a HashiCorp Vault secrets manager, in addition to the secret name, you can provide the following if applicable:

    • Namespace - If the secret is in a specific namespace, the name of the namespace.

    • Mount Path - If the secrets engine does not use the default mount path, the mount path to use.

    For a CyberArk Central Credential Provider secrets manager, in addition to the secret name, you can optionally provide:

    • CyberArk Safe - The name of the CyberArk safe that contains the secrets manager.

    • CyberArk Folder - The name of the folder within the safe that contains the secrets manager.

    If you do not provide these values, they fall back to the values that are configured for the secrets manager.

    Required team permission: Manage secrets managers

    Supported secrets manager tools and formats

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "AllowReadingSecrets",
          "Effect": "Allow",
          "Action": [
            "secretsmanager:GetSecretValue"
          ],
          "Resource": "arn:aws:secretsmanager:us-east-1:111111111111:secret:mySecretNamespace/*"
        }
      ]
    }
    {
      "Version": "2012-10-17",
      "Statement": {
        "Effect": "Allow",
        "Principal": {
          "AWS": "<originating-account-id>"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
          "StringEquals": {
            "sts:ExternalId": "<external-id>"
          }
        }
      }
    }

    Viewing the secrets manager list

    Working with secrets managers

    Creating a secrets manager

    Editing an existing secrets manager

    Deleting a secrets manager

    Providing credentials for AWS Secrets Manager

    Required AWS Secrets Manager permissions

    Selecting the source of the credentials

    Providing an assumed role

    Providing AWS user credentials

    Configuring a HashiCorp Vault secrets manager

    Configuring a CyberArk Central Credential Provider secrets manager

    Configuring an Azure Key Vault secrets manager

    Testing a secrets manager configuration

    Running the configuration test

    Additional test information for HashiCorp Vault

    Additional test information for CyberArk Central Credential Provider

    Creating and managing custom sensitivity rules

    Required team permission: Create and manage sensitivity rules

    By default, when a Structural security scan runs on a workspace, it looks for the built-in sensitivity types.

    You can also define custom sensitivity rules to identify other values and the corresponding recommended generator. Your data might include values that are specific to your organization.

    To identify the columns that the sensitivity rule applies to, you can either:

    • Match text in the column name.

    • Provide a description to tell the LLM what to look for. The LLM-based sensitivity scan then matches the description against the source data columns.

    For both types of custom sensitivity rule, you identify the applicable column data type, and select the generator preset to apply to matching columns.

    LLM-based sensitivity rules are processed as part of the LLM-based sensitivity detection. When LLM-based sensitivity detection is not used, then the LLM-based sensitivity rules also are not used.

    On self-hosted instances, even if LLM-based sensitivity detection is enabled, an additional environment setting specifically enables LLM-based sensitivity rules.

    For more information, go to and .

    To display the current list of sensitivity rules, in the Structural navigation menu, click Sensitivity Rules.

    On the Sensitivity Rules page:

    • The Column Name rules list contains the list of sensitivity rules that check for a text or regular expression match in the column name.

    • If LLM-based sensitivity rules are enabled, then the LLM-based rules list is displayed and contains the list of sensitivity rules that use the rule description as input for the LLM-based security scan. The LLM-based scan uses the description to check for matching columns.

    The lists contain sensitivity rules for a self-hosted Structural instance or a Structural Cloud organization.

    For each rule, the list includes:

    • The rule name and description

    • The recommended generator preset

    • When the rule was most recently modified

    You can filter each rule list by the following:

    • Rule name

    • Rule description

    • Generator preset name

    • Name of the user who most recently updated the rule

    In the filter field, start to type text from any of those values. As you type, the list is filtered to only include matching rules.

    Note that when the list is filtered, you cannot change the display sequence of the rules.

    For each type of rule, Structural applies the rules based on their display order in the list. If a column matches more than one rule, Structural applies the first matching rule.

    Column name rules take precedence over LLM-based rules. When a column matches both a column name rule and an LLM-based rule, the column name rule is used.

    To change the display order of a rule, drag and drop it to the new location in the list.

    Note that you cannot change the rule sequence when the list is filtered.

    To create a sensitivity rule:

    1. On the Sensitivity Rules view, click New Custom Rule.

    2. On the Create Custom Rule view, .

    3. Click Save.

    To change the configuration of a sensitivity rule:

    1. On the Sensitivity Rules view, click the edit icon for the rule.

    2. On the Edit Custom Rule view, .

    3. Click Save.

    Note that any changes to a sensitivity rule do not take effect until the next sensitivity scan.

    In the Name field, type the name of the sensitivity rule. The rule name becomes the sensitivity type for matching columns.

    The rule name:

    • Must be unique.

    • Cannot match the name of a built-in sensitivity type.

    For column name sensitivity rules, use the Description field to provide an optional longer description of the sensitivity rule and how it is used.

    For LLM-based sensitivity rules, the content of the Description field is what Structural sends to the LLM during LLM-based sensitivity detection. When providing the description for the LLM, to ensure the most accurate matches, be as specific as possible.

    Note that if Structural is configured to not send sample data to the LLM, and the description refers to the column value and not the column name, the LLM cannot identify matching columns.

    When both column name and LLM-based sensitivity rules are enabled, then under Match Type, click the type of rule.

    • To create a column name rule, click Column Name.

    • To create an LLM-based rule, click Description (LLM).

    After you save the rule, you cannot change the rule type.

    From the Data Type dropdown list, select the data type for matching columns. For example, a rule might only be used for columns that contain text.

    The available data types are general types that map to specific data types in a given database. The available types are:

    • Array

    • Binary

    • Boolean

    • Continuous Numerical

    Under Column Name Match, provide the criteria to identify matching columns based on the column name.

    Note that a matching column must match both the data type and the column name criteria.

    When you provide a list of text matching conditions, a matching column must match all of the conditions. In other words, the conditions are joined by AND.

    To apply the same generator preset to columns that have completely different names, you must create separate sensitivity rules.

    To create a list of text matching conditions:

    1. Click Text Match.

    2. To add a column name condition, click Add String Match.

    3. For each condition:

      1. From the comparison type dropdown list, select the type of comparison. For example, Contains, Starts with

    To use a regular expression to identify matching columns based on the column name:

    1. Click Regular Expression.

    2. In the field, provide the regular expression.

    From the Recommended Generator Preset dropdown list, select the generator preset that is the recommended generator for matching columns.

    To search for a specific preset, begin to type the generator preset name.

    When you configure a sensitivity rule, you can also create a new generator preset or update the configuration of the selected generator preset.

    To create a new generator preset, click Create Preset. On the generator preset details panel, provide the generator preset configuration, then click Create.

    To edit the selected generator preset, click Edit Current Preset. On the generator preset details panel, update the generator preset configuration, then click Save and Apply.

    For more information about generator preset configuration, go to .

    You cannot preview the results of LLM-based sensitivity rules. You can only test column name rules.

    If you have access to a workspace, then you can use the workspace to preview the sensitivity rule results.

    Under Test Results, from the workspace dropdown list, select the workspace to use.

    Structural searches the workspace schema for matching columns based on the sensitivity rule configuration.

    It displays any matching columns. You can filter the matching columns based on the table or column name.

    For each matching column, the list includes:

    • The column name and table

    • A sample value from the source data. The sample source value is only present if you have the Preview source data permission for the workspace.

    • A sample replacement value, based on the selected generator preset for the sensitivity rule. The sample replacement value is only present if you have the Preview destination data permission for the workspace.

    To delete a sensitivity rule, on the Sensitivity Rules view, click the delete icon for the rule.

    Note that existing generator recommendations for the rule remain in place until the next sensitivity scan.

    Self-hosted LLM configuration

    Self-hosted customers can choose whether to enable AI features on their instance.

    A self-hosted instance can either:

    • Use an LLM that Tonic.ai hosts. This is the default option.

    • Configure a connection to a supported LLM provider.

    • Not use an LLM at all. This disables all Structural AI features.

    You configure the connection either:

    • From the AI Settings tab on Structural Settings.

    • Using . The settings can be added to the Environment Settings list on Structural Settings.

    If you enable AI features, then Structural provides additional options to .

    Tonic Structural supports the following LLM providers:

    • Anthropic

    • Anthropic on Amazon Bedrock

    • OpenAI. Note that we are continuing to test OpenAI for the Structural Agent.

    Structural provides a hosted LLM that you can use to support the Structural AI features. The hosted LLM uses Anthropic on Amazon Bedrock.

    For new self-hosted instances, this is the default option.

    The LLM is hosted in multiple locations. Structural routes the request through a SaaS proxy to the appropriate location for the customer.

    On AI settings view, to use the hosted LLM:

    1. From the LLM Provider dropdown list, select Tonic.ai-hosted LLM.

    2. From the from the Region dropdown list, select the hosted LLM location to use.

    3. To test the connection to the hosted LLM, click Test Tonic.ai-hosted LLM Connection.

    To use the to select the hosted LLM:

    • Set TONIC_LLM_PROVIDER to TonicHosted.

    • Set TONIC_HOSTED_REGION to either US or EU.

    If you use the hosted LLM, make sure to allowlist the SaaS proxy, either:

    • http://us-east-1.saasproxy.tonic.ai/ or 98.84.248.119 (US location)

    • http://eu-central-1.saasproxy.tonic.ai/ or 3.120.214.225 (EU location)

    To not connect to an LLM, which disables Structural AI features, from the LLM Provider dropdown list, select Do not use an LLM.

    To use an environment setting to disable Structural AI features, set TONIC_LLM_PROVIDER to LlmOptOut.

    If you use a hosted Anthropic deployment, Structural requires the following models:

    • claude-opus-5

    • claude-sonnet-5

    • claude-haiku-4-5

    From AI Settings, to configure Anthropic as your LLM:

    1. From the LLM Provider dropdown list, select Anthropic.

    2. In the Anthropic Endpoint field, enter the Anthropic endpoint URL for generative AI services. Should include the endpoint base URL for the Anthropic API, without the version. The default is https://api.anthropic.com.

    3. In the Anthropic API Key field, enter the API key for Anthropic. In production environments, make sure that this is kept secure and encrypted.

    To use to connect to Anthropic:

    • TONIC_LLM_PROVIDER - Set to Anthropic.

    • TONIC_ANTHROPIC_ENDPOINT - The Anthropic endpoint URL. Include the endpoint base URL for the Anthropic API, without the version.

    • TONIC_ANTHROPIC_API_KEY - The Anthropic API key.

    When you use Anthropic on Amazon Bedrock, you can either:

    • Use the environment configuration for AWS authentication.

    • Provide an Amazon Bedrock API key.

    To call Amazon Bedrock, most model and regions require the use of . The AWS documentation includes information about .

    The IAM principal that authenticates the request must be granted the following permissions.

    • bedrock:InvokeModel

    • bedrock:InvokeModelWithResponseStream

    The principal must be able to invoke the cross-Region inference profiles that apply to your Region and model.

    By default, Structural uses the following models:

    • us.anthropic.claude-haiku-4-5-20251001-v1:0

    • us.anthropic.claude-sonnet-5

    • us.anthropic.claude-opus-5

    Cross-Region inference also requires invoke permission on the underlying foundation model in each AWS Region that the profile spans.

    For example, for the US, the principal needs the invoke permission in:

    • us-east-1

    • us-east-2

    • us-west-2

    For the recommended IAM policy structure, go to the .

    From AI Settings, to use Anthropic on Amazon Bedrock, from the LLM Provider dropdown list, select Anthropic+Bedrock.

    To use the environment configuration for authentication, click Environment Configuration.

    Structural uses the credentials set in the following :

    • TONIC_AWS_ACCESS_KEY_ID - An AWS access key that is associated with an IAM user or role.

    • TONIC_AWS_SECRET_ACCESS_KEY - The secret key that is associated with the access key.

    • TONIC_AWS_REGION - The AWS Region to send the authentication request to.

    If these environment settings are not configured, then Structural uses the .

    From AI Settings, to use an Amazon Bedrock API key for authentication:

    1. Click API Key.

    2. In the Amazon Bedrock API Key field, provide the API key.

    Optionally, in the Model Name field, specify the model to use. In most cases, the value is a cross-Region inference profile. For combinations of Region and model that do not require cross-Region inference, the you can provide a model name.

    If you do not specify a model, then the default is:

    • For lightweight, quick tasks, us.anthropic.claude-haiku-4-5-20251001-v1:0.

    • For standard tasks, us.anthropic.claude-sonnet-5.

    • For complex, high-quality tasks, us.anthropic.claude-opus-5.

    To use to connect to Anthropic on Amazon Bedrock:

    • TONIC_LLM_PROVIDER - Set to AnthropicBedrock.

    • TONIC_BEDROCK_API_KEY - To use an API key for authentication, set this to the Amazon Bedrock API key. To use the environment configuration for authentication, do not set this setting.

    • TONIC_LLM_MODEL - The model to use.

    Structural supports any OpenAI deployment that:

    • Supports API key authentication.

    • Supports the OpenAI Responses API.

    • Supports OpenAI proprietary models, such as GPT-4/5.x.

    If you use a hosted OpenAI deployment such as Microsoft Foundry, Structural requires the following model deployments. The deployment name must match the model name.

    • gpt-4.1-mini

    • gpt-4.1

    • gpt-5.2

    From AI Settings, to use OpenAI as your LLM provider:

    1. From the LLM Provider dropdown list, select OpenAI.

    2. In the OpenAI Endpoint field, enter the OpenAI endpoint URL to use for AI-enhanced features. Should include the endpoint base URL for the OpenAI API, without the version. The default is https://api.openai.com. Here is an example URL for Microsoft Foundry: https://my-foundry-instance.ai.azure.com/openai

    3. In the OpenAI API Key field, enter the API key for OpenAI.

    To use to connect to OpenAI:

    • TONIC_LLM_PROVIDER - Set to OpenAI.

    • TONIC_OPENAI_ENDPOINT - The OpenAI endpoint URL. Include the endpoint base URL for the OpenAI API, without the version.

    • TONIC_OPENAI_API_KEY - The API key for OpenAI.

    By default, Structural sensitivity detection includes , which sends database information to an LLM.

    To disable the LLM-based sensitivity detection, set the TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS to false.

    You can set this from the Environment Settings tab on Structural Settings.

    LLM-based sensitivity detection also processes LLM-based sensitivity rules. If you disable LLM-based sensitivity detection, then even if you enable LLM-based sensitivity rules, the sensitivity scan does not process any LLM-based sensitivity rules.

    On a self-hosted instance, to allow LLM-based sensitivity rules to be created and processed:

    • LLM-based sensitivity rules must be enabled - The environment setting TONIC_LLM_ENABLE_CUSTOM_SENSITIVITY_RULES must be set to true. This allows you to create LLM-based sensitivity rules.

    • LLM-based sensitivity detection must be enabled - The environment setting TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS must be set to true. This allows LLM-based sensitivity rules to be processed. If this is false, then those rules are ignored.

    By default, when LLM-based sensitivity detection is enabled, it sends both the database schema and sample source data values to the LLM.

    To exclude the contextual source data, and only send the schema, set the TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS_SAMPLE_DATA to false.

    You can set this from the Environment Settings tab on Structural Settings.

    To configure Structural to never send sample data for any LLM-based feature, either:

    • On AI Settings view, under LLM Data Handling, uncheck Allow LLM access to sample data.

    • Set the TONIC_LLM_ENABLE_SAMPLE_DATA to false. You can configure this setting from the Environment Settings tab on Structural Settings.

    Running the Structural sensitivity scan

    The Structural sensitivity scan identifies sensitive columns in source data. The scan ignores truncated tables.

    When sensitivity scans run

    For most data connectors, Structural runs sensitivity scans automatically based on specific events. You can also run manual sensitivity scans on demand.

    Sensitivity scans can also run automatically at the same time each week.

    Event-based sensitivity scans

    Structural does not run automatic sensitivity scans for Databricks and Spark SDK workspaces.

    By default, Structural does not run automatic sensitivity scans for file connector workspaces. To enable automatic sensitivity scans for the file connector, set the TONIC_ENABLE_FILES_PRIVACY_SCAN_AUTORUN to true.

    Structural automatically runs a sensitivity scan when you:

    • Create a completely new workspace and connect a data source.

    • Change the data connection details for the source database.

    • Add a file group to a file connector workspace.

    A child workspace always inherits the sensitivity designations from its parent workspace.

    When you copy a workspace, Structural runs a new sensitivity scan on the copy to identify sensitive columns. However, it keeps the sensitivity designation for columns that you specifically marked as sensitive or not sensitive.

    To not automatically run event-based sensitivity scans, set the TONIC_DISABLE_PRIVACY_SCAN_AUTORUN to true. You can set this setting from the Environment Settings list on Structural Settings. You can also in individual workspaces.

    In addition to the automatic scans, from Privacy Hub, you can .

    Structural can also runs scheduled weekly sensitivity scans in the background.

    By default, Structural Cloud does not run scheduled scans.

    By default, on self-hosted instances, Structural runs sensitivity scans each Sunday at midnight.

    The weekly scans run only on the workspaces that had the most recent activity.

    • For self-hosted instances, the weekly scans run on the 10 workspaces that had the most recent activity.

    • On Structural Cloud, for each organization, the weekly scans run on the 2 workspaces that had the most recent activity.

    Activity includes:

    • User-initiated updates that are included in the Version History.

    • Data generation jobs.

    On self-hosted instances, you can configure whether to run the weekly scans, and set the schedule.

    To manage the scheduled scans, use the following . You can add these settings to the Environment Settings list on Structural Settings.

    • TONIC_ENABLE_SCHEDULED_SENSITIVITY_SCAN - Boolean to indicate whether to enable the scheduled weekly sensitivity scans. The default value is true. To disable the scheduled weekly scan, set this to false.

    • TONIC_SENSITIVITY_SCAN_DAY - When scheduled scans are enabled, the day of the week on which to run the scans. The value is an integer between 0 and 6, where 0 is Sunday and 6 is Saturday. The default value is 0.

    For improved performance, sensitivity scans can use parallel processing.

    For relational databases such as PostgreSQL and SQL Server, to configure parallel processing, you use the TONIC_PII_SCAN_PARALLELISM_RDBMS. The default value is 4.

    For document-based databases such as MongoDB, you use the environment setting TONIC_PII_SCAN_PARALLELISM_DOCUMENTDB. The default value is 1.

    The Structural sensitivity scan uses the following rules and processes to:

    • Identify sensitive columns.

    • Recommend generators for those columns. For information about applying recommended generators to columns, go to .

    • Indicate its confidence that an identified column is sensitive and is of the detected sensitivity type.

    Note that this process cannot guarantee perfect precision and recall. We strongly recommend that a human reviews the sensitivity scan results and the broader dataset to ensure that nothing sensitive was missed.

    To identify that a column contains sensitive information for a , Structural looks at the data type, column name, and column values. For the values, Structural uses a random sampling of rows from the data.

    This part of the sensitivity scan uses regular expression matching and dictionary lookups. It produces high, medium, or low confidence detections.

    When this part of the sensitivity scan determines that a column contains sensitive data, it:

    • Marks the column as sensitive

    • Assigns the sensitivity type to the column

    • Recommends the generator configuration for the identified sensitivity type. Note that if the recommended generator is not compatible with the column, then Structural discards the recommendation.

    • Marks the sensitivity detection as high, medium, or low confidence. The confidence level is based on a calculation of how well the column matched the applicable rules.

    The sensitivity scan also looks for any columns that match custom sensitivity types that you define in your custom sensitivity rules.

    Custom sensitivity rules are based on the column data type and column name. For more information about custom sensitivity rules, go to .

    Custom sensitivity rules always produce full confidence detections.

    When a column matches a custom sensitivity rule, Structural:

    • Marks the column as sensitive.

    • Assigns the sensitivity rule name as the sensitivity type.

    • Recommends the generator preset from the sensitivity rule.

    • Marks the sensitivity detection as full confidence.

    Note that if LLM-based sensitivity detection is enabled, then model-based analysis is not used.

    To identify additional sensitive columns that might not be captured by the other parts of the scan, the sensitivity scan uses an artificial intelligence (AI) model. Note that the model is pre-trained. Structural does not use customer data to train the model, and it does not send any customer data externally.

    This part of the scan produces medium or low confidence detections for built-in entity types.

    The model considers the table and column name. If the combination of table and column name is similar in meaning to a sensitivity type that Structural has a recommended generator for, then Structural:

    • Marks the column as sensitive.

    • Assigns the sensitivity type to the column.

    • Recommends the generator configuration for that sensitivity type.

    • Uses AI to compare the table name and column name combination to the sensitivity type, and produces a semantic similarity score.

    The sensitivity scan can use LLM-based sensitivity detection instead of the model-based analysis.

    The LLM-based scan uses the database schema and, optionally, contextual source data, to determine whether a column contains a sensitive value and, if so, the sensitivity type.

    This part of the scan produces medium confidence detections for built-in entity types.

    The LLM-based sensitivity scan also processes LLM-based sensitivity rules. If LLM-based sensitivity detection is not enabled, then any LLM-based sensitivity rules are not used.

    For information about the general flow for LLM-based sensitivity detection, go to .

    For information about how to configure the use of LLM-based sensitivity detection, go to or .

    To download the log of the most recent sensitivity scan, either:

    • On the workspace management view, from the download menu, select Download Sensitivity Scan Log.

    • On Privacy Hub, click Reports and Logs, then select Scan Log.

    The log tracks the progress of the scan.

    If a sensitivity scan fails, you can .

    TONIC_SENSITIVITY_SCAN_HOUR - When scheduled scans are enabled, the hour at which to run the scans. The setting uses the local time zone. The value is an integer between 0 and 23, where 0 is midnight and 23 is 11:00 PM. For example, a value of 14 indicates to run the job at 2:00 PM. The default value is 0.
  • TONIC_PII_SCAN_MAX_TIMEOUT_IN_MINUTES_IF_AUTOMATIC - The number of minutes after which a scheduled scan times out. By default, the scan times out after 3 minutes. You can override this setting in an individual workspace.

  • Based on the semantic similarity score, marks the sensitivity detection as either medium or low confidence.

    Manual sensitivity scans

    Weekly sensitivity scans

    Default schedule and coverage for weekly scans

    Configuring weekly scans (self-hosted)

    Configuring parallel processing for sensitivity scans

    How Structural identifies sensitive values

    Rule-based data type, column name, and value analysis - High, medium, or low confidence

    Custom sensitivity rules - Full confidence

    Model-based analysis - Medium and low confidence

    LLM-based sensitivity detection - Medium confidence

    Downloading the sensitivity scan log

    environment setting
    override the setting
    start a sensitivity scan manually
    environment settings
    environment setting
    Reviewing and applying recommended generators
    built-in sensitivity type
    Creating and managing custom sensitivity rules
    Data flow and privacy in Structural AI
    Structural Cloud LLM configuration
    Self-hosted LLM configuration
    ask for AI assistance to troubleshoot the failure
    environment setting

    Optionally, in the Model Name field, specify the model to use. If you do not specify a model, then the default is:

    • For lightweight, quick tasks, claude-haiku-4-5.

    • For standard tasks, claude-sonnet-5.

    • For complex, high-quality tasks, claude-opus-5.

  • To test the connection, click Test Anthropic Connection.

  • TONIC_LLM_MODEL - The model to use.

    Optionally, in the Model Name field, specify the model to use. If you do not specify a model, then the default is:

    • For lightweight, quick tasks, gpt-4.1-mini.

    • For standard tasks, gpt-4.1.

    • For complex, high-quality tasks, gpt-5.2.

  • To test the connection, click Test OpenAI Connection.

  • TONIC_LLM_MODEL - The model to use.

    Supported providers

    Using the hosted LLM

    Using AI settings view

    Using environment settings

    Allowlisting the SaaS proxy

    Disabling AI features on Structural

    Using Anthropic

    Required models

    Using AI settings to configure the Anthropic connection

    Using environment settings to configure the Anthropic connection

    Using Anthropic on Amazon Bedrock

    Required IAM permissions for Amazon Bedrock

    Actions

    Resources

    AWS Regions

    Selecting Anthropic on Amazon Bedrock as your LLM provider

    Using the environment configuration for authentication

    Using an Amazon Bedrock API key for authentication

    Selecting the model to use

    Using environment settings to configure the Anthropic on Amazon Bedrock connection

    Using OpenAI

    Supported deployments

    Required model deployments for hosted deployments

    Using AI Settings to configure the OpenAI connection

    Using environment settings to configure the OpenAI connection

    Configuring AI-based Structural functionality

    Disabling LLM-based sensitivity detection

    Enabling LLM-based sensitivity rules

    Excluding sample data from LLM-based sensitivity detection

    Excluding sample data from all LLM-based features

    environment settings
    configure data use by the LLM
    environment settings
    environment settings
    cross-Region inference profiles
    model support by AWS Region
    AWS documentation
    environment settings
    AWS SDK default resolving chain
    environment settings
    environment settings
    LLM-based sensitivity detection
    environment setting
    environment setting
    environment setting
    AI Settings page with configuration to not use an LLM
    AI Settings page configured to use Anthropic as the LLM provider
    AI Settings page configured to use Anthropic on Amazon Bedrock using environment credentials
    AI Settings page configured to use Anthropic on Amazon Bedrock using an API key
    AI Settings page configured to use OpenAI as the LLM provider
    .
    .
    environment settings
    environment setting
    environment settings
    Date Range
  • Datetime

  • Integer

  • JSON

  • MAC Address

  • Network Address

  • Text

  • UUID

  • XML

  • ,
    Ends with
    .
  • In the comparison text field, provide the text to check for. The comparison text is case insensitive. For example, if you set a condition to match column names that contain the text term, it also matches column names that contain TERM or Term or tErM.

  • To remove a column name condition, click its delete icon.

  • Enabling LLM-based sensitivity rules

    Displaying the list of custom sensitivity rules

    Filtering the rules

    Setting the rule sequence

    Creating and editing a sensitivity rule

    Creating a sensitivity rule

    Editing a sensitivity rule

    Sensitivity rule configuration

    Rule name

    Rule description

    Rule type

    Data type

    Column name criteria

    Configuring text matching conditions

    Providing a regular expression

    Generator preset to apply

    Managing generator preset configuration

    Required global permission: Create and manage generator presets

    Previewing the rule results

    Deleting a sensitivity rule

    Structural Cloud LLM configuration
    Self-hosted LLM configuration
    configure the new rule
    update the configuration
    Sensitivity Rules view with the lists of custom sensitivity rules
    Details view for a custom sensitivity rule
    Match Type setting to determine the type of custom sensitivity rule
    Column name text match rules for a custom sensitivity rule
    Column name regular expression field for a custom sensitivity rule
    Test Results section to preview the results for a sensitivity rule

    Writing output to a container repository

    Requires Kubernetes.

    For self-hosted Docker deployments, you can install and configure a separate Kubernetes cluster to use. For more information, go to .

    For information about required Kubernetes permissions, go to .

    Not compatible with upsert.

    Not compatible with Preserve Destination or Incremental table modes.

    Only supported for PostgreSQL, MySQL, and SQL Server.

    You can configure a workspace to write destination data to a container repository instead of to a database server.

    Diagram showing how data is written to and accessed from a container artifact

    When Structural writes data generation output to a repository, it writes the destination data to a container volume. From the list of container artifacts, you can copy the volume digest, and download a Docker Compose file that provides connection settings for the database on the volume. Structural generates the Compose file when you make the request to download it. For more information about getting access to the container artifacts, go to Viewing and downloading container artifacts.

    For an overview of writing destination data to container artifacts, you can also view the video tutorial.

    Indicating to write destination data to container artifacts

    Under Destination Settings, to indicate to write the destination data to container artifacts, click Container Repository.

    For a Structural instance that is deployed on Docker, unless you set up a separate Kubernetes cluster, the Container Repository option is hidden.

    You can switch between writing to a database server and writing to a container repository at any time. Structural preserves the configuration details for both options. When you run data generation, it uses the currently selected option for the workspace.

    From the Database Image dropdown list, select the image to use to create the container artifacts.

    Select an image version that is compatible with the version of the database that is used in the workspace.

    For a MySQL workspace, you can provide a customization file that helps to ensure that the temporary destination database is configured correctly.

    To provide the customization details:

    1. Toggle Use customization to the on position.

    2. In the text area, paste the contents of the customization file.

    To provide the location where Structural publishes the container artifacts:

    1. In the Registry field, type the path to the container registry where Structural publishes the data volume.

      Do not include the HTTP protocol, such as http:// or https://.

    2. In the Repository Path field, provide the path within the registry where Structural publishes the data volume.

      For a Google Artifact Registry (GAR) repository, the path format is PROJECT-ID/REPOSITORY/IMAGE.

    You next provide the credentials that Structural uses to read from and write to the registry.

    When you provide the registry, Structural detects whether the registry is from Amazon Elastic Container Registry (Amazon ECR), Google Artifact Registry (GAR), or a different container solution.

    It displays the appropriate fields based on the registry type.

    For a registry other than an Amazon ECR or a GAR registry, the credentials can be either a username and access token, or a secret.

    In general, the credentials must be for a user that has read and write permissions for the registry.

    The secret is the name of a Kubernetes secret that lives on the pod that the Structural worker runs on. The secret type must be kubernetes.io/dockerconfigjson. The Kubernetes documentation provides information on .

    To use a username and access token:

    1. Click Access token.

    2. In the Username field, provide the username.

    3. In the Access Token field, provide the access token.

    To use a secret:

    1. Click Secret name.

    2. In the Secret Name field, provide the name of the secret.

    For ACR, the provided credentials must be for a service principal that has sufficient permissions on the registry.

    For Structural, the service principal must at least have the permissions that are associated with the.

    For a GAR registry, you upload a service account file, which is a JSON file that contains credentials that provide access to Google Cloud Platform (GCP).

    The associated service account must have the Artifact Registry Writer role.

    For Service Account File, to search for and select the file, click Browse.

    For an Amazon ECR registry, you can either:

    • Provide the AWS access and secret key that is associated with the IAM user that will connect to the registry

    • Provide an assumed role

    • (Self-hosted only) Use the credentials configured in the Structural environment settings TONIC_AWS_ACCESS_KEY_ID and TONIC_AWS_SECRET_ACCESS_KEY.

    To provide an AWS access key and secret key:

    1. Click Access Keys.

    2. In the AWS Access Key field, enter an AWS access key that is associated with an IAM user or role.

    3. In the AWS Secret Key field, enter the secret key that is associated with the access key.

    4. Optionally, in the AWS Session Token

    To provide an assumed role:

    1. Click Assume Role.

    2. In the Role ARN field, provide the Amazon Resource Name (ARN) for the role.

    3. In the Session Name field, provide the role session name. If you do not provide a session name, then Structural automatically generates a default unique value. The generated value begins with TonicStructural.

    For the assumed role, Structural generates the external ID that is used in the assume role request. Your role’s trust policy must be configured to condition on your unique external ID.

    Here is an example trust policy:

    On a self-hosted instance, to use the credentials configured in the environment settings, click Environment Variables.

    On a self-hosted instance, to use the AWS credentials from the EC2 instance, click Instance Profile.

    The IAM user must have permission to list, push, and pull images from the registry. The following example policy includes the required permissions.

    For additional security, a repository name filter allows you to limit access to only the repositories that are used in Structural. You need to make sure that the repositories that you create for Structural match the filter.

    For example, you could prefix Structural repository names with tonic-. In the policy, you include a filter based on the tonic- prefix:

    In the Tags field, provide the tag values to apply to the container artifacts. You can also change the tag configuration for individual data generation jobs.

    Use commas to separate the tags.

    A tag cannot contain spaces. Structural provides the following built-in values for you to use in tags:

    • {workspaceId} - The identifier of the workspace.

    • {workspaceName} - The name of the workspace.

    • {timestamp} - The timestamp when the data generation job that created the artifact completed.

    For example, the following creates a tag that contains the workspace name, job identifier, and timestamp:

    {workspaceName}_{jobId}_{timestamp}

    To also tag the artifacts as latest, check the Tag as "latest" in your repository checkbox.

    You can also optionally configure custom resource values for the Kubernetes pods. You can specify the ephemeral storage, memory, and CPU millicores.

    To provide custom resources:

    1. Toggle Set custom pod resources to the on position.

    2. Under Storage Size:

      1. In the field, provide the number of megabytes or gigabytes of storage.

    In the Custom Database Name field, provide the name to use for the destination database.

    If you do not provide a custom database name, then the destination database uses the same name as the source database.

    In the Custom Password field, provide the password for the destination database user.

    If you do not provide a password, then Structural generates a password.

    The destination database username is always the default user for the database:

    • For PostgreSQL, postgres

    • For MySQL, root

    • For SQL Server, sa

    If your Kubernetes nodes are configured with taints, then on a self-hosted instance, you can configure the tolerations that enable the datapacker pods to be scheduled on the nodes. The datapacker pod hosts the temporary database that Structural uses during the data generation.

    For an overview of taints and tolerations, go to the .

    To configure the tolerations, you configure the following . You can add these settings to the Environment Settings list on Structural Settings.

    • CONTAINERIZATION_POD_NODE_TOLERATION_KEY - The toleration key value to apply to the datapacker pods. This setting is required. If you do not configure this setting, then Structural ignores the other settings.

    • CONTAINERIZATION_POD_NODE_TOLERATION_VALUES - A comma-separated list of toleration values to apply to the datapacker pods.

    • CONTAINERIZATION_POD_NODE_TOLERATION_EFFECT - The toleration effect to apply to the datapacker pods.

    For more information about repository and image names, go to the .

    (Self-hosted only) If Structural is deployed in Amazon Elastic Kubernetes Service (Amazon EKS), then you can use the AWS credentials that live on the EC2 instance.

    field, enter the session token to use for the connection.
    In the Duration (in seconds) field, provide the maximum length in seconds of the session. The default is 3600, indicating that the session can be active for up to 1 hour. The provided value must be less than the maximum session duration that is allowed for the role.

    {jobId} - The identifier of the data generation job that created the artifact.

    From the dropdown list, select the unit to use.

    The storage can be between 32MB and 25GB.

  • Under Memory Size:

    1. In the field, provide the number of megabytes or gigabytes of RAM.

    2. From the dropdown list, select the unit to use.

    The memory can be between 512MB and 4 GB.

  • Under Processor Size:

    1. In the field, provide the number of millicores.

    2. From the dropdown list, select the unit.

    The processor size can be between 250m and 1000m.

  • CONTAINERIZATION_POD_NODE_TOLERATION_OPERATOR - The toleration operator to apply to the datapacker pods.

    {
      "Version": "2012-10-17",
      "Statement": {
        "Effect": "Allow",
        "Principal": {
          "AWS": "<originating-account-id>"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
          "StringEquals": {
            "sts:ExternalId": "<external-id>"
          }
        }
      }
    }
    {
      {
        "Sid": "ManageTonicRepositoryContents",
        "Effect": "Allow",
        "Action": [
          "ecr:DescribeRepositories",
          "ecr:ListImages",
          "ecr:DescribeImages",
          "ecr:BatchGetImage",
          "ecr:BatchCheckLayerAvailability",
          "ecr:InitiateLayerUpload",
          "ecr:UploadLayerPart",
          "ecr:CompleteLayerUpload",
          "ecr:PutImage"
        ],
        "Resource": [
           "arn:aws:ecr:<region>:<account_id>:repository/<optional name filter>"
        ]
      },
      {
        "Sid": "GetAuthorizationToken",
        "Effect": "Allow",
        "Action": [
          "ecr:GetAuthorizationToken"
        ],
        "Resource": "*"
      }
    }
    "Resource": [
      "arn:aws:ecr:<region>:<account_id>:repository/tonic-*"
    ]

    Identifying the base image to use to create the container artifacts

    Providing a customization file for MySQL

    Setting the location for the container artifacts

    Providing the credentials to write to the registry

    Fields for registries other than Amazon ECR or GAR

    The option to use a secret is not available on Structural Cloud.

    Azure Container Registry (ACR) permission requirements

    Providing a service file for GAR

    Structural only supports Google Artifact Registry (GAR). It does not support Google Container Registry (GCR).

    Amazon ECR registries

    Using AWS access keys

    Using an assumed role

    Using the credentials from the environment settings (self-hosted only)

    Using the AWS credentials from the EC2 instance (self-hosted only)

    Required permissions for the IAM user

    Providing tags for the container artifacts

    Specifying custom resources for the Kubernetes pods

    Setting a custom database name

    Only available for PostgreSQL and SQL Server. Not available for MySQL.

    Setting a custom database user password

    Configuring the required tolerations for datapacker node taints

    how to create a registry credentials secret
    AcrPush role
    Kubernetes documentation
    environment settings
    Setting up a Kubernetes cluster to use to write output data to a container repository
    Required access to write destination data to a container repository

    About Tonic Structural

    The Tonic Structural synthetic data platform combines sensitive data detection and data transformation to allow users to create safe, secure, and compliant datasets.

    Common Structural use cases include creating staging and development environments, and trying out a new cloud provider without complex data agreements.

    Structural allows you to reduce bug counts, shorten testing life cycles, and share data with partners, all while helping to ensure security and compliance with the latest regulations, from GDPR to CCPA.

    You can use the Structural API to integrate with CI/CD pipelines or to create automated processes that ensure that the generated data is available on demand.

    Google Cloud documentation

    Structural data generation workflow

    Overview of the Structural steps to generate de-identified data.

    Structural deployment types

    You can use Structural Cloud or set up a self-hosted Structural instance.

    Structural implementation roles

    Functions that participate in a Structural implementation.

    Structural license plans

    View the license options and their available features.

    Languages and international values

    How Structural supports non-English languages and values from outside of the United States.

    Providing your own client certificates

    For data connectors that support it, to specify your own client certificates for authentication:

    1. Click the expand icon for Client certificate settings.

    2. The settings can include one or more of the following:

      1. For Client Cert, choose the client certificate file.

      2. For Client Key, choose the key file for the client certificate.

      3. For Root Cert, choose the root certificate file.

    For all of these settings, if secrets managers are available, you can instead .

    Structural Cloud LLM configuration

    For its AI-enhanced features, Structural Cloud uses Opus 5, Sonnet 5, and Haiku 4.5 from Anthropic deployed on Amazon Bedrock.

    Cloud organizations can choose not to use any Structural AI features.

    On the of Structural Settings, toggle TONIC_LLM_PROVIDER to the off position.

    By default, LLM-based sensitivity detection sends both the database schema and sample source data values to the LLM.

    To exclude the contextual source data, and only send the schema, on the Organization Settings tab of Structural Settings, set TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS_SAMPLE_DATA to false.

    On Structural Cloud, LLM-based sensitivity detection is enabled by default.

    You can disable LLM-based sensitivity detection within individual workspaces.

    Structural AI features

    Tonic Structural currently offers the following AI features.

    For information about the associated flows for these features, go to .

    For information about how to configure LLM connections and usage, go to or .

    The is a chat-based AI assistant that helps you to configure your workspace for data generation.

    During sensitivity detection, the uses the schema and, optionally, sample source data values to help to identify sensitive columns.

    LLM-based use the sensitivity rule description to identify matching columns.

    They are processed during LLM-based sensitivity detection.

    For the generator, you can provide a prompt for the LLM to use to create the available values.

    For example, 1,000 common pharmaceuticals

    Connecting through an SSH bastion

    An SSH bastion is a hardened, publicly accessible server that provides a single entry point into a private network.

    When you use an SSH bastion, you do not need to allow direct access from Structural to to your database servers. Instead, you only allow access to the bastion. Structural then SSHes through the bastion to reach the servers.

    Before you configure a workspace to connect through an SSH bastion, make sure to configure the bastion as follows.

    To offload encryption tasks, the CPU must support the AES-NI instruction set.

    For example, on AWS, you would use Generation 4 instances, such as C4 or M4, or newer.

    Avoid burstable instances, such as AWS t3 and t3a. During high-throughput transfers, these can be throttled, which can lead to dropped connections.

    Use instances that have dedicated CPU resources.

    High-speed ciphers require a modern version of OpenSSH (OpenSSH 6.5+ ).

    We recommend that you privilege ChaCha20-Poly1305 and AES-GCM.

    Manually indicating whether a column is sensitive

    You can also manually indicate that a column is sensitive or not sensitive.

    For example, the sensitivity scan might incorrectly identify a column as sensitive. Or a column might contain data that you consider sensitive but that does not match a detected sensitivity type.

    When you manually change a column from not sensitive to sensitive, Structural marks the sensitivity detection as full confidence.

    To change the sensitivity, you can prompt the Structural Agent. For example:

    • Change all datetime columns to be not sensitive.

    select a secret name from a secrets manager

    To do this, in the advanced workspace overrides, set TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS to false.

    LLM-based sensitivity detection also processes LLM-based sensitivity rules. If you disable LLM-based sensitivity detection, then the sensitivity scan does not process any LLM-based sensitivity rules.

    To configure Structural to not send sample data for any LLM-based features at all, on the Organization Settings tab of Structural Settings, set TONIC_LLM_ENABLE_SAMPLE_DATA to false.

    Models used on Structural Cloud

    Configuring AI-based Structural functionality

    Opting out of all Structural AI features

    Excluding sample data from LLM-based sensitivity detection

    Disabling LLM-based sensitivity detection in a workspace

    Organization Settings page

    Excluding sample data from all LLM-based Structural features

    .

    For the Text Composition generator, you can provide a prompt for the LLM to use to create the template.

    For example, Use the first character of the FIRST_NAME column, followed by a period and the value of the LAST_NAME column, followed by email.com, all lowercase.

    When a data connection test fails, Structural displays an Ask AI prompt that sends a request to an LLM to troubleshoot the issue and suggest next steps.

    For the the following types of jobs:

    • Sensitivity scans

    • Data generation jobs

    • Upsert jobs

    • Collection scans

    The job details include Ask AI prompt that sends a request to an LLM to troubleshoot the issue and suggest next steps.

    The Structural MCP server allows you to use AI clients such as Claude Code to interact with Structural using the same natural language prompts that you provide in the Structural Agent.

    For information on how to set up access to the MCP server, go to Using the Structural MCP server.

    Structural Agent

    LLM-based sensitivity detection

    LLM-based sensitivity rules

    Custom Categorical value creation

    Data flow and privacy in Structural AI
    Structural Cloud LLM configuration
    Self-hosted LLM configuration
    Structural Agent
    LLM-based detection process
    sensitivity rules
    Custom Categorical

    Text Composition template creation

    Data connection test failure troubleshooting

    Job failure troubleshooting

    Required workspace permission: Download job logs

    Structural MCP server

    To prioritize high-speed encryption, in /etc/ssh/sshd_config, add or update the Ciphers line:

    To allow for high-concurrency and multiplexing, and prevent Structural from being throttled, at the end of /etc/ssh/sshd_config, add this Match block:

    In the workspace configuration, to connect to an SSH bastion :

    1. If the data connector supports multiple tunneling options:

      1. Toggle Enable Connection Tunnel to the on position.

      2. From the Tunnel type dropdown list, select SSH Tunnel.

      If the data connector only supports SSH tunneling, then toggle Enable SSH Tunnel to the on position.

    2. In the SSH Host field, provide the host for the SSH bastion.

    3. In the SSH Port field, provide the port for the SSH bastion.

    4. In the SSH User field, provide the name of the user to use to connect to the SSH bastion.

    5. If you do not use a private key, then in the SSH Passphrase field, provide the passphrase to use for authentication. If secrets managers are available, you can instead .

    6. If you do use a private key, then in the SSH Private Key field, provide the private key. If secrets managers are available, you can instead . If the private key uses a passphrase, then in the SSH Passphrase field, provide the passphrase for the private key.

    Setting up your SSH bastion

    CPU architecture

    Instance type

    Software and cipher compatibility

    Prioritize high-speed encryption

    Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
    # Global setting for unauthenticated connections
    MaxStartups 100:30:200
    
    # Specific limits for Structural Source IP
    Match Address <Structural_Source_IP>
        MaxAuthTries 100
        MaxSessions 1000

    Allowlist Structural

    Connecting to an SSH bastion from a Structural workspace

    Change the Occupation column to be sensitive.

    For information on other ways to change whether a column is sensitive:

    • For Privacy Hub, go to .

    • For Database View, go to:

      • For a single column, Indicating whether a column is sensitive

      • For multiple selected columns, Marking the columns as sensitive or not sensitive

    • For Table View, go to .

    The Structural API also provides endpoints to designate columns as sensitive or not sensitive.

    Character Substitution

    Performs a random character replacement that preserves formatting (spaces, capitalization, and punctuation).

    Characters are replaced with other characters from within the same Unicode Block. A given source character is always mapped to the same destination character. For example, M might always map to V.

    For example, for the following input string:

    Miami Store #162

    The output would be something like:

    Vgkjg Gmlvf #681

    Note that for a numeric column, when a generated number starts with a 0, the starting 0 is removed. This could result in matching output values in different columns. For example, one column is changed to 113 and the other to 0113, which also becomes 113.

    Character Substitution is similar to , with a couple of key differences. Because Character Substitution always maps the same source character to the same destination character, it is always consistent. It also can be used for unique columns.

    In Character Scramble, the character mapping is random, which makes Character Scramble slightly more secure. However, Character Scramble cannot be used for unique columns.

    If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Identifying sensitive data

    Tonic Structural uses its sensitivity scan to identify source data columns that contain sensitive information. The scan ignores truncated tables.

    The sensitivity scan identifies Structural's built-in sensitivity types. It also looks for custom types that you define.

    You can also manually mark a column as sensitive or not sensitive.

    Using secrets managers for authentication

    Required license: Enterprise

    Your organization might use a secrets manager to secure credentials, including database connection credentials.

    You can configure a set of available secrets managers. In the workspace configuration, for fields that support it, you can then select a secret name from a secrets manager.

    Logging into Structural for the first time

    When you go to Tonic Structural for the first time, you create an account. How you create an account depends on the type of user you are.

    A new Structural user can be one of the following:

    • A completely new user who is starting a Structural 14-day free trial. Free trial users use Structural Cloud to explore and experiment with Structural before they decide whether to purchase it.

    • A new user on a self-hosted Structural instance. Self-hosted instances are installed on-premises. The customer administers the Structural users.

    • New users are added to existing organizations based on their email domain.

    Character Scramble

    This generator replaces letters with random other letters and numbers with random other numbers. Punctuation, whitespace, and mathematical symbols are preserved.

    For example, for the following input string:

    ABC.123 123-456-789 Go!

    The output would be something like:

    PRX.804 296-915-378 Ab!

    This generator securely masks letters and numbers. There is no way to recover the original data.

    Character Scramble is similar to , with a couple of key differences.

    While you can enable consistency for the entire value, Character Scramble does not always replace the same source character with the same destination character. Because there is no guarantee of unique values, you cannot use Character Scramble on unique columns.

    Company Name

    Generates a random company name-like string.

    Run the Structural sensitivity scan

    Run, configure, and get the results of the sensitivity scan.

    Set column sensitivity manually

    Options to override the results of the sensitivity scan.

    Built-in sensitivity types

    Types of sensitive data that the sensitivity scan can identify.

    Configure custom sensitivity rules

    Set up rules to enable the scan to identify other sensitive columns based on the column data types and names.

    Configure available secrets managers Set up and test the secrets managers that can be used.

    Select a secret from a secrets manager In the workspace configuration, identify a secret to use for the connection.

    select a secret name from a secrets manager
    select a secret name from a secrets manager
    A new user in an existing Structural Cloud organization.

    Data-free

    No

    Allowed for primary keys

    Yes

    Allowed for unique columns

    Yes

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    4

    Generator ID (for the API)

    Consistency

    This generator is implicitly self-consistent. You do not specify whether the generator is consistent. Every occurrence of a character always maps to the same substitute character. Because of this, it can be used to preserve a join between two text columns, such as a join on a name or email.

    Linking

    No, cannot be linked.

    Differential privacy

    No

    Characteristics

    How to configure

    Character Scramble
    Structural data encryption
    Character Substitution, however, does always map the same source character to the same destination character. Character Substitution is always consistent, which makes it less secure than Character Scramble. You can use Character Substitution on unique columns.

    Consistency

    Yes, can be made self-consistent

    Linking

    No, cannot be linked

    Differential privacy

    No

    To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

    By default, the generator is not consistent.

    If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Character Substitution

    Characteristics

    How to configure

    Differential privacy

    Yes, if consistency is not enabled.

    Data-free

    Yes, if consistency is not enabled.

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • 1 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    To configure the generator, toggle the Consistency setting to indicate whether to make the generator consistent.

    By default, the generator is not consistent.

    If consistency is enabled, then by default it is self-consistent. To make the generator consistent with another column, from the Consistent to dropdown list, select the column.

    When the generator is consistent with itself, then a given source value is always mapped to the same destination value. For example, My Company is always mapped to New Company.

    When the generator is consistent with another column, then a given source value in that other column always results in the same destination value for the company name column. For example, if the company name column is consistent with a name column, then every instance of John Smith in the name column in the source database has the same company name in the destination database.

    If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Consistency

    Yes, can be made self-consistent or consistent with another column.

    Linking

    No, cannot be linked.

    This generator is deprecated. Use the Business Name generator instead.

    Characteristics

    How to configure

    Advanced workspace overrides

    For self-hosted instances, Structural provides to configure features that include:

    • Consistency across runs and databases

    • Data generation performance

    The Advanced Workspace Overrides section of the workspace details view allows you to override those environment settings for an individual workspace.

    For example, the environment setting TONIC_TABLE_PARALLELISM determines the number of tables that Structural processes simultaneously. You can then override that value within individual workspaces.

    AI in Structural

    Tonic Structural supports AI features to improve sensitivity detection and data generation. These features are enabled by default on Structural Cloud. Self-hosted customers must first connect to an LLM provider.

    Data-free

    No

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • 3 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    TextMaskGenerator

    StringMaskGenerator
    CompanyNameGenerator

    AI features Structural features that use AI.

    Data flow and privacy Flows for LLM-based features. How LLM requests are stored and used.

    LLM on Structural Cloud Configure LLM use on Structural Cloud.

    LLM on a self-hosted instance Connect to an LLM provider and configure LLM use.

    The workspace overrides are available on both self-hosted instances and on Structural Cloud.

    By default, the list is collapsed.

    Initial collapsed display of the Advanced Workspace Overrides section

    To display the available override settings, expand the list.

    Expand list of settings in the Advanced Workspace Overrides section for a workspace

    You can use the search field to find a specific setting. The search looks for matches in both the setting name and the setting description.

    Applying a search filter to the advanced workspace overrides

    For information on how to configure the statistics seed, go to .

    For other settings, to enable the override and set the override value:

    1. Toggle the setting to the on position.

    Enabling and setting an override value
    1. Set the value.

    On the workspace settings page, the configured settings display above the collapsed list of unset settings.

    Advanced Workspace Overrides section with a configured setting

    To remove the override, toggle the setting to the off position.

    For generators where consistency is enabled, a statistics seed enables consistency across data generation runs. The Structural-wide statistics seed value ensures consistency across both data generation runs and workspaces.

    You use the Override Statistics Seed setting to override the Structural-wide statistics seed value.

    You can either disable consistency across data generations, or provide a seed value for the workspace. The workspace seed value ensures consistency across data generation runs for that workspace, and across other workspaces that have the same seed value.

    For details about using seed values to ensure consistency across data generation runs and databases, go to .

    For document-based databases, you can configure the maximum amount of time in seconds to scan a schema.

    For sensitivity scans, you can configure the number of minutes after which a scheduled scan times out.

    From Advanced Workspace Overrides, you can override both of these timeouts.

    Structural provides environment settings to manage data generation performance. For example, these settings include configuration for parallel processing and client-side compression.

    From Advanced Workspace Overrides, you can override some of these data generation performance settings for an individual workspace.

    To use Structural data encryption, you must provide encryption and decryption keys.

    You use the Override Data Decryption Key and Override Data Encryption Key settings to override the Structural-wide keys that are provided in the environment settings.

    Some data connectors allow you to configure whether you provide the schema for the destination database. For more information, go to related information for Databricks, MySQL, Oracle, Snowflake, and SQL Server.

    From Advanced Workspace Overrides, you can override the instance-wide configuration.

    Databricks allows you to configure how Structural handles overwrites of existing data.

    You use the Workspace Default Error on Override and Workspace Default Save Mode settings to override the instance-wide configuration.

    Structural provides an environment setting to determine whether to automatically run a sensitivity scan when you create a workspace, copy a workspace, or change the connection details.

    From Advanced Workspace Overrides, you can override the instance-wide configuration.

    By default, LLM-based sensitivity detection is enabled on Structural Cloud.

    You use the Tonic LLM Enable Enhanced Recommendations setting to determine whether to enable LLM-based sensitivity detection in the workspace.

    For information about LLM-based sensitivity detection, go to LLM-based sensitivity detection - Medium confidence.

    environment settings

    Configuring the overrides

    Enabling and setting an override

    Removing an override

    Available overrides

    Workspace statistics seed for cross-run consistency

    Collection and sensitivity scan timeouts

    Data generation performance settings

    Data encryption and decryption keys

    Destination database schema creation

    Overwrite handling for Databricks

    Enable or disable event-based sensitivity scans

    Enable or disable LLM-based sensitivity detection on Structural Cloud

    Viewing your list of workspaces

    Workspaces view lists the workspaces that you have access to. To display Workspaces view, in the Tonic Structural heading, click Workspaces.

    Workspaces view

    How the workspace list is displayed

    The workspace list contains:

    • Workspaces that you own

    • Workspaces that you are granted access to

    If you have the team permission Copy any workspace or Manage user access within the team, then list includes all of the workspaces.

    The Permissions column lists the workspace permission sets that you are granted in each workspace. The permission sets include both permission sets that were granted to you directly as a user, and permission sets that were granted to an SSO group that you are a member of.

    always display under their parent workspace. The list only includes child workspaces that you have access to. If you have access to a child workspace, but not to its parent workspace, then the parent workspace is grayed out. You cannot select it.

    You can filter the workspaces based on the following information:

    • Name - In the filter field, begin to type text that is in the name of the workspaces to display in the list.

    • Owner - From the Filter by Owner dropdown list, select the owner of the workspaces to display in the list.

    • Database type - From the Filter by Database Type dropdown list, select the type of database for the workspaces to display in the list.

    You can combine different filters. For example, you can filter the list to only include workspaces that use PostgreSQL and for which the generation status is Canceled or Failed.

    Child workspaces always display under their parent workspace, even if the parent workspace does not match the filter.

    You can sort the workspace list by name, status, or owner.

    By default, the list is sorted alphabetically by name.

    To sort by a column, click the column heading. To reverse the order of the sort, click the column heading again.

    Child workspaces always display under their parent workspace. The child workspaces are sorted within the parent.

    Workspaces view provides the following information about each workspace:

    • Name - Contains the name and database type for the workspace. To view the workspace description, hover over the name.

    • Generation status - The status for the most recent generation job. To display the job details for the job, click the job status. To display more details about the date, time, and duration for the job, hover over the generation timestamp. If a job failed recently, you are given additional information about how long this job has been failing (the date of the first failure occurrence among a continuous series of failures).

    • Schema changes - Indicates whether Structural detected changes to the source database schema. If there are changes, the column shows the number of changes. Hover over the column value to display additional details, and to navigate to the Schema Changes

    On Workspaces view, when you click the workspace name, the for the workspace is displayed. The Privacy Hub tab is selected.

    The Name column also provides access to a menu of workspace configuration options. When you select an option, the is displayed, open to the view for the selected option.

    The last column in the workspaces list provides additional workspace options:

    • Subsetting icon - Displays the subsetting configuration for the workspace. Go to .

    • Post-job actions icon - Displays the post-job actions for the workspace. For more information, go to and .

    • Actions menu - Provides access to additional options.

    The Actions menu at the top left of the workspaces list allows you to to perform bulk actions on multiple workspaces. It is enabled when you check one or more of the checkboxes in the first column of each row. The Actions menu provides options for the selected workspaces.

    Structural data generation workflow

    Tonic Structural data generation combines sensitive data detection and data transformation to create safe, secure, and compliant datasets.

    The Structural data generation workflow involves the following steps:

    Overview diagram of the Tonic Structural data generation workflow

    You can also view this video overview of the Structural data generation workflow.

    1. To get started, you create a workspace. When you create a workspace, you identify the type of source data, such as PostgreSQL or MySQL, and establish the connections to the source database and the destination location. The source database contains the original data that you want to synthesize. The destination location is where Structural stores the synthesized data. It might be a database, a storage location, or a container repository.

    2. Next, you . The sensitivity scan identifies columns that contain sensitive data. These columns need to be protected by a generator.

    3. Based on the sensitivity scan results, you configure the data generation. The configuration includes:

      • The table mode controls the number of rows and columns that are copied to the destination database.

      • You can make adjustments to the initial sensitivity assignments. For example, you can mark additional columns as sensitive that the initial scan did not identify as sensitive.

    4. After you complete the configuration, you . The data generation job uses the configured table modes and generators to transform the data from the source database and write the transformed data to the destination location. You can track the job progress and view the job results.

    Workspace identification and worker zone

    The Workspace Details section for every workspace includes the following settings to identify the workspace and to route the workspace jobs to a specific set of Structural workers.

    Fields to identify the workspace

    All workspaces have the following fields that identify the workspace:

    1. In the Name field, enter the name of the workspace.

    2. In the Tags field, provide a comma-separated list of tags to assign to the workspace. For more information on managing tags, go to Assigning tags to a workspace.

    3. In the Description field, provide a brief description of the workspace. The description can contain up to 200 characters.

    Workspace jobs are processed by Tonic Structural workers.

    Each can optionally be assigned to a zone. Each zone can contain multiple workers.

    Structural Cloud provides a set of zones, as well as a set of workers that are not assigned to zones.

    On a self-hosted instance, you can .

    On the workspace settings view, from the Zone dropdown list, select the name of the zone to route the workspace jobs to.

    If no worker zones are configured, then the dropdown is not displayed.

    By default, a workspace is not assigned to a zone. The workspace jobs are processed by a worker that is also not assigned to a zone.

    Note that the following cases cause the workspace jobs to not be processed:

    • A workspace is not assigned to a zone, but all of the workers are assigned to zones.

    • A workspace is assigned to a zone, but the zone has no assigned workers.

    Tutorial videos

    Use these tutorial videos to learn more about how to use Tonic Structural.

    For the full set of tutorial and demo videos, go to the .

    Provides an overview of the Structural workflow and how to use Structural to generate de-identified data. For more information, go to .

    Provides an overview of what a Structural workspace is and how to create a new Structural workspace. For more information, go to .

    Provides an overview of how Structural detects sensitive values and how you can apply recommended generators to the detected values.

    Provides an overview of workspace owners, permissions, and permission sets. Explains how to share and transfer ownership of a workspace. For more information, go to .

    Identifies the types of generators and transformations that you can use in Structural, and explains how to assign a generator to a column. For more information, go to .

    Built-in sensitivity types that Structural detects

    Structural identifies the following types of sensitive values. These include some information types that are considered by many privacy standards and frameworks such as HIPAA, GDPR, CCPA, and PCI.

    For more information about the HIPAA and Safe Harbor information types that Structural detects, go to the Tonic.ai guide .

    Names

    • First

    • Last

    Workspace job routing zone

    About worker zones

    Assigning a workspace to a zone

    configure the list of zones and assign your Structural workers to those zones
    Full

    Organization

    Location

    • Street address

    • ZIP

    • PO Box

    • City

    • State and two-letter abbreviation

    • Country

    • Postal code

    • GPS coordinates

    Contact information

    • Email address

    • Telephone number

    User credentials

    • Username

    • Password

    Financial information

    • Credit card number

    • International bank account number (IBAN)

    • SWIFT code for bank transfers

    • Money amount

    • BTC (Bitcoin) address

    Identification

    • Social Security Number

    • Passport number

    • Driver's license number

    • Birth date

    • Gender

    • Biometric identifier, such as a fingerprint or voiceprint

    • Full face photographic images and similar images

    Medical information

    • ICD-9 and ICD-10 codes (Used to identify diseases)

    • Medical record number

    • Health plan beneficiary number

    • Admission date

    • Discharge date

    • Date of death

    Other personal information

    • Marital status

    Accounts and licenses

    • Account number

    • Certificate or license number

    Network and web location

    • IP address

    • IPv6 address

    • MAC address

    • Web URL

    International Mobile Equipment Identity (IMEI)

    Vehicle information

    • Vehicle identification number (VIN)

    • License plate number

    Using Tonic Structural and the Safe Harbor method to de-identify PHI
    Assigning and configuring column generators. To protect the data in a column, especially a sensitive column, you assign a generator to it. The generator replaces the source value with a different value in the destination database. For example, the generator might scramble the characters or assign a random value of the same type.
    analyze the results of the initial sensitivity scan
    Assigning table modes to tables.
    Indicating column sensitivity.
    run the data generation job

    Generation status - In the Generation Status column heading, click the filter icon. Check the checkbox next to the generation status values for the workspaces to display in the list.

  • Tags - In the Tags column heading, click the filter icon. By default, the workspaces are not filtered by tag, and all of the checkboxes are unchecked. To only include workspaces that have specific tags, check the checkbox next to each tag to include. To uncheck all of the selected tags, click Reset Tags. When you filter by tag, Structural checks whether each workspace contains any of the selected tags.

  • Permissions - In the Permissions column heading, click the filter icon. You can check and uncheck checkboxes to include or exclude specific permission sets. For example, you can filter the list to only display workspaces for which the Editor permission set is granted either to you or to an SSO group that you belong to. For users that have the team permission Copy any workspace, the Permissions filter panel also contains an Any permissions checkbox. By default, Any permissions is unchecked, and the list includes workspaces for which you are not assigned any workspace permission sets. To display all of the workspaces for which you have any assigned workspace permission sets, check Any permissions. If you filter the list based on a specific permission set, to clear the filter and show all workspaces for which you have any permission set, check Any permissions. To display all workspaces, including workspaces that you do not have any permissions for, uncheck Any permissions.

  • view. Go to
    .
  • Tags - The tags that are assigned to the workspace.

  • Permissions - The permission sets that are assigned to you for the workspace.

  • Owner - The name and email address of the workspace owner.

  • Filtering the workspace list

    Sorting the workspace list

    Workspace details on Workspaces view

    Getting access to workspace tools and actions

    Displaying the workspace management view

    Options column

    Actions menu for bulk actions

    Child workspaces
    workspace management view
    workspace management view
    Viewing the current subsetting configuration
    Post-job scripts
    Webhooks
    Workspace tools menu
    Options column and dropdown for a workspace
    Actions menu for selected workspaces
    Viewing and resolving schema changes
    Provides an overview of generator presets. Includes how to create and update them, and how to track where each generator preset is used. For more information, go to Managing generator presets.

    Provides an overview of the file connector and how to manage file groups in a file connector workspace. For more information, go to File connector.

    Provides an overview of the consistency generator property and how it works. For more information, go to Enabling consistency.

    Provides an overview of how to enable Document View for a JSON column and how to use it to configure generators for JSON fields.

    Provides an overview of subsetting, how it is configured, and how Structural uses the configuration to generate a subset. For more information, go to Subsetting data.

    Provides an overview of upsert data generation. Includes how it works and how to enable and run it for a workspace. For more information, go to Enabling and configuring upsert.

    Provides an overview of how to write destination data to a container repository instead of a database server. For more information, go to Writing output to a container repository.

    Tonic Structural 101

    Creating a Structural workspace

    Sensitivity detection and generator recommendations

    Managing workspace access

    Structural generators overview

    Generator presets

    Structural playlist
    Structural data generation workflow
    Creating and managing workspaces
    Managing access to workspaces
    Generator information

    File connector overview

    Generating data with consistency

    Using Document View to configure JSON columns

    Subsetting your data

    Upsert data generation

    Writing destination data to a container repository

    Working with document-based data

    For document-based data connectors - currently MongoDB - Database View and Table View are replaced by Collection View. "Collection" is the term that Structural uses to refer to MongoDB collections.

    For JSON columns in file connector and PostgreSQL workspaces, you can use Document View to view and assign generators to JSON fields.

    What can you ask the Structural Agent to do?

    You can ask the Structural Agent to perform a variety of analysis and configuration tasks. The following are just a few examples.

    Task
    Example prompts

    Commenting on columns

    From Database View, you can add comments to columns. For example, you might use a comment to explain why you selected a particular generator or marked a column as sensitive or not sensitive.

    If a column does not have any comments, then to add a comment:

    1. In the Applied Generator column, click the comment icon.

    2. In the comment field, type the comment text.

    Scan collections Collection scans identify the fields and data types in a collection.

    Use Collection View Configure generators and collection modes.

    Use Document View for JSON columns Assign generators to fields in JSON columns.

    Assign generators to path expressions In Collection View or Document View, assign a generator to fields that match a JSONPath expression.

    Display Database View. Show me Table View for the transactions table.

    Ask questions about the workspace data

    How many name columns were detected in the data?

    Are there any sensitive columns in the customers table? Are there any JSON columns?

    Filter the workspace data columns

    Show me all of the unprotected datetime columns.

    Show me all of the primary key and foreign key columns.

    Set column sensitivity

    Make sure that all datetime columns are marked as sensitive. Mark the currently displayed columns as not sensitive.

    Ask for generator recommendations

    Suggest a generator for the occupation column.

    Configure column generators

    Apply the Timestamp Shift generator to all of the datetime columns. Shift the dates by 1 day before or after the current value. Apply the Product List custom generator preset to the product column. Apply the Conditional generator to the seniority column. If the value of the column is less than 4, use the Random Integer generator to generate a replacement value between 0 and 4. If the value of the column is greater than or equal to 5, use the Random Integer generator to generate a value between 5 and 10. Apply the JSON Mask generator to the customer_summary column so that properties under name have the Name generator applied with the correct name type, the location properties have the Address generator applied with the correct location type, and the name property under work/properties has the Business Name generator applied. Apply the Regex Mask generator to the Product ID field to use the Random Integer generator to mask the numeric parts of the value.

    Display sample data

    Show me sample source and destination values for the last name column in the customers table.

    Manage document-based data

    Enable Document View for the customer_summary column.

    How many name fields are there across the database collections?

    Mark the occupation field as not sensitive. Apply the recommended generators to the fields under customer/name. Apply the Timestamp Shift generator to all fields that contain datetime values.

    Manage subsetting

    Is subsetting enabled? How many tables are in the subset? How large will the subset be based on the current configuration? What is the subset target table configuration? Should any tables be lookup tables? Enable subsetting and configure the transactions table as the target table. Use 5% of the records. Change the target table configuration to use records where the date is within the past 3 months. For the attendees table, filter optional records to include records where organization is test. Make vendor_type a lookup table. Enable subsetting and include tables that are not in the subset.

    Create and configure post-job scripts

    Create a post-job script to add demo users to the users table in the destination database.

    Enable the add demo users post-job script.

    Change the order of the post-job scripts to make the add demo users script run first.

    Delete the add demo users script.

    Identify and resolve schema changes

    Are there any schema changes that need to be resolved? Resolve the sensitive schema changes. Apply the Categorical generator to the new column. Dismiss the schema change notifications.

    Get information about jobs and diagnose job failures

    How many data generation jobs have run for this workspace? How many columns were de-identified during the most recent successful data generation? When was the most recent sensitivity scan for this workspace? How many sensitive columns did it identify? Why did the most recent data generation job fail? What's the solution to enable the job to complete successfully?

    Compose a draft email message to Tonic.ai support. The Agent creates the draft message and then displays an Email support button that opens the draft message in your email client.

    Can you create a draft email message to support to report the issue with access to workspaces?

    Generate scripts and API calls Note that the Agent can only use the information available from . It does not have access to the generated API reference.

    Can you show me the API call for assigning the Email generator to the email address columns?

    Note that Structural Agent requests are still subject to your global and workspace permissions. You cannot ask the Structural Agent to perform a task that you do not have permission to do.

    Explore workspaces

    Which workspace did I update most recently? Which workspace did I most recently generate data for? What's the most recently failed job in my workspaces? Give me a breakdown of the number of workspaces by data connector. How many workspaces don't yet have a completed data generation job? Are all of the name columns in my workspaces protected?

    Navigate through the workspace

    Click Comment.

    When a column has existing comments, the comment icon is green. To add comments:

    Replying to existing comments on a column
    1. Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user.

    2. In the comment field, type the comment text.

    3. Click Reply.

    Required license: Professional or Enterprise

    Creating a new comment

    Comment panel for a column that has no comments

    Responding to existing comments

    Structural deployment types

    Self-hosted Tonic Structural instance

    You can deploy a self-hosted, on-premises instance of Tonic Structural.

    For a self-hosted instance, Structural provides administrator tools that allow you to monitor Structural services and .

    You can configure Structural environment settings to customize your instance.

    On a self-hosted instance, based on your license plan, you have access to the full set of supported data connectors.

    Structural Cloud

    Structural Cloud is our secure hosted environment. On Structural Cloud, Tonic.ai handles monitoring Structural services and updating Structural.

    For single sign-on (SSO), Structural Cloud only supports Okta.

    Structural Cloud does not include:

    • , except for specific settings that can be set for a Cloud organization from the Organization Settings tab on Structural Settings. Other than that, Structural Cloud uses a single configuration.

    • Access to the following data connectors:

    Each Structural Cloud user belongs to a Structural Cloud organization, which is determined either by the user's email domain or by a workspace invitation. Structural Cloud users do not have any access to workspaces or users from other organizations.

    Each free trial user is in a separate organization, along with any users that they invite to have access to a free trial workspace.

    For information about Structural Cloud organizations, go to .

    The Account Admin permission set allows a Structural Cloud user to manage organization users and workspaces. For information about granting access to the Account Admin permission set, go to .

    Using Document View for JSON columns

    For columns that contain JSON content, you can use the JSON Mask generator to assign generators to individual JSON fields. To identify the fields, you use JSONPath expressions.

    Another option is to use Document View, which allows you to view the structure of the JSON content and then assign generators to individual JSON fields.

    You can also view this .

    For a JSON column, the Document View option is available from Privacy Hub, Database View, and Table View.

    On the column configuration panel, to enable Document View, toggle Use Document View to the on position.

    You can also use the Structural Agent to enable Document View. For example, Enable document view for the customer_summary column.

    manage Structural users
    Custom permission sets
    Environment setting configuration
    Structural data encryption
    Amazon Redshift
    Db2 for LUW
    Spark SDK
    Structural organizations
    Granting Account Admin access for a Structural Cloud organization
    this guide
    When you enable Document View:
    • The generator dropdown changes to an Open in Document View button.

    • If this is the first column that you enabled Document View for, then the Document View tab becomes visible.

    • Any existing generator assignment is discarded.

    • On Privacy Hub, in the protection status display, each JSON path is displayed as a separate column. In the Database Tables list, each JSON path is a separate entry.

    Structural also runs a scan on the column to detect the JSON structure and identify sensitive fields.

    On workspace management view, you use Document View to view the JSON structure.

    Document View is only available when it is enabled for at least one JSON column.

    From the Column dropdown list, select the JSON column to configure. The dropdown contains the columns that have Document View enabled.

    Column dropdown list on Document View

    From the View dropdown list, select the view to use for the selected column.

    View dropdown on Document View

    Hybrid view provides a consolidated view of the schema across all of the rows.

    Hybrid view of Document view

    For example, for an array, hybrid view contains a single entry with all of the possible fields.

    Single view shows the structure for one row at a time. You can then page through up to 100 rows. For each row, Structural displays the row structure.

    Single view of Document view

    For example, for an array, single view shows the actual array entries for each record.

    For each JSON field, Document View always displays:

    • The field name and data type.

    • The assigned generator.

    • An example value. In hybrid view, you can use the magnifying glass icon to display additional example values.

    Hybrid view also displays a Field Freq column. Field Freq shows the percentage of rows that contain that permutation of field and type. For example, a field might be Null 33% of the time and contain a numeric value 67% of the time. Or a field value might be an Int32 value 3% of the time and an Int64 value 6% of the time. The percentages apply to the first 100 rows.

    The Preview toggle at the top right of Document View allows you to choose whether to display original source data or the transformed data. You can switch back and forth to determine exactly how Tonic Structural transforms the data based on the field configuration.

    By default, the Preview toggle is in the on position, and the displayed data reflects the assigned generators.

    To display the original source data, toggle Preview to the off position.

    In single view, you can filter by either a field name or a field value.

    In hybrid view, you can filter by either field name or field properties.

    You can filter single view to only display fields that have specific text in either the field name or the field value.

    To filter by value, toggle Search by Value to the on position.

    After you select the filter type, in the search field, type text that is in the field name or value. As you type, Structural filters the list to only include fields that contain the filter text.

    Searching single view by field value

    To filter hybrid view by field name, in the search field, begin typing text that is in the field name. As you type, Structural filters the list to only include fields with names that include the filter text.

    Searching hybrid view by field name

    From the hybrid document view, you can filter the fields based on field properties.

    To display the Filters panel, click Filters.

    Filters panel on Document View

    To search for a filter or a filter value, in the search field, start to type the value. The search looks for text in the individual settings.

    Filter search for Document View

    To add a filter, depending on the filter type, either check the checkbox or select a filter option. As you add filters, Structural applies them to the field list.

    Above the list, Structural displays tags for the selected filters.

    Document View with applied filters

    To clear all of the currently selected filters, click Clear All.

    The Filters panel in hybrid view includes the following options.

    An at-risk JSON field:

    • Is marked as sensitive

    • Is assigned the Passthrough generator.

    To only display at-risk JSON fields, on the Filters panel, check At-Risk Field.

    When you check At-Risk Field, Structural adds the following filters under Privacy Settings:

    • Sets the sensitivity filter to Sensitive.

    • Sets the protection status filter to Not protected.

    You can filter the JSON fields based on the field sensitivity.

    On the Filters panel, under Privacy Settings, the sensitivity filter is by default set to All, which indicates to display both sensitive and non-sensitive JSON fields.

    • To only display sensitive JSON fields, click Sensitive.

    • To only display non-sensitive JSON fields, click Not sensitive.

    Note that when you check At-risk Field, Structural automatically selects Sensitive.

    You can filter the JSON fields based on whether they have any generator other than Passthrough assigned.

    On the Filters panel, under Privacy Settings, the field protection filter is by default set to All, which indicates to display both protected and not protected JSON fields.

    • To only display JSON fields that have an assigned generator, click Protected.

    • To only display JSON fields that do not have an assigned generator, click Not protected.

    Note that when you check At-Risk Field, Structural automatically selects Not protected.

    When Structural detects that a JSON field is sensitive, it can also determine a recommended generator.

    For example, when it detects a name value, it also recommends the Name generator.

    You can filter the fields to display the fields that have recommended generators.

    On the Filters panel, under Recommended Generators, check the checkbox next to the recommended generator for which to display the fields that have that recommendation.

    You can filter the fields by the field data type. For example, you might only display columns that contain either numeric or integer values.

    To only display fields that have specific data types, on the Filters panel, under Database Data Types, check the checkbox for each data type to include.

    The list of data types only includes data types that are present in the currently displayed fields and that are compatible with other applied filters.

    To search for a specific data type, in the Filters search field, begin to type the data type.

    When the structure of the JSON changes, you might need to update the configuration to reflect those changes. If you do not resolve the changes, then the data generation might fail.

    To only display fields that have unresolved changes to the JSON structure, on the Filters panel, check Unresolved Schema Changes.

    For detected sensitive fields, the sensitivity type indicates the type of data that was detected. Examples of sensitivity types include First Name, Address, and Email.

    To only display fields that contain specific sensitivity types, on the Filters panel, under Sensitivity Type, check the checkbox for each sensitivity type to include.

    The list of sensitivity types only includes sensitivity types that are present in the currently displayed fields.

    To search for a specific sensitivity type, in the Filters search field, type the sensitivity type.

    When the document scan identifies a value as belonging to a sensitivity type, it also determines how confident it is in that determination.

    You can filter the columns based on the confidence level.

    To only display columns that have a specific confidence level, on the Filters panel, under Sensitivity confidence, check the checkbox next to each confidence level to include.

    You can add comments to fields. For example, you might use a comment to explain why you selected a particular generator or marked a field as sensitive or not sensitive.

    Note that you can only add comments from Hybrid view. You cannot add comments from Single view.

    If a field does not have any comments, then to add a comment:

    1. Click the comment icon.

    2. In the comment field, type the comment text.

    3. Click Comment.

    When a field has existing comments, the comment icon is green. To add comments:

    1. Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user and timestamp.

    2. In the comment field, type the comment text.

    3. Click Reply.

    On the field configuration panel, the sensitivity toggle at the top right indicates whether the field is marked as sensitive.

    Field configuration panel on Document View

    To mark a field as sensitive, toggle the setting to the Sensitive position.

    To mark a field as not sensitive, toggle the setting to the Not Sensitive position.

    You can also use the Structural Agent to set sensitivity. For example, Mark the occupation field as not sensitive.

    For each node, you assign a generator.

    To assign a generator:

    1. Click the generator value for the JSON field.

    2. On the configuration panel, from the Generator dropdown list, select the generator. Other than the Conditional and Regex Mask generators, you cannot assign a composite generator to a JSON field.

    3. Configure the generator options. For details about the available configuration options for each generator, go to the generator reference.

    When you configure a generator in Document View:

    • You can only link to other JSON fields.

    • You can only enable self-consistency.

    You can also use the Structural Agent to apply generators to fields. For example:

    • Apply the recommended generators to all name fields.

    • Apply the Timestamp Shift generator to the renewal_date field.

    In addition to assigning generators to individual fields, you can assign generators to generic paths. The paths use JSONPath syntax.

    For more information, go to Assigning generators to path expressions.

    Only supported for the file connector and PostgreSQL.

    Note that for PostgreSQL, Document View cannot be used when the entire JSON document is an array. The JSON can contain arrays, but the document itself cannot be an array.

    Enabling Document View for a JSON column

    video overview of Document View

    Displaying Document View

    Selecting the JSON column to configure

    Selecting the type of view

    Hybrid view

    Single view

    Information in the field list

    Toggling between source and preview data

    Required workspace permission:

    • Source data: Preview source data

    • Destination data: Preview destination data

    Filtering Document View fields

    Filtering single document view by field name or value

    Filtering hybrid view by field name

    Filtering hybrid view by field properties

    Searching for a filter

    Adding a filter

    Clearing the selected filters

    Filters panel filters

    At-risk JSON fields

    Sensitivity

    Protection status

    Recommended generators

    Field data type

    Unresolved schema changes

    Sensitivity type

    Sensitivity confidence

    Commenting on fields

    Adding a new comment

    Replying to an existing comment

    Indicating whether a JSON field is sensitive

    Required workspace permission: Configure column sensitivity

    Assigning a generator to a JSON field

    Required workspace permission: Configure column generators

    Assigning generators to fields that match JSONPath expressions

    Assigning generators to path expressions

    On Collection View and Document View, Structural can automatically assign generators to fields that match a configured path expression. Each collection or JSON column has its own set of path generators.

    Structural always applies the path generator to matching fields that do not have an assigned generator (are set to Passthrough).

    Structural does not apply the path generator to matching fields that have a generator configuration applied directly.

    However, in a child workspace, Structural does apply the path generator to matching fields that inherit their current generator configuration from the parent workspace.

    Displaying the list of path generators

    To display the list of path generators for the current collection or JSON column, click Path Generators.

    For each path generator, the list includes:

    Path generators list
    • The priority order. Structural checks the fields against the paths in the order that the paths are displayed. The first matching path wins.

    • The path expression to identify matching fields.

    • The data type filter for matching fields. You can configure a path generator to only apply to fields of a specific type or types.

    • The name of the generator preset that Structural applies to matching fields.

    To create a path generator, you can either create a completely new path generator, or start from a duplicate of an existing path generator.

    Structural saves the new generator automatically when the configuration is complete. The Saved button at the bottom right indicates when the generator is saved.

    To create a path generator:

    1. On the path generators panel, click Add path generator.

    2. On the path generator details panel, in the Path Expression field, provide the path expression to use to identify matching paths. The path expression uses the syntax. Note that for a path generator, you cannot use the expression to check for a field value. For more information about the supported operators and some examples, go to . When you provide a path expression, the matching fields list displays the fields that match the expression.

    3. You can optionally filter the matching fields based on the data type. For example, you might only want to apply a generator to text or integer fields. By default, the data type filter list is empty. The available data types are general types that map to specific data types in a given database. Under Data Types, to add a data type to the filter, select it from the dropdown list. To remove a data type, click its delete icon. When you configure data type filters, the matching fields list is updated to only include fields that have one of the specified data types.

    You can create a path generator based on an existing one. For example, for the same path expression, you might want to assign a different generator based on the data type.

    To create a new path generator based on an existing path generator:

    1. On the path generators list, click the options menu for the path generator to copy.

    1. Click Duplicate path generator.

    2. On the path generator details panel, edit the configuration.

    To update a path generator:

    1. On the path generators list, click the options menu for the path generator.

    2. Click Edit path generator.

    3. On the path generator details panel, edit the configuration.

    Structural saves the changes automatically.

    For fields that were assigned a generator based on the previous configuration, but that do not match the updated path generator configuration:

    • If the field matches other path generators, then the next matching configuration is applied.

    • If the field does not match any other path generators, then the field reverts to Passthrough.

    When you delete a path generator, the generator assignment is removed from the matching fields. If a field matched more than one path generator, then the next match is used.

    To delete a path configuration:

    1. On the path generators list, click the options menu for the path configuration.

    2. Click Delete path generator.

    For fields that were assigned a generator based on the path generator:

    • If the field matches other path generators, then the next matching path generator is applied.

    • If the field does not match any other path generators, then the field reverts to Passthrough.

    For a path generator path expression, Structural supports the following operators:

    • $ - Root

    • . - Child operator

    • .. - Recursive descent operator

    For example, a document includes an array of objects. Each object contains name, address, and email address fields.

    You can configure a path generator that assigns a generator to the address field in all of the array objects. You cannot only assign a generator to the address field in one of the array objects.

    Here are some example path expressions, based on the following JSON:

    When Structural looks for matching fields, it checks the path generators in the order that they are displayed on the Path Generators panel.

    For each field, it uses the first matching configuration.

    To change the order of the path generators, drag and drop each configuration to the appropriate location in the list.

    On Collection View and Document View, when the assigned generator comes from a path generator, the generator assignment is marked with an icon.

    When you click the generator, the configuration panel indicates that the generator is assigned based on a path generator.

    When you change the configuration for the field, the icon is removed. The override tooltip indicates that the path generator was overridden.

    If you set the generator to Passthrough, then the field reverts to the path generator.

    Sharing workspace access

    Required permission

    • Team permission: View team users. This permission is only required for the Tonic Structural application. It is not needed when you use the Structural API.

    • Either:

    About workspace access

    Tonic Structural uses workspace permission sets for role-based access (RBAC) of each workspace.

    A workspace permission set is a set of workspace permissions. Each permission provides access to a specific workspace feature or function.

    Structural provides built-in workspace permission sets. Enterprise instances can also configure custom workspace permission sets.

    To share workspace access, you assign workspace permission sets to users and, if you use SSO to manage Structural users, to SSO groups.

    Before you assign a workspace permission set to an SSO group, make sure that you are aware of who is in the group. The permissions that are granted to an SSO group automatically are granted to all of the users in the group. For information on how to configure Structural to filter the allowed SSO groups, go to Synchronizing SSO groups with Structural.

    Limitations on workspace sharing

    Cannot remove the owner permission set from the owner

    You cannot remove the owner workspace permission set from the workspace owner. By default, the owner permission set is the built-in Manager permission set.

    Cannot grant or remove a permission you do not have

    Within a workspace, the Share workspace access permission grants you the ability to share access to the workspace.

    However, you cannot either grant or revoke access to a workspace permission that you do not have.

    For example, for a given workspace, the workspace permission set that is assigned to you includes Share workspace access, but does not include Run data generation. Because of this, when you share workspace access:

    • You cannot grant access to a workspace permission set that includes Run data generation.

    • You cannot remove access to a workspace permission set that includes Run data generation.

    Note that this requirement does not apply to users who have the team permission Manage user access within the team, which is by default granted to Team Admin users. Those users can grant or revoke any workspace permission set.

    To change the current access to the workspace:

    1. To manage access to a single workspace, either:

      • On the workspace management view, in the heading, click the share icon.

      • On Workspaces view, click the actions menu for the workspace, then select Share.

    Table modes

    Each table is assigned a table mode. The table mode determines at a high level how the table is populated in the destination database.

    Both Database View and Table View allow you to view and update the selected table mode for a table.

    For Database View, go to .

    For Table View, go to .

    This is the default table mode for new tables.

    In this mode, Tonic Structural copies over all of the rows to the destination database.

    For columns that have the generator set to Passthrough, Structural copies the original source data to the destination database.

    Connection type

    When you create a workspace, under Workspace Details, from the Connection Type dropdown list, select the type of data connector to use for the workspace data.

    You cannot change the connection type on a .

    The Professional license limits the number and type of data connectors you can use. A Professional instance can use up two different data connector types, which can be any type other than Oracle or Db2 for LUW. After you create workspaces that use two different data connector types, any subsequent workspaces must use one of those data connector types.

    If the database that you want to connect to isn't in the list, or you want to have different database types for your source and destination database, contact support@tonic.ai.

    When you select a connector type, Structural updates the view to display the connection fields used for that connector type. The specific fields vary based on the .

    After you save a new workspace, you cannot change the workspace connection type.

    To manage access for multiple workspaces:
    1. Check the checkbox for each workspace to grant access to.

    2. From the Actions menu, select Share Workspaces.

  • The workspace access panel contains the current list of users and groups that have access to the workspace. To add a user or group to the list of users and groups, begin to type the user email address or group name. From the list of matching users or groups, select the user or group to add. Free trial users can invite other users to start their own free trial. Provide the email addresses of the users to invite. The email addresses must have the same corporate email domain as your email address. When the invited users sign up for the free trial, they are added to the Structural organization for the free trial user that invited them and have access to the workspace.

  • For a user or group, to change the assigned workspace permission sets:

    1. Click Access. The dropdown list is populated with the list of custom and built-in workspace permission sets. If you selected multiple workspaces, then on the initial display of the workspace sharing panel, for each permission set that a user or group currently has access to, the list shows the number of workspaces for which the user or group has that permission set. For example, you select three workspaces. A user currently has Editor access for one workspace and Viewer access for the other two. The Editor permission set has 1 next to it, and the Viewer permission set has 2 next to it.

    2. Under Custom Permission Sets, check the checkbox next to each workspace permission set to assign to the user or group. Uncheck the checkbox next to each workspace permission set to remove from the user or group.

    3. Under Built-In Permission Sets, check the workspace permission set to assign to the user or group. You can only assign one built-in permission set. By default, for an added user or group, the Editor permission set is selected. To select a built-in workspace permission set that is lower in access than the currently selected permission set, you must first uncheck the selected permission set. For example, if Editor is currently checked, then to change the selection to Viewer, you must first uncheck Editor.

  • To remove all access for a user or group, and remove the user or group from the list, click Access, then click Revoke.

  • To save the new access, click Save.

  • Workspace permission: Share workspace access

  • Team permission: Manage user access within the team

  • Changing the workspace access

    Workspace access panel
    child workspace
    connector type
    Connection Type dropdown list for a new workspace
    Locked connection type value on a saved workspace
  • From the generator dropdown list, select the generator to apply to matching fields. The available generators are affected by the data type filter. When the data type filter is empty, you can only select from generators that can be used for any type of column. When you specify a list of data types, you can only select from generators that can be used for all of those data types.

  • Configure the selected generator. For a generator assigned to a path expression:

    • Linking is not supported.

    • Consistency with other columns is not supported.

  • * - Wildcard operator
  • [*] - Array operator. Note that a path generator must always target all of the items in an array. Any use of the array operator must include the wildcard operator.

  • $.bookstore_name

    Find the bookstore_name field at the top level of the JSON.

    $.mailing_address.zip_code

    Find the zip_code field in the mailing address.

    $.books[*].isbn

    Find the isbn field in each entry in the array of books.

    $..country

    Creating a path generator

    Creating a completely new path generator

    Copying an existing path generator

    Updating a path generator

    Deleting a path generator

    Supported JSONPath operators and examples

    Determining the priority order for the path generators

    Identifying matching fields on Collection View and Document View

    Overriding the path generator assignment

    JSONPath
    Options menu for a path generator
    Field with a generator assigned by a path generator
    Configuration panel for a field that matches a path generator
    Field where the path generator is overridden by a different generator configuration
    Supported JSONPath operators and examples

    Find all country fields in the JSON.

    {
      [
        {
          "name": "John Smith",
          "address": "1 Main Street",
          "email_address": "jsmith@example.com"
        },
        {
          "name": "Mary Jones",
          "address": "5 Elm Avenue",
          "email_address": "mjones@example.com"
        },
      ]
    }
    {
      "bookstore_name": "Read & Brew Books",
      "mailing_address": {
        "street": "123 Literary Lane",
        "city": "Bookville",
        "state": "MA",
        "zip_code": "02451",
        "country": "USA"
      },
      "phone_number": "555-123-4567",
      "books": [
        {
          "title": "The Great Gatsby",
          "author": "F. Scott Fitzgerald",
          "isbn": "978-0743273565",
          "publication_year": 1925,
          "country": "USA"
        },
        {
          "title": "Moby Dick",
          "author": "Herman Melville",
          "isbn": "978-1503280786",
          "publication_year": 1851,
          "country": "USA"
        },
        {
          "title": "To the Lighthouse",
          "author": "Virginia Woolf",
          "isbn": "978-0156907392",
          "publication_year": 1927,
          "country": "UK"
        },
        {
          "title": "The Catcher in the Rye",
          "author": "J.D. Salinger",
          "isbn": "978-0316769174",
          "publication_year": 1951,
          "country": "USA"
        }
      ]
    }
    For columns that are assigned a generator other than Passthrough, Structural uses the generator to replace the column data in the destination database.

    This mode drops all data for the table in the destination database. Sensitivity scans ignore truncated tables.

    For data connectors other than Spark-based data connectors, the table schema and any constraints associated with the table are included in the destination database.

    For Spark-based data connectors (Databricks, Spark SDK), the table is ignored completely.

    For the file connector, file groups are treated as tables. When a file group is assigned Truncate mode, the data generation process ignores the files that are in that file group.

    Any existing data in the destination database is removed. For example, if you change the table mode to Truncate after an initial data generation, the next data generation clears the table data. For Spark-based data connectors, the table is removed.

    If you assign Truncate mode to a table that has a foreign key constraint, it fails during data generation. If this is a requirement, contact support@tonic.ai for assistance.

    When upsert is enabled, the Truncate table mode does not actually truncate the destination table. Instead, it works more like Preserve Destination table mode, which preserves existing records in the destination table.

    This mode preserves the data in the destination database for this table. It does not add or update any records.

    This feature is primarily used for very large tables that don't need to be de-identified during subsequent runs after the data exists in the destination database.

    When you assign Preserve Destination mode to a table, Structural locks the generator configuration for the table columns.

    The destination database must have the same schema as the source database.

    You cannot use Preserve Destination mode when you:

    • Enable upsert for a workspace.

    • Write destination data to a container repository.

    Incremental mode only processes the changes that occurred to the source table since the most recent data generation or other changes in the destination. This can greatly reduce generation time for large tables that don't have a lot of changes.

    For Incremental mode to work, the following conditions must be satisfied:

    • The table must exist in the destination database. Either Structural created the table during data generation, or the table was created and populated in some other way.

    • A reliable date updated column must be present. When you select Incremental mode for a table, Structural prompts you to select the date updated column to use.

    • The table must have a primary key.

    To maximize performance, we recommend that you have an index on the date updated field.

    For tables that use Incremental mode, Structural checks the source database for records that have an updated date that that is greater than the maximum date in that column in the destination database.

    When identifying records to update, Structural only checks the updated date. It does not check for other updates. Records where the generator configuration is changed are not updated if they do not meet the updated date requirement.

    For the identified records, Structural checks for primary key matches between the source and destination databases, then does one of the following:

    • If the primary key value exists in the destination database, then Structural overwrites the record in the destination database.

    • If the primary key value does not exist in the destination database, then Structural adds a new record to the destination database.

    This mode currently only updates and adds records. Rows that are deleted from the source database remain in the destination database.

    To ensure accurate incremental processing of records, we recommend that you do not directly modify the destination database. A direct modification might cause the maximum updated date in the destination database to be after the date of the last data generation. This could prevent records from being identified for incremental processing.

    Incremental mode is currently supported on PostgreSQL, MySQL, and SQL Server. If you want to use this table mode with another database type, contact support@tonic.ai.

    You cannot use Incremental mode when you:

    • Enable upsert for a workspace.

    • Write destination data to a container repository.

    For the Databricks data connector, the table mode configuration includes an Error on Overwrite setting. The setting indicates whether to return an error when Structural attempts to write data to a destination table that already contains data. The option is not available when you write destination data to Databricks Delta tables.

    To return the error, toggle the setting to the on position.

    To not return the error, toggle the setting to the off position.

    For workspaces that use following data connectors, the table mode configuration for De-Identify mode includes an option to apply a filter to the table:

    • Amazon Redshift

    • Databricks

    • Google BigQuery

    • Snowflake

    Table filters provide a way to generate a smaller set of data when a data connector does not support subsetting. For more information, go to Using table filtering for data warehouses and Spark-based data connectors.

    This option is only available for workspaces that use the following data connectors:

    • Databricks

    On the table mode configuration panel, you can use the Repartition or Coalesce option to indicate a number of partitions to generate.

    Table mode configuration panel for a Spark-based workspace

    By default, the destination database uses the same partitioning as the source database. The partition option is set to Neither.

    The Repartition option allows you to provide a specific number of partitions to generate.

    To use the Repartition option:

    1. Click Repartition.

    2. In the field, enter the number of partitions.

    The Coalesce option allows you to provide a maximum number of partitions to generate. If the source data has fewer partitions than the number you specify, then Structural only generates that number.

    The Coalesce option should be more efficient than the Repartition option.

    To use the Coalesce option:

    1. Click Coalesce.

    2. In the field, enter the number of partitions.

    Selecting the table mode for a table

    Required workspace permission: Assign table modes

    Available table modes

    De-Identify

    Assigning table modes to tables

    Truncate

    Preserve Destination

    Incremental

    Indicating whether to return an error when destination data already exists (Databricks only)

    Applying a filter to tables

    Configuring partitioning for the destination database

    Using the Repartition option

    Using the Coalesce option

    Generator reference

    This generator reference provides the details for each of the the supported generators in Tonic Structural.

    Information provided for each generator

    For each generator, the reference provides:

    • Overview description

    • A table that contains:

    The generators are in alphabetical order by the generator name.

    Here are some groupings to help to identify generators that are used for different types of values. Generator hints and tips also provides some suggestions for generators to use for specific uses cases.

    Composite generators

    Transform data that uses complex formats or based on a condition. For more information, go to .

    Information type generators

    These generators produce specific types of values.

    • (and the deprecated )

    Datetime value generators

    These generators are used to specifically transform datetime values.

    Key generators

    Intended for use with primary key columns. For more information, go to .

    Numeric value generators

    These generators are specifically intended to work with numeric values.

    String value generators

    These generators are useful for transforming string values that aren't covered by a specific information type generator.

    Other value substitution and replacement generators

    These generators perform other types of transformation on column values.

    Generator information

    Generators transform the data in a source database column. You assign the generators to use. Tonic Structural offers a variety of generators to transform different types of data.

    For details about how to assign and configure generators, and manage generator presets, go to .

    You can also view this .

    About the available generators

    Generator characteristics and types

    Generator assignment and configuration
    video overview of generators and how they work

    Generator summary

    Summary list of generators.

    Generator reference

    Details about the characteristics and configuration options for each generator.

    Generator API reference

    Details about the structure of each generator assignment in the API.

    Generator characteristics

    Common generator characteristics to be aware of, such as consistency and linking.

    Composite generators

    Composite generators apply a generator to a specific data element or based on a condition.

    Primary key generators

    Learn about generators that you can apply to primary key columns.

    Generator characteristics that you might want to take into account when you select the generator.

  • The generator privacy ranking, which indicates the level of protection that the generator provides.

  • The generator ID to use in the Structural API. The generator ID is linked to the API details for the generator.

  • Instructions for how to configure the generator

  • The generator characteristics include:

    • Consistency - Whether you configure the generator to base the the destination values on the source values.

    • Linking - Whether you can link columns that use the generator to indicate that there is a relationship between them.

    • Differential privacy - Whether the generator supports differential privacy, which ensures that the source value cannot be reverse engineered from the output value.

    • Data-free - Whether the generator is data-free, meaning that the output data is completely unrelated to the source data.

    • - Whether you can assign the generator to primary key columns.

    • - Whether you can assign the generator to columns that require unique values.

    • - Whether the generator uses FPE to encrypt the values.

    Conditional

  • CSV Mask

  • HStore Mask

  • HTML Mask

  • JSON Mask

  • Regex Mask

  • Struct Mask

  • XML Mask

  • Email

  • File Name

  • Finnish Personal Identity Code

  • FNR

  • Geo

  • HIPAA Address

  • Hostname

  • IBAN

  • International Address

  • IP Address

  • MAC Address

  • Name

  • Phone

  • Shipping Container

  • SIN

  • SSN

  • Unique Email

  • URL

  • Valid Address

  • Random Timestamp
  • Timestamp Shift Generator

  • Integer Key
  • Numeric String Key

  • UUID Key

  • Cross Table Sum
  • Noise Generator

  • Random Double

  • Random Integer

  • Sequential Integer

  • Character Substitution
  • Constant - Also useable for numeric columns.

  • Custom Categorical - Also useable for numeric columns.

  • Find and Replace

  • Regex Mask

  • Text Composition

  • Random Boolean

  • Random Hash

  • Random UUID

  • Composite generators
    Array JSON Mask
    Array Regex Mask
    Address
    Business Name
    Company Name
    Date Truncation
    Event Timestamps
    Primary key generators
    Alphanumeric String Key
    ASCII Key
    Algebraic
    Continuous
    Categorical
    Character Scramble
    Array Character Scramble
    Null

    Connecting through Tailscale

    is a managed virtual private network (VPN). It creates a private mesh network called a tailnet.

    Each device on the Tailscale network receives a stable private IP address, and can communicate directly with other devices on the tailnet, without the need to open ports to the public internet or to configure VPN gateways.

    Tonic Structural can join your corporate tailnet as a node. It can then reach database servers that are on the same tailnet, even if those servers are in a private network and have no public IP address.

    To allow Tonic Structural to reach a database through Tailscale, you:

    Managing your user account

    From the User Settings view, you can manage settings for your individual Tonic Structural account.

    To display the User Settings view:

    1. Click your user image at the top right.

    2. In the menu, click User Settings.

    The User Settings view includes options to:

    Managing access to workspaces

    When you create a workspace, you become the owner of the workspace, and by default are assigned the built-in Manager workspace permission set for the workspace. The Manager permission set provides full access to the workspace configuration, data, and results.

    You can also assign workspace permission sets to other users and to SSO groups in your team. You can also transfer a workspace to a different owner.

    If you are granted access to any workspace permission set for a workspace, then you have access to all of the workspace management views for that workspace. However, you can only perform tasks that you have permission for in that workspace.

    Workspace access is managed from the Workspaces view. You cannot assign workspace permission sets from Structural Settings view.

    You can also view an .

    Allowed for primary keys
    Allowed for unique columns
    Uses format-preserving encryption (FPE)
    - Install the Tailscale agent on the server, or configure a subnet router.
  • Declare an access control list (ACL) tag - Register the tag in the tailnet policy that Structural uses to identify itself.

  • Create an OAuth credential - Generate the client secret that Structural uses to authenticate to your tailnet.

  • Configure the Structural workspace - Enable the Tailscale tunnel and provide the connection details.

  • Connections from Structural arrive at the server's Tailscale IP address, not the server's actual IP address.

    You must configure your database server to allow connections from your tailnet's IP address range.

    By default, the range is 100.64.0.0/10, although it can be customized.

    For a PostgreSQL database server, you add an entry for the subnet in pg_hba.conf.

    To make your database server reachable through Tailscale, you can either:

    • Install Tailscale directly on the database server.

    • Use a subnet router.

    If you have shell access to the database server, then this is the simplest approach.

    1. Install Tailscale on the server. For platform-specific instructions, go to tailscale.com/download.

    2. If a browser is available:

      1. Run sudo tailscale up. This returns a link that looks like:

        https://login.tailscale.com/a/0123456789abcdef
      2. Open the URL in a browser. Tailscale prompts for authentication and then adds the server to your tailnet.

    3. For headless servers where no browser is available, use the --auth-key flag instead:

      1. To generate an auth key, go to https://login.tailscale.com/admin/settings/keys .

      2. Click Generate auth key, then indicate whether the key is:

    4. Retrieve the server's Tailscale IP address:

    5. Verify that the device appears at https://login.tailscale.com/admin/machines.

    The server now has a stable Tailscale IP address, which is by default in the 100.x.x.x range.

    You use the Tailscale IP address as the value of Server in the workspace data connection settings.

    If you cannot install Tailscale on the database server itself, for example because it is a managed database service, you can instead:

    1. Install Tailscale on any other machine that is on the same private network as the database server.

    2. Configure the machine as a subnet router.

    A subnet router advertises a CIDR range to your tailnet. Other nodes such as Structural can then use regular IP addresses to reach the devices on that subnet. Tailscale does not need to be installed on those devices.

    To configure a subnet router:

    1. Install Tailscale on a machine that can reach the database server.

    2. Start Tailscale with the --advertise-routes flag, which specifies the subnet that your database is on:

      sudo tailscale up --advertise-routes=10.0.1.0/24
    3. Approve the advertised route:

      1. Go to https://login.tailscale.com/admin/machines.

      2. Click the machine.

      3. Click Edit route settings.

      4. Enable the route.

    When you use a subnet router, then in Structural, you set the value of Server to the database server's regular private IP address - for example, 10.0.1.50 - instead of a Tailscale IP address.

    Tailscale uses ACL tags to control the devices that a given credential can authenticate to.

    Before you create an OAuth credential, you must declare the tag that you intend to use in your tailnet's ACL policy.

    1. Go to https://login.tailscale.com/admin/acls.

    2. Edit the policy JSON.

    3. If the policy does not already contain a tagOwners section, then add the section.

    4. Add the tag to tagOwners. For example, to create a tag called tag:tonic:

    1. Save the policy.

    Structural authenticates to your tailnet using an OAuth client credential. You create this from the Tailscale admin console under Trust credentials.

    1. Go to https://login.tailscale.com/admin/settings/trust-credentials.

    2. Click + Credential.

    3. Select OAuth as the credential type.

    New Tailscale credential with OAuth selected as the credential type
    1. Add a description, then click Next.

    2. To set the credential scope:

      1. From the Scopes dropdown, select Custom scopes.

      2. Under Keys, for Auth Keys, check the Write checkbox.

      3. In the Tags field, provide the ACL tag that you created.

    3. Click Generate credential. Tailscale displays the Credential created panel with the client ID and the client secret.

    Credential created panel with the client ID and client secret for the Tailscale OAuth credential
    1. Copy the Client secret immediately. It is only shown once. The client secret starts with tskey-client-. In the Structural workspace configuration, you paste this value into the Tailscale Auth Key field. Structural does not use the client ID.

    After you complete the setup in Tailscale, you can configure a Structural workspace to connect through Tailscale.

    When you connect through Tailscale, in the database connection fields, to set the value of Server:

    • If you installed Tailscale on the database server, set Server to the Tailscale IP address, which is the stable address that is assigned to the device on your tailnet. By default, the address in the 100.x.x.x range.

    • If you use a subnet router, then in Structural, set Server to the database server's regular private IP address - for example, 10.0.1.50.

    To enable and configure the Tailscale connection:

    1. If the connector supports multiple tunneling options:

      1. Toggle Enable Connection Tunnel to the on position.

      2. From the Tunnel type dropdown list, select Tailscale Tunnel.

      If the data connector only supports Tailscale tunneling, then it is selected automatically.

    2. In the Tags field, provide a comma-separated list of ACL tags to use to join the tailnet. For example, tag:tonic-worker. Each tag name must exactly match the tag name that you configured in Tailscale, and must include the tag: prefix. If the tags don't match, then when Structural tries to connect, Tailscale returns an authentication error.

    3. Optionally, in the Control Server URL field, provide the URL. This is intended for customers who use Headscale or a self-hosted Tailscale control plane.

    4. In the Tailscale Auth Key field, you can either:

      • Provide the Tailscale OAuth client credential (value starting with tskey-client-) that you generated and copied from Tailscale. Structural does not support other types of Tailscale keys, such as user auth keys.

      • If secrets managers are available, .

    When Structural uses Tailscale tunneling to establish a connection:

    1. Structural uses the provided credential (OAuth credential or secrets manager secret) to authenticate as a new ephemeral node on your tailnet.

    2. The node is tagged with the specified ACL tags. The tags control what the node can reach based on your tailnet policy.

    3. Structural uses its Tailscale IP address to connect to the database server. Traffic flows through the encrypted tailnet mesh, not the public internet.

    4. When the connection is torn down, the ephemeral node is removed from your tailnet.

    Here are some errors that might occur during the connection:

    Error
    Likely cause

    Tailscale authentication failed

    Either:

    • The tag in the Tags field does not match the tag on the OAuth credential's auth_keys scope.

    • The tag is not declared in tagOwners in your ACL policy.

    Connection times out

    Either:

    • The database server is not on the tailnet.

    • The ACL policy does not permit the tagged node to reach it.

    Invalid auth key type

    Overview of the steps to enable the connection

    Tailscale
    Allow connections from the tailnet to your database server.
    "tagOwners": {
        "tag:tonic": ["your-tailscale-email@example.com"],
    },

    Allowing connections to the database server from the tailnet

    Adding your database server to the tailnet

    Installing Tailscale directly on the database server

    Using a subnet router

    Declaring an ACL tag

    Creating an OAuth credential

    Configuring a workspace to use Tailscale tunneling

    Setting the database connection server value

    Enabling and configuring the Tailscale connection

    How the connection works

    Troubleshooting the connection

    Add your database server to the tailnet
  • Configure your user image.

  • View and copy your organization identifier.

  • Configure email notifications for column comments.

  • Generate and manage API tokens.

  • (if your Structural instance does not use SSO).

  • You can select an image to associate with your account. The image is displayed next to your name and email address throughout Structural.

    If your instance uses Google or Azure single sign-on (SSO) to manage Structural users, then by default your Structural account image is the image from the SSO.

    Otherwise, the default image displays your initials.

    To change your user image, click Upload, then select the image file.

    Below your user image file name is the identifier of the organization that your account belongs to.

    To copy the identifier, click the copy icon.

    Structural allows users to provide comments on columns. You can do this from Privacy Hub and Database View.

    From the Comment Notification Settings section of User Settings, you can configure when to receive email notifications for comments.

    The available options are:

    • I am an owner, editor, auditor, or am being replied to This is the default option. You receive email notifications when comments are made on columns in a workspace that you are an owner, editor, or auditor for. You also receive an email notification when someone replies to a comment that you made.

    • I am @ mentioned You only receive an email notification if someone specifically mentions you in a comment.

    • Never You never receive email notifications for column comments.

    Before you can use the Structural API, you must create an API token. From the User API Tokens section of the User Settings view, you can create and manage API tokens.

    User API Tokens list on the User Settings view

    To create an API token:

    1. Click Create Token.

    2. On the Create New Token dialog, enter a name for the new token.

    3. Click Confirm.

    In the list, the new token displays as clear text. To copy the new token, click the copy icon next to the token.

    The new token text and copy icon only display during the current session. After that, Structural masks the token and removes the copy icon.

    To revoke a token, click the Revoke option for the token.

    If your Structural account is not managed using SSO, then from User Settings, you can change your Structural password.

    If your Structural instance uses SSO to manage users, then your user credentials are managed in the SSO system. You cannot change your user password in Structural.

    Under Password Change, to change your Structural password:

    1. In the Old Password field, type your current Structural password.

    2. In the New Password field, type your new Structural password.

    3. In the Repeat New Password field, type your new Structural password again.

    4. Click Confirm.

    From User Settings, you can delete your Structural account. If your instance uses SSO to manage users, then deleting your account only affects your access to Structural.

    You cannot delete your Structural account if you are the owner of a workspace for which other users are granted access. Before you can delete your Structural account, you must either:

    • Revoke access from other users.

    • Transfer ownership to a different user.

    To delete your Structural account, click Delete Account.

    When you delete your account, you are logged out of Structural.

    Choosing your user image

    Viewing and copying your organization identifier

    Configuring notifications for column comments

    Required license: Professional or Enterprise

    Generating and managing API tokens

    Creating an API token

    Revoking an API token

    Changing your Structural password

    Deleting your Structural account

    overview video tutorial about workspace access

    Share workspace access

    Grant access to other team users and groups. The assigned workspace permission sets determine the level of access.

    Managing the workspace schema cache

    You can configure a workspace to cache the source database schema, to reduce the number of times that Tonic Structural needs to query the source database.

    Viewing the schema cache status

    When schema caching is enabled for a workspace, then in the expanded version of the workspace management view header, below the workspace name, Structural shows the current status of the schema cache.

    Schema cache status for a workspace

    The status indicates when:

    • Structural is retrieving the schema information.

    • Structural is checking for schema updates.

    • Structural is refreshing the schema cache.

    • Structural fails to connect to the source database.

    • The schema cache refresh fails.

    • The schema cache is updated. The status includes the timestamp of the most recent refresh.

    When the schema cache is updated, or when Structural has detected updates to the schema, then the schema cache status includes an option to refresh the schema cache.

    To refresh the schema:

    1. Click the schema cache status.

    2. On the panel, click Refresh Schema.

    Structural starts a new schema retrieval job. To track the progress of the job, go to the .

    Data locations

    Source database connection

    For data connectors that connect to a database, the Source Settings section provides connection information for the source database.

    You cannot change the source data configuration for a child workspace.

    For information about the source connection fields for a specific data connector, go to the workspace configuration topic for that connector type.

    Upsert configuration

    For data connectors that support upsert, the workspace configuration includes an Upsert section to allow you to enable and configure upsert. Upsert adds and updates rows in the destination database, but keeps all other existing rows intact.

    If you enable upsert, then you cannot write output to a container repository. You must write the output to a destination database.

    For more information, go to Enabling and configuring upsert.

    Destination data location

    For data connectors that connect to a database, the Destination Settings section provides information about where and how Structural writes the output data from data generation.

    Depending on the data connector type, you might be able to write to either:

    • Destination database - Writes the output data to a destination database on a database server.

    • Container repository - Writes the output data to a data volume in a container repository.

    When you write the output to a destination database, the destination database must be of the same type as the source database.

    Structural does not create the destination database. It must exist before you generate data.

    In Destination Settings, you provide the connection information for the destination database. For information about the destination database connection fields for a specific data connector, go to the workspace configuration topic for that .

    If available, the Copy Settings from Source allows you to copy the source connection details to the destination database, if both databases are in the same location. Structural does not copy the connection password.

    Some data connectors allow you to write the transformed data to a data volume in a container repository instead of to a database server.

    For more information, go to .

    A workspace uses files as its source data and produces transformed versions of those files as its output.

    For file connector workspaces, the File Location section indicates where the source files are obtained from - either a local file system or a cloud storage solution (Amazon S3 or Google Cloud Storage).

    When the files come from cloud storage, the Output Location section indicates where to write the transformed files. You must also provide the cloud storage connection credentials.

    For more information, go to .

    Transfer ownership of a workspace

    Make another team user the workspace owner. You can also assign yourself workspace permission sets.

    Destination database

    Container repository

    File connector source and destination data

    connector type
    Writing output to a container repository
    file connector
    Configuring the file connector storage type and output options
    Change your Structural password
    Delete your Structural account.

    Refreshing the schema cache

    workspace Jobs view
    Refresh Schema option for a workspace schema cache

    A one-off key - single use, for a single server.

  • A reusable key - for multiple servers.

  • The generated key starts with tskey-.

  • After you generate the key, run:

  • You used a user auth key or a pre-auth key instead of an OAuth client secret.

    select a secret name from a secrets manager
    sudo tailscale up --auth-key=<key>
    tailscale ip -4

    Array Character Scramble

    A version of the Character Scramble generator that can be used for array values.

    This generator replaces letters with random other letters, and numbers with random other numbers. Punctuation and whitespace are preserved.

    For example, for the following array value:

    ["ABC.123", 3, "last week"]

    The output might be something like:

    ["KFR.860", 7, "sdrw mwoc"]

    This generator securely masks letters and numbers. There is no way to recover the original data.

    Characteristics

    To configure the generator, toggle the Consistency setting to indicate whether to make the generator self-consistent.

    By default, the generator is not consistent.

    If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • 3 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    Consistency

    Yes, can be made self-consistent.

    Linking

    No, cannot be linked.

    Differential privacy

    No

    Data-free

    How to configure

    Structural data encryption

    No

    Viewing the column list

    The column list on Database View contains information about the sensitivity and generator configuration for each column.

    Column list on Database View

    Column - Column name and type

    The Column column provides general information about the columns and their content, including:

    • Table and column name. When you click the column name, Table View for the column table displays.

    • The name of the schema that contains the table.

    • The data type for the column.

    • An indicator when the column is a primary key

    The Column column also contains the option to .

    The Status column provides information about whether the column contains sensitive data and whether it has an assigned generator.

    The protection status can be one of the following values:

    • Protected - The column has an assigned generator.

    • Not Sensitive - The column is marked as not sensitive.

    • At Risk - The column is sensitive and does not have an assigned generator.

    At the right of the Status column is a confidence indicator. For At Risk columns, the confidence indicator shows how confident Structural is that the column is sensitive and contains values of the displayed sensitivity type. Protected columns also reflect the original confidence level.

    For more information about how Structural identifies values and assigns the confidence level, go to .

    From the Status column, you can .

    The Applied Generator column is where you .

    The generator dropdown indicates the currently assigned generator. It also indicates when an unprotected column has a recommended generator.

    For foreign key columns, the generator dropdown is disabled and the column is marked as a foreign key. Foreign key columns always inherit the generator that is assigned to the primary key.

    In a child workspace, when the generator configuration overrides the parent workspace, the generator dropdown displays the override icon.

    The Applied Generator column also contains the option to .

    You can use the to display a filtered list of columns. For example:

    • Display Database View filtered to show all of the unprotected columns.

    • Display Database View filtered to show non-sensitive columns in the stores and vendors tables.

    To filter the column list manually from Database View, you can:

    • Use the table list to filter the displayed columns based on the table that the columns belong to.

    • Use the filter field to filter the columns by table or column name.

    • Use the Filters panel to filter the columns based on column attributes and generator configuration.

    You can use column filters to quickly find columns that you want to verify or to update the configuration for.

    To filter the column list to only include columns for specific tables, either:

    • .

    • Check the checkbox for each table to display columns for.

    To filter the column list by table or column name, in the filter field, begin to type text that is in the table or column name.

    As you type, Structural filters the column list.

    The Filters panel provides access to column filters other than the table and column name.

    To display the Filters panel, click Filters. The list only includes the filters that apply to the displayed data.

    To search for a filter or a filter value, in the search field, start to type the value. The search looks for text in the individual settings.

    For each filter, the Filters panel indicates the number of matching columns, based on the selected tables and the current filters.

    To add a filter, depending on the filter type, either check the checkbox or select a filter option. As you add filters, Structural applies them to the column list. Above the list, Structural displays tags for the selected filters.

    To clear all of the currently selected filters, click Clear All.

    To only display detected sensitive columns for which there is a recommended generator, on the Filters panel, check Has Generator Recommendation.

    An at-risk column:

    • Is marked as sensitive.

    • Is included in the destination data.

    • Is assigned the Passthrough generator.

    To only display at-risk columns, on the Filters panel, check At-Risk Column.

    When you check At-Risk Column, Structural adds the following filters under Privacy Settings:

    • Sets the sensitivity filter to Sensitive

    • Sets the protection status filter to Not protected

    • Sets the column inclusion filter to Included

    You can filter the columns based on the column sensitivity.

    On the Filters panel, under Privacy Settings, the sensitivity filter is by default set to All, which indicates to display both sensitive and non-sensitive columns.

    • To only display sensitive columns, click Sensitive.

    • To only display non-sensitive columns, click Not sensitive.

    Note that when you check At-risk Column, Structural automatically selects Sensitive.

    You can filter the columns based on whether they have any generator other than Passthrough assigned. To filter the columns based on specific assigned generators, use the Applied Generator filter.

    On the Filters panel, under Privacy Settings, the column protection filter is by default set to All, which indicates to display both protected and not protected columns.

    • To only display columns that have an assigned generator, click Protected.

    • To only display columns that do not have an assigned generator, click Not protected.

    Note that when you check At-Risk Column, Structural automatically selects Not protected.

    You can filter the columns based on whether they are populated in the destination database. For example, if a table is truncated, then the columns in that table are not populated.

    On the Filters panel, under Privacy Settings, the column inclusion filter is by default set to All, which indicates to display both included and not included columns.

    • To only display columns that are populated in the destination database, click Included.

    • To only display columns that are not populated in the destination database, click Not included.

    Note that when you check At-Risk Column, Structural automatically selects Included.

    To only display columns that are assigned specific generators, on the Filters panel, under Applied Generator, check the checkbox for each generator to include.

    The list of generators only includes generators that are assigned to the currently displayed columns and that are compatible with other applied filters.

    To search for a specific generator, in the Filters search field, begin to type the generator name.

    You can filter the columns by the column data type. For example, you can only display varchar columns, or only columns that contain either numeric or integer values.

    To only display columns that have specific data types, on the Filters panel, under Database Data Types, check the checkbox for each data type to include.

    The list of data types only includes data types that are present in the currently displayed columns and that are compatible with other applied filters.

    To search for a specific data type, in the Filters search field, begin to type the data type.

    When the source database schema changes, you might need to update the configuration to reflect those changes. If you do not resolve the schema changes, then the data generation might fail. The data generation fails if there are unresolved conflicting changes, or if you configure Structural to always fail data generation when there are any unresolved changes.

    For more information about schema changes, go to .

    To only display columns that have unresolved schema changes, on the Filters panel, check Unresolved Schema Changes.

    For detected sensitive columns, the sensitivity type indicates the type of data that was detected. Examples of sensitivity types include First Name, Address, and Email.

    To only display columns that contain specific sensitivity types, on the Filters panel, under Sensitivity Type, check the checkbox for each sensitivity type to include.

    The list of sensitivity types only includes sensitivity types that are present in the currently displayed columns.

    To search for a specific sensitivity type, in the Filters search field, type the sensitivity type.

    When the Structural sensitivity scan , it also determines how confident it is in that determination. The Status column displays the confidence level.

    You can filter the columns based on the confidence level.

    To only display columns that have a specific confidence level, on the Filters panel, under Sensitivity confidence, check the checkbox next to each confidence level to include.

    You can filter the column list based on whether the column is nullable.

    On the Filters panel, under Data Attributes, the nullability filter is by default set to All, which indicates to display both nullable and non-nullable columns.

    • To only display columns that are nullable, click Nullable.

    • To only display columns that are not nullable, click Non-nullable.

    You can filter the column list based on whether the column must be unique.

    On the Filters panel, under Data Attributes, the uniqueness filter is by default set to All, which indicates to display both unique and not unique columns.

    • To only display columns that must be unique, click Unique.

    • To only display columns that do not require uniqueness, click Not unique.

    You can filter the column list to indicate whether to include:

    • Columns that are not primary or foreign keys.

    • Columns that are foreign keys.

    • Columns that are primary keys.

    On the Filters panel, under Column Type:

    • To display columns that are neither a primary key nor a foreign key, check Non-keyed.

    • To display columns that are primary keys, check Primary key.

    • To display columns that are foreign keys, check Foreign key.

    In a child workspace, to only display columns that override the generator configuration that is in the parent workspace, on the Filters panel, check Overrides Inheritance.

    You can enable Structural data encryption, a configuration that allows Structural to:

    • Decrypt source data before applying the generator.

    • Encrypt generated data before writing it to the destination database.

    For more information, go to .

    When Structural data encryption is enabled, the generator configuration panel includes an option to use Structural data encryption.

    To only display columns that are configured to use Structural data encryption, on the Filters panel, check Uses Data Encryption.

    You can filter the columns to only display those for which Structural automatically assigned the generator. Based on the , this can happen during data generation in response to changes to the source data schema.

    To only display columns that have automatically applied generators, on the Filters panel, toggle Automatically Applied Generators to the on position.

    By default, the column list is sorted first by table name, then by column name. The columns for each table display together. Within each table, the columns are in alphabetical order.

    You can also sort the column list by column name first, then by table. Columns that have the same name display together. Those columns are sorted by the name of the table.

    The button at the right of the Column column heading indicates the current sort order.

    • T.C indicates that the table is sorted by table, then by column

    • C.T indicates that the table is sorted by column, then by table

    To switch the sort order, click the button.

    ArrayTextMaskGenerator

    Status - Protection and sensitivity status

    Applied Generator - Column configuration

    Filtering the column list

    Filter by table

    Filter by table or column name

    Using the Filters panel

    Searching for a filter

    Adding a filter

    Clearing the selected filters

    Filters panel filters

    Columns with generator recommendations

    At-risk columns

    Sensitivity

    Protection status

    Inclusion in the destination database

    Assigned generator

    Column data type

    Unresolved schema changes

    Sensitivity type

    Sensitivity confidence

    Column nullability

    Column uniqueness

    Primary or foreign keys

    Generator overrides in a child workspace

    Uses Structural data encryption

    Columns with automatically assigned generators

    Sorting the column list

    display sample data for the column
    change whether a column is sensitive
    select and configure the generator to assign
    display and create column comments
    Apply a filter to the table list
    Viewing and resolving schema changes
    identifies a value as belonging to a sensitivity type
    Configuring and using Structural data encryption
    workspace configuration
    Status Values for a column
    Confidence level indicators for database columns
    Unprotected column that has a recommended generator
    Disable generator dropdown for a foreign key column
    Column with a generator configuration that overrides the parent workspace
    Filtering columns by name
    Filters panel for columns
    Using the column filter search
    Filters panel with filters selected
    Sort button in the Column heading
    How Structural identifies sensitive values
    Structural Agent
    Tutorial video: Sensitivity detection and generator recommendations
    Tutorial video: Creating a Structural workspace
    Tutorial video: Managing workspace access
    Tutorial video: Tonic Structural generators overview
    Tutorial video: Subsetting your data
    Tutorial video: Generating data with consistency
    Tutorial video: Using Document View to configure JSON columns
    Tutorial video: Tonic Structural 101
    Tutorial video: File connector overview
    Tutorial video: Enabling upsert data generation
    Tutorial video: Outputting data to a container repository
    Tutorial video: Generator presets

    Using a connection tunneling option

    For additional security, for data connectors that support it, you can specify a connection tunneling option for your data connections.

    Structural currently supports:

    • Routing the data connections through an SSH bastion.

    • Routing the data connections through a Tailscale tailnet. Tailscale is currently supported on MySQL, Oracle, PostgreSQL, and SQL Server. Snowflake also supports Tailscale as an option instead of a proxy connection for the source and destination connections.

    Identifying similar columns

    During sensitivity scans and schema change scans, Tonic Structural identifies groups of similar columns.

    To identify similar columns, Structural uses a text embedding model to calculate the semantic similarity between any two column names in the database. When a column name's semantic similarity to the name of a given column is above a specified threshold, then the column is similar to the given column.

    If a column has similar columns, then the Applied Generator column contains an icon that includes the count of similar columns.

    Similar columns icon with the count of similar columns

    By default, the similar columns icon is hidden. To display the similar columns icon, hover over the column row.

    When you assign a generator to a column, the similar columns icon for that column remains visible during your current session.

    When you click the similar columns icon, Structural displays a panel with an option to filter the list to display the current column and its similar columns. To apply the filter, click Filter.

    Similar columns panel with filter option

    The similar columns filter is applied, and other table and column filters are removed.

    Tonic Structural User Guide

    The Tonic Structural platform creates safe, realistic datasets to use in staging environments or for local development. The Structural web application, which includes the AI chat, and the can be used by engineers, data analysts, or security experts.

    Structural connects to source databases that contain sensitive data such as personally identifiable information (PII) or protected health information (PHI). To protect that data, Structural transforms the sensitive values and then writes the transformed data to a destination location.

    New to Structural? Review the . For information on how to create a Structural account and start a Structural free trial, go to .

    Want to know what's in the latest Structural releases? Go to the .

    Frequently Asked Questions

    The minimum screen width is 1120 pixels.

    If the locally running database that you want to connect to runs in a Docker container:

    1. Run: docker inspect

    2. In the networks section of the results, find the Gateway IP address. Use this IP address as the server address in Structural.

    About the Structural Agent

    The Structural Agent is an LLM-powered, chat-based tool to help you to understand workspace data and to configure workspaces for data generation.

    The Structural Agent chat initially starts when you save your first workspace.

    The Structural Agent can automatically perform the following tasks.

    For workspaces that have not yet generated data, the Structural Agent checks whether the workspace has a valid connection to source data. If not, it prompts you to go to workspace settings and configure the connection.

    If there is a source data connection, then the Agent checks the status of the initial sensitivity scan, and prompts you to run a scan if the scan hasn't started.

    For a file connector workspace, the Structural Agent checks that you have created at least one file group with files.

    When the sensitivity scan is complete, the Structural Agent provides a summary of the scan results.

    Depending on the number of tables and columns, the Structural Agent can provide a complete list of the sensitive columns, or a more condensed summary.

    ASCII Key

    Generates unique alphanumeric strings based on any printable ASCII characters. The length of the source string is not preserved. You can choose to exclude lowercase letters from the generated values.

    Displaying sample data for a column

    For each column on Database View, you can display a sample list of the column values.

    For columns that have an assigned generator, the sample shows both the current values and the possible values after the generator is applied.

    To display the sample values, in the Column column, click the magnifying glass icon.

    If the generator is Passthrough, then the sample data panel contains only Original Data.

    If a different generator is assigned, then the sample data panel contains both Original Data and Protected Output.

    Connect through an SSH bastion Set up and use an SSH bastion for connection tunneling.

    Connect through Tailscale Use Tailscale for connection tunneling.

    Required workspace permission:

    • Source data: Preview source data

    • Destination data: Preview destination data

    Sample data for a column that does not have an assigned generator
    Sample data for a column that has an assigned generator
    Database View column list with a similar columns filter applied

    To check the current operational status of Structural Cloud, go to status.tonic.ai.

    Need help with Structural? Contact support@tonic.ai.

    Connect to your data

    Configure and generate transformed data

    Structural Agent
    Structural API
    Tonic Structural workflow overview
    Getting started with the Structural free trial
    Tonic Structural release notes
    Data flow from the source database through Tonic Structural to the destination database

    Workspaces

    A workspace contains the data connections and data generation configuration.

    Data connectors

    Each data connector allows Structural to read from and write to a specific type of data source.

    Configure and manage Structural

    If the locally running database does NOT run in a container, but runs on the machine, then:
    • On Windows or Mac, use host.docker.internal.

    • On Linux, use 172.17.0.1, which is the IP address of the docker0 interface.

    If you use Structural Cloud, and your database only allows connections from allowlisted IP addresses, then you need to allowlist Structural static IP addresses.

    This is not required for self-hosted instances of Structural.

    For the United States-based instance (app.tonic.ai), the static IP address is:

    • 54.92.217.68

    For the Europe-based instance (app-de.tonic.ai), the static IP address is:

    • 3.69.249.144

    The URL https://telemetry.tonic.ai/ is used for our Amplitude telemetry.

    https://telemetry.tonic.ai/logs is used specifically for log sharing.

    Allowlist https://telemetry.tonic.ai/ or the following IP address:

    • 44.193.110.147

    Telemetry sharing is required. These metrics are valuable for us as we debug, make product roadmaps, and determine feature viability.

    No customer data is included. For more information about the specific telemetry data that we collect, go to Data that Tonic.ai collects.

    For more information on how to verify that telemetry is shared, go to Verifying and enabling telemetry sharing.

    To support the one-click update option, Structural needs to be able to retrieve information about the latest Structural version.

    For more information, go to .

    On a self-hosted instance, you can use Structural's hosted LLM to support Structural AI features.

    If you do use the hosted LLM, then you must allowlist the SaaS proxy that the LLM requests are routed through, either:

    • http://us-east-1.saasproxy.tonic.ai/ or 98.84.248.119 (US location)

    • http://eu-central-1.saasproxy.tonic.ai/ or 3.120.214.225 (EU location)

    Click your user image at the top right. The menu includes the Tonic version.

    We recommend that you use a static copy of your production database that was restored from a backup.

    If that's not possible, consider the following when you connect Structural to your source data:

    • Structural cannot guarantee referential integrity of the output data if the source database is written to while data is generated. For this reason we recommend that you connect to a static copy of production data.

    • Read replicas and fast followers can be problematic for Structural because of how long it takes some queries to run. Read replicas tend to have short query timeout limits, which causes the queries to time out. Read replicas also reflect recent writes, which means that we cannot guarantee the referential integrity of the output.

    For details about the available AI features and how they are supported, go to AI in Structural.

    For details about the types of data that Tonic.ai does and does not collect, go to Data that Tonic.ai collects.

    What is the minimum required screen width for the Tonic Structural application?

    How do you connect to a local database when running Structural in a Docker container locally?

    I allowlist access to my database. What are your static IP addresses?

    United States-based instance

    Europe-based instance

    I allowlist network calls. What do I need to allowlist?

    URLs for telemetry sharing

    URLs for Structural version information

    SaaS proxy for the Structural hosted LLM

    How do I check my current version of Structural?

    How should we provision our source database?

    How does Structural use AI?

    What data does Tonic.ai collect from Structural?

    Based on the scan results, the Structural Agent suggests generators to apply. It can suggest a built-in or custom generator preset, or another specific generator configuration.

    Agent prompt to assign generators

    When all of the sensitive columns are protected, after it verifies that the workspace has a valid destination connection, the Agent prompts you to run the data generation.

    The Agent cannot start the data generation.

    What is the Structural Agent?

    What can the Agent do automatically?

    Check for a source data connection

    Summarize the sensitivity scan results

    Prompt to apply recommended generators

    Prompt to generate data

    Agent summary of the sensitivity scan

    Differential privacy

    No

    Data-free

    No

    Allowed for primary keys

    Yes

    Allowed for unique columns

    Yes

    Uses format-preserving encryption (FPE)

    Yes

    Privacy ranking

    • 3 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    To configure the generator:

    1. To exclude lowercase letters from the generated values, toggle Exclude Lowercase Alphabet to the on position.

    2. Toggle the Consistency setting to indicate whether to make the generator consistent. By default, the generator is not consistent.

    3. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Consistency

    Yes, can be made self-consistent.

    Linking

    No, cannot be linked.

    Characteristics

    How to configure

    Structural implementation roles

    A Tonic Structural implementation can involve the following roles - from those who set up the Structural environment to the consumers of the data that Structural processes.

    Note that these roles are not related to role-based access (RBAC) within Structural, which is managed using permission sets.

    Infrastructure engineers

    For self-hosted instances of Structural.

    Infrastructure engineers set up the Structural application and its relevant dependencies. They are typically DevOps, Site Reliability Engineering (SRE), or Kubernetes cluster administrators.

    Infrastructure engineers perform the following Structural-related tasks:

    • Ensure that the proper infrastructure is ready for Structural installation based on the deployment checklist.

    • . Works with Tonic.ai support as needed.

    • Perform routine maintenance of Structural and the Structural environment. and its dependencies as needed.

    • Create Structural-processed data pipelines for development and testing workflows.

    For both self-hosted instances of Structural and Structural Cloud.

    Database administrators integrate Structural into your data architecture to support .

    They ensure that source databases are available to Structural, and that Structural can write to destination databases.

    perform the following Structural-related tasks:

    • Set up the required Structural access to source databases.

    • Set up destination databases for Structural to write transformed data to.

    Structural users are the actual users of the Structural application.

    Depending on the use case, Structural users might be compliance analysts, DevOps, or data engineers.

    Tonic users perform the following Structural-related tasks:

    • Use the to configure the logic used to transform the source data and to generate the transformed data.

    • Work with data consumers to produce usable data.

    Data consumers are the end users of transformed destination data.

    They are typically QA testers, developers, or analysts.

    Data consumers perform the following Structural-related tasks:

    • Validate the usability of the destination data.

    • Provide guidance on application-specific requirements for data.

    Security and compliance specialists ensure and validate that the data that Structural produces meets expectations, and that Structural is compliant with other security-related processes.

    Security and compliance specialists perform the following Structural-related tasks:

    • Provide guidance on what data is sensitive.

    • Sign off on proposed approaches to mask sensitive data.

    • Approve data access and permissions.

    Database View

    Database View provides a complete view of your source database structure and configuration.

    To display Database View, either:

    • On the workspace management view, in the workspace navigation bar, click Database View.

    • On Workspaces view, from the dropdown menu in the Name column, select Database View.

    Database View consists of:

    • On the left, the list of tables in the source database.

    • On the right, the list of columns in those tables.

    Transferring ownership of a workspace

    Required permission

    • Team permission: View team users. This permission is only required for the Tonic Structural application. It is not needed when you use the Structural API.

    • Either:

    About workspace ownership

    Every workspace has an owner. The owner is always a user.

    The user who creates the workspace is automatically the owner of the workspace.

    By default, the workspace owner is assigned the built-in Manager workspace permission set. On Enterprise instances, you can choose a different workspace permission set to assign to all workspace owners.

    You cannot remove that permission set from the workspace owner.

    About ownership transfer

    You can transfer a workspace to a different owner.

    When you transfer ownership of a workspace, the new owner is assigned the owner permission set.

    If the previous owner was not separately granted access to the owner permission set, then that permission set is removed.

    Completing the ownership transfer

    To transfer workspace ownership:

    1. To transfer ownership of a single workspace, from the workspace actions menu, select Transfer Ownership.

    2. To transfer ownership of multiple workspaces:

      1. Check the checkbox for each workspace to grant access to.

      2. From the Actions menu, select Transfer Ownership.

    3. On the transfer ownership panel, from the User dropdown list, select the new owner.

    4. If you are the current owner of the workspace, then to grant yourself non-owner access after you transfer the ownership:

      1. Toggle Receive access to workspace to the on position.

      2. Select the workspace permission set to assign to yourself.

    5. Click Transfer Ownership.

    Categorical

    The Categorical generator shuffles the existing values within a field while maintaining the overall frequency of the values. It disassociates the values from other pieces of data. Note that NULL is considered a separate value.

    For example, a column contains the values Small, Medium, and Large. Small appears 3 times, Medium appears 4 times, and Large appears 5 times. In the output data, each value still appears the same number of times, but the values are shuffled to different rows.

    This generator is optimized for categories with fewer than 10,000 unique values. If your underlying data has more unique values (for example, your field is populated by freeform text entry), we recommend that you use the or generator.

    Workspace permission: Transfer workspace ownership

  • Team permission: Manage user access within the team

  • To grant yourself access after the transfer:

    • Workspace permission: Share workspace access

    Privacy Hub

    View and update the current protection status based on the sensitivity scan and workspace configuration.

    Database View

    Configure transformation options for tables and columns.

    Generators

    A generator is assigned to a column and performs a data transformation.

    Subsetting

    Configure a subset of source data to include in the transformed destination data.

    Generate data

    Run the data generation process to produce transformed destination data.

    Schema changes

    Review and address changes to the source data schema.

    Structural architecture

    Components of the Structural application.

    Manage Structural access

    Manage who has access to Structural instances and organizations.

    Configure Structural settings

    Customize environment settings to control Structural behavior.

    Monitor Structural and view logs

    Monitor Structural services and share logs with Tonic.ai.

    Database administrators

    Structural users

    Data consumers

    Security and compliance

    Follow the installation instructions
    Updates Structural
    Structural data connectors
    Database administrators
    Structural data generation workflow
    AsciiPkGenerator

    View table and column information

    Configure and comment on columns

    Database View

    View and configure tables

    Filter the table list, and assign table modes to tables.

    View the column list

    Apply filters to and sort the list of columns.

    View sample data

    View example source and destination data for a column.

    Configure an individual column

    Assign a generator and determine the column sensitivity.

    Configure multiple columns

    Use the bulk edit option to configure multiple columns.

    Identify similar columns

    Identify and filter to columns that are similar to a column, based on the column name.

    Comment on columns

    Add and respond to column comments.

    Consistency

    No, cannot be made consistent.

    Linking

    Yes, can be linked.

    Differential privacy

    Configurable

    To configure the generator:

    1. From the Link To dropdown, select the columns to link to the current column. You can select from other columns that use the Categorical generator.

    2. Toggle the Differential Privacy setting to indicate whether to make the output data differentially private. By default, differential privacy is disabled.

    3. If Structural data encryption is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Characteristics

    Character Scramble
    Custom Categorical

    How to configure

    Conditional

    This is a .

    Applies different generators to the value conditionally based on any value in the table.

    For example, a Users table contains Name, Username, and Role columns. For the Username column, you can use a conditional generator to indicate that if the value of Role is something other than Test, then use the Character Scramble generator for the Username value. For Test users, the name is not masked.

    You can also create conditions against the current column. For example, if the value is less than 4, use the Random Integer generator to generate a replacement value between 0 and 4. If the value is greater than or equal to 5, use the Random Integer generator to generate a value between 5 and 10.

    Data-free

    No

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • 2 if differential privacy enabled

    • 3 if differential privacy not enabled

    Generator ID (for the API)

    CategoricalGenerator

    Determined by the selected generators.

    Linking

    Determined by the selected generators.

    Differential privacy

    Determined by the selected generators.

    Data-free

    Determined by the selected generators.

    Allowed for primary keys

    Yes, but:

    • Make sure that the configuration preserves uniqueness.

    • Do not use on primary key columns that are used for subsetting.

    Allowed for unique columns

    Yes

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    • If a fallback generator is selected, then the lower of either 5 or the fallback generator.

    • 5 if no fallback generator is selected

    Generator ID (for the API)

    The generator consists of a list of options. Each option includes the required conditions and the generator to use if those conditions are met.

    The generator always contains a Default option. The Default option is used if the value does not meet any of the conditions. To configure the Default option:

    1. From the Default dropdown list, select the generator to use by default.

    2. Configure the selected generator.

    To add a condition option:

    1. Click + Conditional Generator.

    2. To add a condition:

      1. Click + Condition.

      2. From the column list, select the column for which to check the value. To check the value of the current column, select This field.

      3. Select the comparison type.

      4. Enter the column value to check for.

      To remove a condition, click the delete icon for the condition.

    3. From the Generator dropdown list, select the generator to run on the current column if the conditions are met. You cannot select another composite generator.

    4. Choose the configuration options for the selected generator.

    To view details for and edit a condition option, click the expand icon for that option.

    To remove a condition option, click the delete icon for the option.

    Characteristics

    composite generator

    Consistency

    How to configure

    Setting the default generator

    Adding a condition option

    Viewing and editing condition options

    Removing a condition option

    Privacy Hub

    About Privacy Hub

    Privacy Hub tracks the current protection status of source data columns based on:

    • Column sensitivity, either from the most recent sensitivity scan or from manual assignments

    • Assigned table modes

    • Assigned generators

    To display Privacy Hub, either:

    • On the workspace management view, in the workspace navigation bar, click Privacy Hub.

    • On Workspaces view, click the workspace name.

    From Privacy Hub, you can:

    • Review and apply the recommended generators for all detected sensitive columns

    • View the current protection status of columns

    • Manually mark columns as sensitive or not sensitive

    • Configure protection for sensitive columns

    The sensitivity scan detects specific types of sensitive data.

    If your workspace contains any columns that the sensitivity scan identified, and for which you have not either:

    • Assigned a generator

    • Marked as not sensitive

    Then Tonic Structural displays a Sensitivity Recommendations banner that contains a count of those columns.

    The count only includes sensitive columns that the sensitivity scan detects. If you manually mark a column as sensitive, it is not included in the list.

    On the banner, the Review Recommendations option allows you to review the detected columns and the recommended generators for each detected sensitive data type.

    You can then apply the recommended generators or ignore the recommendations. When you ignore a recommendation, you either:

    • Indicate to remove the generator recommendation for the column.

    • Indicate that the column data is not sensitive.

    For more information, go to .

    The protection status panels at the top of Privacy Hub provide an overview of the current protection status of the columns in the source data.

    Each panel displays:

    • The number of columns that are in that category.

    • The estimated percentage of columns that are in that category.

    Note that for a , the protection status displays a separate box for each combination of JSON path and data type.

    From each panel, you can .

    The column counts do not include columns that do not have data in the destination database. For example, if a table is assigned Truncate table mode, then Privacy Hub ignores the columns in that table.

    The information on these panels updates automatically as you change whether columns are sensitive and assign generators to columns.

    The At-Risk Columns panel reflects columns that:

    • Are populated in the destination database.

    • Are marked as sensitive.

    • Have the generator set to Passthrough, which indicates that Structural does not perform any transformation on the data.

    For each column, the At-Risk Columns panel also indicates the sensitivity confidence, from full confidence (completely red) to low confidence (a small percentage of red).

    The goal is to have 0 at-risk columns.

    When you click Open in Database View, you navigate to . The column list is filtered to show columns that are at risk.

    The Protected Columns panel reflects columns that:

    • Are populated in the destination database.

    • Are assigned a generator other than Passthrough.

    It includes both sensitive and non-sensitive columns.

    Note that a column is considered protected based solely on the assigned generator. Some more complex generators, such as JSON Mask or Conditional, allow you to apply different generators to specific portions of a value or based on a specific condition. However, the protection status does not reflect these sub-generators. An applied sub-generator could be Passthrough.

    When you click Open in Database View, you navigate to . The column list is filtered to show all included columns that are protected.

    The Not Sensitive Columns panel reflects columns that:

    • Are populated in the destination database.

    • Are marked as not sensitive.

    • Have the generator set to Passthrough.

    When you click Open in Database View, you navigate to . The column list is filtered to show included columns that are not sensitive and are not protected.

    The Database Tables list shows the protection status for each table in the source database. You can view the number of columns that have each protection status, and update the column configuration.

    The list does not include tables where the table mode is Truncate or Preserve Destination. Truncated tables are not populated in the destination database. For Preserve Destination tables, the existing data in the destination database does not change.

    For each table, Database Tables provides the following information:

    • Name - The table name. For a workspace, each table corresponds to a file group. Each is also in a separate row. For JSON columns, the Name column displays both the table name and the column name. When you click a table name, you can navigate to either:

      • Database View, filtered to to display the columns for that table.

      • Table View for that table.

    You can filter the Database Tables list either by the table name or by the schema.

    To filter the list by table name, in the filter field, begin to type text that is in the table name. As you type, Structural updates the list to only display matching tables.

    To filter the list to only include tables that belong to a specific schema:

    1. Click Filter by Schema.

    2. From the schema dropdown list, select the schema.

    When you select a schema, Structural adds it to the filter field.

    You can sort the Database Tables list by any column except for the Privacy Status column.

    To sort by a column, click the column heading. To reverse the sort order, click the heading again.

    The Privacy Status column in the Database Tables list indicates the protection status of the columns in the table.

    This column provides the same as the protection status panels at the top of Privacy Hub, but is limited to the columns in a specific table.

    Each protection status panel displays a series of boxes to represent the columns that apply to that status. For example, if the source data contains four columns that are at-risk, then the At-Risk Columns panel displays four boxes, one for each column.

    The Privacy Status column in the Database Tables list displays the same set of boxes for the columns in an individual table.

    If the number of columns is too large to fit, then the last box shows the number of additional columns that apply. For example, if there are 15 columns that don't fit, then the last box is labeled +15.

    When you hover over a box, the column name displays in a tooltip.

    When you click a box, the details panel for that column displays.

    When you click the box for remaining columns, the details panel for the first column in the remaining columns displays.

    You can use the next and previous icons at the bottom right of the details panel to display the details for the next or previous column.

    The column details panel opens to the settings view. The settings view contains the following information:

    • The table and column name.

    • Whether the column is flagged as sensitive.

    • The type of sensitive data that the column contains.

    • The data type for the column data.

    From the settings view of the column details, you can configure the column sensitivity.

    You cannot change the sensitivity of columns in a child workspace. A child workspace always inherits the sensitivity from its parent workspace. For more information, go to .

    As you change the column sensitivity, Structural updates the protection status panels.

    To change whether the column is sensitive, toggle the Sensitive option. The column is moved if needed to reflect its new status. However, you remain on the current panel.

    For example, from the At-Risk Columns panel, you change a column to be not sensitive. The column is moved to the Not Sensitive Columns panel. When you click the next or previous icons, you view the details for the next or previous column on the At-Risk Columns panel.

    From the column details, you can assign and configure the column generator.

    When you change the column generator, Structural updates the protection status panels.

    If the column generator was previously Passthrough, then the column is moved to the Protected Columns panel. However, you remain on the current panel. For example, you assign a generator to a column that is on the At-Risk Columns panel. The column is moved to the Protected Columns panel, but when you click the next or previous icons, you view the details for the next or previous column on the At-Risk Columns panel.

    For sensitive columns that are not protected, Structural displays the recommended generator as a button.

    For self-hosted instances that have an Enterprise license, the recommended generator is the built-in generator preset.

    To assign the recommended generator to the column, click the button.

    Otherwise, select the generator from the Generator Type dropdown list.

    For more information about selecting a generator, go to .

    If the selected generator requires additional configuration, then below the Generator Type dropdown list is an Edit Generator Options link.

    To display the configuration fields for the generator, click Edit Generator Options.

    For information about configuring a selected generator or generator preset, go to .

    After you configure the generator, to return to the settings view, click Back.

    From the column details, you can display sample data for the column. The sample data allows you to compare the source and destination versions of the column values.

    To display the sample data, click the view sample (magnifying glass) icon.

    On the sample data view of the column details:

    • The Original Data tab shows the values in the source data.

    • The Protected Output tab shows the values that the generator produced.

    For a JSON column, instead of assigning a generator, you can enable Document View.

    From Document View, you can view the JSON schema structure and assign generators to individual JSON fields. For more information, go to .

    To enable Document View, on the column details panel, toggle Use Document View to the on position. When Document View is enabled, the generator dropdown is replaced with the Open in Document View option.

    From the column details, you can view and add comments on the column. You might use a comment to explain why you selected a particular generator or marked a column as sensitive or not sensitive.

    From the column details, to display the comments for the column, click the comment icon.

    The comments view displays any existing comments on the column. The most recent comment is at the bottom of the list. Each comment includes the name of the user who made the comment.

    To add the first comment to a column, type the comment into the comment text area, then click Comment.

    To add an additional comment, type the comment into the comment text area, then click Reply.

    The Privacy Report files that you download from Privacy Hub or the workspace download menu provide an overview of the current protection status based on the current configuration.

    This is different from the Privacy Report files that you download from the data generation job details, which show the protection status for the data produced by that data generation.

    You can download either:

    • The Privacy Report .csv file, which provides details about the table columns, the column content, and the current protection configuration.

    • The Privacy Report PDF file, which provides charts that summarize the privacy ranking scores for the table columns. It also includes the table from the .csv file.

    For more information about the Privacy Report files and their content, go to .

    To download the report from the workspace management view, click the download icon. In the download menu:

    • To download the Privacy Report PDF file, click Download Privacy Report PDF.

    • To download the Privacy Report .csv file, click Download Privacy Report CSV.

    To download the report from Privacy Hub, click Reports and Logs, then:

    • To download the Privacy Report .csv file, click Privacy Report CSV.

    • To download the Privacy Report PDF file, click Privacy Report PDF.

    Privacy Hub provides an option to manually start a new . For example, you might want to run a new sensitivity scan when:

    • You add columns to the source database. The new scan identifies whether the new columns contain sensitive data.

    • The data in a column changes significantly, and a column that Structural originally marked as not sensitive might now contain sensitive data.

    You cannot run a sensitivity scan on a . Child workspaces always inherit the sensitivity results from their parent workspace.

    To run a new sensitivity scan, click Run Sensitivity Scan.

    When Structural runs a new sensitivity scan:

    • Structural analyzes and determines the sensitivity of any new columns.

    • It does not change the sensitivity of existing columns that you marked as sensitive or not sensitive.

    • For existing columns that you did not change the sensitivity of:

    The protection status panels are updated to reflect the results of the new scan.

    Indicating whether a column is sensitive

    Selecting a secrets manager secret

    For fields that support secrets managers, such as database password fields, if at least one secrets manager is available, then at the top right of the field is a Use Secret link.

    Use Secret option for a field that supports using a secret from a secrets manager

    Indicating to use a secret

    To use a secret to populate the value:

    1. Click Use Secret.

    Use Secret panel to select the secret to use
    1. On the Use Secret panel, from the Secrets Manager dropdown list, select the name of the secrets manager that contains the secret.

    2. Based on the secrets manager type, Structural prompts you for the information needed to identify and retrieve the secret.

    3. After you provide the required information, click Confirm.

    4. Structural changes the link to Using Secret, and disables the field.

    To change the secret selection or other information about the selected secret:

    1. Click the Using Secret link.

    2. On the Use Secret panel, update the information.

    3. Click Confirm.

    When you select a secrets manager from AWS Secrets Manager:

    1. In the Secret ARN or Name field, provide the name or ARN of the secret.

    2. If the secret is part of a structured key-value pair, then in the Property Name field, provide the property name that contains the secret value.

    When you select a secrets manager from HashiCorp Vault:

    For HashiCorp vault, for additional security, you can choose to use chained credentials. When you enable chained credentials, you provide a set of credentials that is used in turn to retrieve the credentials that are used to retrieve the specified secret.

    To enable chained credentials, toggle Use chained credentials to the on position. The Chained Credentials Configuration is displayed.

    From the Method dropdown list, select the type of authentication to use:

    • AppRole

    • LDAP

    • Token

    For all authentication types:

    1. If the selected authentication method is enabled in a specific namespace, then in the first Namespace field, provide the namespace.

    2. If the selected authentication method does not use the default mount path, then in the first Mount path field, provide the mount path.

    3. In the Secret Name field, provide the name of the secret.

    4. If the vault is enabled in a specific namespace, then in the second

    For app role authentication:

    1. In the Role ID field, provide the name of the secret property that contains the identifier of the application role.

    2. In the Secret ID field, provide the name of the secret property that contains the secret identifier of the application role.

    For token authentication, in the Token field, provide the name of the secret property that contains the authentication token to use.

    For LDAP authentication:

    1. In the LDAP Username field, provide the name of the secret property that contains the LDAP username.

    2. In the LDAP Password field, provide the name of the secret property that contains the password for the LDAP user.

    If you use chained credentials, then the database secret fields are under Database Secret Configuration.

    To provide information about the secret to retrieve:

    1. In the Secret Name field, provide the name of the secret.

    2. If the secret is in a specific namespace, then in the Namespace field, provide the namespace.

    3. If the authentication does not use the default mount path, then in the Mount Path field, provide the mount path.

    4. If the secret is part of a structured key-value pair, then in the Property Name

    When you select a secrets manager from CyberArk Central Credential Provider:

    1. In the Secret Name field, provide the name of the secret.

    2. Optionally:

      1. In the CyberArk Safe field, provide the name of the CyberArk safe that contains the secrets manager.

      2. In the CyberArk Folder field, provide the name of the folder within the safe that contains the secrets manager. If you do not specify a folder here, Structural uses the folder configured in the secrets manager. If a folder is not configured in the secrets manager, then the folder defaults to

    When you select a secrets manager from Azure Key Vault:

    1. In the Secret Name field, provide the name of the secret.

    2. If the secret is part of a structured key-value pair, then in the Property Name field, provide the property name that contains the secret value.

    To remove the secret selection entirely, and enable a value to be entered manually.

    1. Click the Using Secret link.

    2. On the Use Secret panel, click Remove.

    ConditionalGenerator

    Download a preview Privacy Report

  • Run a new sensitivity scan

  • Not Sensitive - The number of not sensitive columns in the table. Not sensitive columns are not marked as sensitive and have Passthrough as the generator. When you click the value, you navigate to Database View, filtered to display the not sensitive columns for the table.

  • Protected - The number of protected columns in the table. Protected columns have an assigned generator. A protected column can be either sensitive or not sensitive. When you click the value, you navigate to Database View, filtered to display the protected columns for the table.

  • At-Risk - The number of at-risk columns in the table. These columns are marked as sensitive, but have Passthrough as the generator. The goal is to have 0 unprotected sensitive columns. When you click the value, you navigate to Database View, filtered to display the at-risk columns for the table.

  • Privacy Status - Indicates the current protection status of the columns in the table. It provides the same view and configuration options as the protection status panels at the top of Privacy Hub.

  • The generator that is assigned to the column.

  • For a child workspace, whether the column configuration is inherited from the parent workspace. For columns that have overrides, you can reset to the parent configuration.

  • Structural does not change the sensitivity of columns that the original scan marked as sensitive.
  • It can change the sensitivity of columns that the original scan marked as not sensitive.

  • Viewing the count of detected sensitive columns that are not protected

    Viewing the protection status for each column

    At-Risk Columns

    Protected Columns

    Not Sensitive Columns

    Viewing the protection status for each table

    Information in the list

    Filtering the list

    Filtering by table name

    Filtering by schema

    Sorting the list

    Managing columns from the table list

    Viewing and configuring columns

    Navigating through columns and viewing column details

    Indicating whether a column is sensitive

    Required workspace permission: Configure column sensitivity

    Selecting and configuring a generator for the column

    Required workspace permission: Configure column generators

    Selecting the generator

    Configuring the generator

    Displaying sample data for a column

    Required workspace permission:

    • Source data: Preview source data

    • Destination data: Preview destination data

    Enabling Document View for JSON columns

    Supported only for the file connector and PostgreSQL.

    Commenting on a column

    Required license: Professional or Enterprise

    Downloading a preview Privacy Report

    Required license: Enterprise

    From workspace management view

    From Privacy Hub

    Running a new sensitivity scan on the data

    Required workspace permission: Run sensitivity scan

    Reviewing and applying recommended generators
    JSON column that uses Document View
    display details for and configure protection for each column
    Database View
    Database View
    Database View
    file connector
    JSON column that uses Document View
    options to view and configure columns
    About workspace inheritance
    Assigning and configuring generators
    Assigning and configuring generators
    Using Document View for JSON columns
    Using the Privacy Report to verify data protection
    sensitivity scan
    child workspace
    Privacy Hub
    Sensitivity Recommendations banner on Privacy Hub
    Protection status panels
    Settings view of column details panel
    Column details panel with generator selected
    Configuration options for a selected generator
    Sample data view on the column details panel
    Comment view of the column details panel
    Download menu for a workspace
    Reports and Logs menu on Privacy Hub
    Buttons at the top of Privacy Hub
    Running data generation manually
    Namespace
    field, provide the namespace.
  • If the secrets engine does not use the default mount path, then in the second Mount path field, provide the mount path.

  • field, provide the property name that contains the secret value.
    Root
    . To specify a folder path, use Root followed by the rest of the path, with each path component separated by backslashes. For example:
    Root\OS\Linux
    .

    If you do not provide these values, they fall back to the values that are configured for the secrets manager.

    Updating the secret selection

    Selecting a secret from AWS Secrets Manager

    Selecting a secret from HashiCorp Vault

    Using chained credentials

    Selecting the authentication method

    Configuring the shared authentication settings

    Configuring app role authentication

    Configuring token authentication

    Configuring LDAP authentication

    Providing the database secret

    Selecting a secret from CyberArk Central Credential Provider

    Selecting a secret from Azure Key Vault

    Removing the secret selection

    Field marked as using a secret from a secrets manager
    Fields to identify a secret from an AWS secrets manager
    Secret selection panel for a HashiCorp Vault secret
    Chained credentials configuration for a HashiCorp Vault secret
    Secret selection panel for a CyberArk Central Credential Provider secret
    Secret selection panel for an Azure Key Vault secret

    Structural license plans

    The Tonic Structural license plans are designed to accommodate organizations that are of different sizes and that have more or less complex data architectures.

    Professional license

    The Professional license is designed for larger organizations that have more complex data architectures. The organization might have a larger team that supports multiple databases.

    Users

    The Professional license allows up to 10 users. You can purchase access for unlimited users as an add-on.

    You can use single sign-on (SSO) to manage your Structural users.

    Data connectors

    With a Professional license, you can create workspaces for up to two types of data connectors. You can purchase one additional data connector type as an add-on.

    Those data connectors can be of any type except for Oracle and Db2 for LUW.

    Structural features

    A Professional license provides access to most Structural features.

    It does NOT provide access to the following features, which require an Enterprise license:

    With a Professional license, you only have access to the basic version of the Structural API.

    You cannot use the basic Structural API to perform the following API tasks, which require the advanced API:

    The Enterprise license is ideal for very large organizations that have multiple teams that support very large and complex data structures, and that might have more requirements related to scale and compliance.

    It provides full access to all Structural features.

    An Enterprise instance does not limit the number of users.

    You can use any number of any of the available data connectors.

    The Enterprise license provides exclusive access to the and data connectors.

    The following features are exclusive to the Enterprise license:

    The Enterprise license provides exclusive access to the advanced API.

    The advanced Structural API provides access to all of the available API tasks, including the following tasks that are not available in the basic API:

    The following table compares the available features for the Structural license plans.

    Feature
    Professional
    Enterprise

    Schema management settings

    On Workspace Settings view for a workspace, the Schema Changes section contains the schema management settings..

    Schema changes include:

    • Schema changes that could expose data, which if not addressed can result in data leakage. These changes include new tables and columns, and changes to data types.

    • Notifications, which Structural can handle automatically during each data generation. These include removed tables and columns.

    For more information, go to .

    Overriding the Structural seed value for a workspace
    Enabling consistency across runs or multiple databases

    Granting Viewer and Auditor access to workspaces

  • Secrets managers for database connections

  • Migration scripts for upsert

  • Granting Viewer and Auditor access to workspaces

  • Secrets managers for database connections

  • Upsert migration service

  • Unlimited

    2 data connectors

    1 additional data connector available as an add-on

    Any data connector except for Oracle or Db2 for LUW

    Unlimited number from any available data connector

    Manager, Editor

    Manager, Editor, Auditor, Viewer

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    Concurrent jobs (more than 1 worker)

    ✓

    ✓

    ✓

    ✓

    ✓

    ✓

    Structural API

    Number of users

    Structural API

    Enterprise license

    Users

    Data connectors

    Structural features

    Structural API

    Feature comparison across Structural license plans

    Custom permission sets
    Global permission set assignment
    Privacy Report
    Workspace inheritance
    Assigning table modes to tables
    Assigning generators to columns
    Oracle
    Db2 for LUW
    Custom permission sets
    Global permission set assignment
    Privacy Report
    Workspace inheritance
    Assigning table modes to tables
    Assigning generators to columns
    Managing generator presets
    Managing custom sensitivity rules

    10

    Unlimited users available as an add-on

    On the Workspace Settings view, under Block Data Generation on Schema Changes, select how Structural responds when there are unaddressed changes to the database schema.

    The options are:

    • Do Not Block - With this option, schema changes never block data generation. When you select this option, then Structural automatically handles notifications during data generation.

      The Automatically apply generators toggle determines how Structural responds to changes that could expose data.

      • When this is enabled, then when Structural detects schema changes, it automatically applies generators to the affected columns.

      • When it is disabled, Structural does not change the column configuration.

    • Block On Changes That Could Expose Data - Indicates to only block data generation if there are schema changes that might expose data, such as new columns. Structural automatically handles notifications during data generation. For this option, Structural does not block data generation for schema changes on truncated tables.

    • Block On All Changes - For this option, if there are any unaddressed schema changes at all, either sensitive changes or notifications, then data generation fails.

    When Structural automatically applies generators to columns affected by a schema change:

    • For new columns or document fields that are detected as sensitive, Structural applies the recommended generator for the sensitivity type.

    • For new columns or document fields that are not sensitive, Structural applies an appropriate generator based on the data type.

    • For changes to column data types, nullability, or uniqueness, Structural applies a new recommended generator or an appropriate generator for the data type.

    Here is a summary of how Structural determines the generator to apply to columns that are not sensitive and do not match an existing sensitivity type or custom sensitivity rule:

    Data type
    Generator selection

    Integer

    If unique or a primary key, . Otherwise .

    Boolean

    Datetime

    By default, every time you load a workspace, Structural queries the source database to retrieve the schema.

    You can instead configure the workspace to cache the schema. Structural then updates the cache at a regular interval, and whenever a change to the workspace triggers a schema cache update.

    You can also trigger a cache update manually.

    By default, the schema cache is only used by calls from within Structural. To enable an external API request to use the cached schema, add the query parameter useSchemaCache=true to the request.

    In the application, each update to the schema cache is represented by a schema retrieval job. Schema retrieval jobs are short-lived, and run on the Structural web server. You can view the schema retrieval jobs from the workspace Jobs view.

    Note that the schema cache does not include the schema for JSON columns that use Document View. Those schemas are detected by a different scan.

    To enable and configure the caching:

    1. On the Workspace Settings view, toggle Cache source schema for faster loading to the on position.

    2. Under Schema Freshness, configure the maximum length of time between schema retrievals.

      1. In the field, provide the value.

      2. From the dropdown list, select the unit of time. You can configure the length of time in minutes, hours, or days.

      If the cached schema is older than that length of time, then the next time the application loads, it queries the source database for the current schema. The default value is 6 hours. Note that for some data connectors, schema retrievals run automatically in the background. This setting does not affect the frequency of those schema retrievals. For example, a schema retrieval runs automatically in the background every 2 hours. If you set the schema freshness to 6 hours, the background retrieval still runs every 2 hours. However, if you set the schema freshness to 1 hour, then schema retrieval occurs no more than 1 hour after the previous schema retrieval.

    3. You can optionally enable diagnostic logging for the schema retrieval. Diagnostic logging adds additional diagnostic errors to help with troubleshooting. Note that this additional information might contain sensitive information such as schema identifiers. To enable diagnostic logging:

      1. Click Show advanced options.

      2. Toggle Enable diagnostic logging to the on position.

    Responding to schema changes

    Selecting the handling option

    Viewing and resolving schema changes
    Schema management settings for a workspace

    How Structural selects generators to apply automatically

    Indicating whether to cache the source schema

    Schema caching is not available for document-based databases (MongoDB).

    About the schema cache

    Enabling and configuring schema caching

    Data connectors
    Workspace permission sets (built-in)
    Create workspaces
    View Privacy Hub
    Run sensitivity scans
    Assign table modes
    Assign generators to columns
    Run data generation
    View job details
    Schema change monitoring (conflicting changes)
    View the Version History
    Schema change monitoring (non-conflicting changes)
    Single sign-on (SSO)
    Commenting and notifications
    Structural data encryption
    Custom value processors
    Custom sensitivity rules
    Generator presets
    Virtual foreign keys
    Subsetting
    Upsert
    Post-job scripts
    Webhooks
    Workspace inheritance
    Secrets managers for database connections
    Privacy Report
    Custom permission sets
    Basic
    Advanced
    Managing generator presets

    Either , , or

    Text

    If unique or a primary key, either , , or .

    Otherwise either or .

    Continuous

    If unique or a primary key, .

    Otherwise .

    Network

    MAC address

    UUID

    If unique or a primary key, .

    Otherwise .

    JSON

    If Struct format, .

    Otherwise .

    XML

    Either or

    Other

    If HStore format,

    User-defined

    Integer Key
    Random Integer
    Random Boolean
    Timestamp Shift
    Date Truncation
    Random Timestamp
    Aphanumeric String Key
    ASCII Key
    Numeric String Key
    Character Scramble
    Categorical
    Integer Key
    Random Double
    IP Address
    MAC Address
    UUID Key
    Random UUID
    Struct Mask
    JSON Mask
    XML Mask
    HTML Mask
    HStore Mask
    Categorical
    Indicating whether a column is sensitive
    Selecting the table mode

    Table View

    Table View displays source or preview data for a single table. For a file connector workspace, each table corresponds to a file group.

    Required workspace permission:

    • Source data: Preview source data

    • Destination data: Preview destination data

    To display Table View:

    • On the workspace management view, click Table View.

    • On Workspaces view, from the dropdown menu in the Name column, select Table View.

    • From Database View, either click the arrow icon for the table, or click a row in the table.

    From Table View, you can view and update the table and column configuration.

    When you display Table View from Database View, it displays the data for the selected table.

    When you display Table View from the workspace management view or Workspaces view, it displays the most recently displayed table.

    If Table View was never displayed before, then it displays the first table in the workspace.

    To change the selected table, from the Table dropdown list, select the table to view.

    To change the table mode that is assigned to the table:

    1. Click the current table mode.

    2. On the table mode panel, from the table mode dropdown list, select the new table mode.

    When you change the table mode, Tonic Structural updates the preview data as needed. For example, if you change the table mode to Truncate, then the preview data is empty.

    For a , the table mode selection panel indicates whether the selected table mode is inherited from the parent workspace.

    If the child workspace currently overrides the parent workspace configuration, then to reset the table mode to the table that is assigned in the parent workspace, click Reset.

    The Model section of Table View displays the configured generators for the table columns.

    The header for each Model entry is the column name.

    Linked columns share an entry. The heading is a comma-separated list of the linked columns.

    Each entry contains the following information:

    • The column and generator, in the format Column Name >> Generator Name. For example, First_Name >> Name indicates that the First_Name column has the Name generator applied. For linked columns, there is a Column Name >> Generator Name entry for each column.

    • The selected configuration options for the generator.

    For a , each Model entry indicates whether the configuration overrides the parent configuration. For configurations that override the parent, to remove the overrides and restore the inheritance, click Reset.

    The Model entry also indicates when is enabled for the column.

    To remove the generator from a column, click the delete icon.

    The Preview toggle at the top right of Table View allows you to choose whether to display original source data or the transformed data. You can switch back and forth to understand exactly how Structural transforms the data based on the table and column configuration.

    By default, the Preview toggle is in the on position, and the displayed data reflects the selected table mode and the assigned generators. For tables that use Truncate mode, the preview data is empty. Truncated tables do not have data in the destination database.

    To display the original source data, toggle Preview to the off position.

    Note that for , you cannot preview the destination data from Table View. You must preview the data from Document View.

    You can provide a query to filter the source data. The query is always against the source data, not the preview data, regardless of whether the Preview toggle is off or on.

    For example, you configure a first name field to use the Name generator and enable consistency. You can then query the source data for a specific first name value to check that the preview data uses the same destination value for all of those records.

    To apply a query to the source data:

    1. Click the query filter icon, located between the table name and the table mode.

    2. On the Table Filter dialog, provide the WHERE clause for the query.

    3. To apply the query, click Apply.

    To clear an applied query, on the Table Filter dialog, click Clear.

    If no filter is applied, then the query filter icon has a white background.

    If a valid filter is applied, then the query filter icon has a gray background.

    If the provided WHERE clause is not valid, then the query filter icon has a red background.

    When the width of the table is more than 1.5 times the visible display area, then the Jump to column option displays.

    To bring a specific column into view:

    1. At the top right of Table View, click Jump to column. The list of table columns is displayed.

    2. To filter the list, in the filter field, begin to type text that is in the column name.

    1. Click the column. Structural scrolls the columns to make the selected column visible.

    In addition to the column name, the column heading provides details about the column type and protection status. It also provides access to change the column configuration.

    The column heading indicates when a column is either a primary key or a foreign key.

    The column heading indicates the column protection status:

    • At risk columns are sensitive and do not have an assigned generator.

    • Protected columns have an assigned generator.

    • Not sensitive columns are not sensitive and do not have an assigned generator.

    The sensitivity confidence indicator indicates the confidence in the detection.

    For sensitive columns that Structural detected, the confidence level can be high, medium, or low.

    For custom sensitivity rule matches or columns that you manually marked as sensitive, the confidence level is full confidence.

    For more information about how Structural identifies values and assigns the confidence level, go to .

    The column heading displays the type of data that the column contains.

    In a , when a column overrides the parent configuration, an Overriding label displays in the column heading.

    To filter Table View to only display columns with overrides, toggle Show Overrides Only to the on position.

    When a sensitivity scan identifies a column, Structural recommends a generator for the column. For example, when the sensitivity scan identifies a column as a first name, Structural recommends the Name generator configured to generate a first name value.

    For unprotected columns that have a recommended generator, the column heading displays the available recommendation icon.

    When you click the dropdown, the column configuration panel includes the following information:

    • The sensitivity confidence level

    • The recommended generator

    • Sample source and destination values based on the recommended generator

    From the panel, you can choose whether to assign or ignore the recommended generator for that type.

    • To assign the recommended generator, click Apply.

    • To ignore the recommendation, click Ignore. Structural clears the recommendation.

    To assign a generator to a column that does not have an assigned generator, or to change the current configuration, click the dropdown in the column heading.

    On the generator configuration panel, from the generator type dropdown list, select the generator to assign to the column.

    Structural displays the available configuration options for the selected generator. For details about the configuration options for each generator, go to the .

    To remove the selected generator or generator preset, and reset the generator to Passthrough, click Reset, then click Reset to Passthrough.

    For more information about selecting and configuring generators and generator presets, go to .

    On the column configuration panel, the Sensitive Data toggle indicates whether the column is marked as sensitive. The initial configuration is based on the sensitivity scan.

    • To mark a column as sensitive, toggle the setting to the on position.

    • To mark a column as not sensitive, toggle the setting to the off position.

    In a , you cannot configure whether a column is sensitive. A child workspace always inherits the sensitivity designation from its parent workspace.

    When you copy a workspace, Structural performs a new sensitivity scan on the copy. It does not copy the sensitivity designations from the original workspace.

    For a JSON column, instead of assigning a generator, you can enable Document View.

    From Document View, you can view the JSON schema structure and assign generators to individual JSON fields. For more information, go to .

    To enable Document View, on the column configuration panel, toggle Use Document View to the on position. Note that if you have , or enabled , then the Use Document View toggle is in the advanced options.

    When Document View is enabled, the generator dropdown is replaced with the Open in Document View option.

    To close the dialog, click Close.
    If you do not have either of these permissions, then you cannot display Table View.

    Selecting and configuring tables

    Selecting the table to view

    Selecting the table mode

    Required workspace permission: Assign table modes

    Viewing the generator configuration summary

    Changing the column data display

    Toggling between source and preview data

    Using a query to filter the source data

    Navigating to a specific column

    Information in the column headings

    Primary and foreign key indicators

    Protection status

    Sensitivity confidence

    Column data type

    Child workspace overrides

    Required license: Enterprise

    Configuring a column

    Applying or ignoring a recommended generator

    Required workspace permission: Configure column generators

    Changing the column generator configuration

    Required workspace permission: Configure column generators

    Indicating whether a column is sensitive

    Required workspace permission: Configure column sensitivity

    Enabling Document View for JSON columns

    Supported only for the file connector and PostgreSQL.

    child workspace
    child workspace
    Structural data encryption
    JSON columns that use Document View
    child workspace
    Generator reference
    Assigning and configuring generators
    child workspace
    Using Document View for JSON columns
    custom value processors
    Structural data encryption
    Table View with highlighted sections
    Table Mode selection on Table View
    Table mode configuration that overrides the parent workspace
    Model section of Table View
    Model entry for linked columns
    Model entry for a configuration that overrides the parent workspace
    Table Filter dialog for Table View
    Query filter icon when no query is applied
    Query filter icon when a query is applied
    Query filter icon when a bad query is applied
    Jump to column list with column name filter text
    Column headings for primary and foreign keys
    Protection status indicators in the column headings
    Sensitivity confidence level indicator
    Data type information in the column headings
    Table View column with a configuration override
    Table View with Show Overrides Only enabled
    Table View column heading with the recommended generator icon
    Recommended generator panel for a column on Table View
    Generator dropdown list for a Table View column
    How Structural identifies sensitive values

    Workspace configuration settings

    The workspace settings for a new workspace (New Workspace view) or edited workspace (Workspace Settings tab) provide information about the workspace and its data.

    Identification and worker zone

    Settings to identify the workspace and to route the workspace jobs to a specific set of Structural workers.

    Data connection settings

    Select the connector type. Connect to source and destination data.

    Use secrets managers Configure and select secrets managers secrets to use for authentication.

    Schema management settings

    Block data generation on schema changes.

    Enable and configure upsert

    Add new destination records and update changed destination records. Ignore other unchanged destination records.

    Write output to a container repository

    Use the data generation output to populate a container data volume.

    Advanced workspace overrides

    Workspace-specific settings for cross-run consistency and data generation performance.

    Enabling and configuring upsert

    Not compatible with writing output to a container repository.

    By default, Tonic Structural data generation replaces the existing destination database with the transformed data from the current job.

    Upsert adds and updates rows in the destination database, but keeps all of the other existing rows intact. For example, you might have a standard set of test records that you do not want to replace every time you generate data in Structural.

    If you enable upsert, then you cannot write the destination data to a container repository. You must write the data to a database server.

    Upsert is currently only supported for the following data connectors:

    • MySQL

    • Oracle

    • PostgreSQL

    • SQL Server

    For an overview of upsert, you can also view the .

    When upsert is enabled, the data generation job writes the generated data to an intermediate database. Structural then runs the upsert job to write the new and updated records to the destination database.

    The destination database must already exist. Structural cannot run an upsert job to an empty destination database.

    The upsert job adds and updates records based on the primary keys.

    • If the primary key for a record already exists in the destination database, the upsert job updates the record.

    • If the primary key for a record does not exist in the destination database, the upsert job inserts a new row.

    To only update or insert records that Structural creates based on source records, and ignore other records that are already in the destination database, ensure that the primary keys for each set of records operate on different ranges. For example, allocate the integer range 1-1000 for existing destination database records that you add manually. Then ensure that the source database records, and by extension the records that Structural creates during data generation, use a different range.

    Also note that when upsert is enabled, the Truncate does not actually truncate the destination table. Instead, it works more like Preserve Destination table mode, which preserves existing records in the destination table.

    To enable upsert, in the Upsert section of the workspace details, toggle Enable Upsert to the on position.

    When you enable upsert for a workspace, you are prompted to configure the upsert processing and provide the connection details for the intermediate database.

    When you enable upsert, Structural displays the following settings to configure the upsert process.

    The intermediate database must have the same schema as the destination database. If the schemas do not match, then the upsert process fails.

    To ensure that schema changes are automatically reflected in the intermediate database, you can connect the workspace to your own database migration script or tool. Structural then runs the migration script or tool whenever you run upsert data generation.

    When you start an upsert data generation job:

    1. If migration is enabled, Structural calls the endpoint to start the migration.

    2. Structural cannot start the upsert data generation until the migration completes successfully. It regularly calls the status check endpoint to check whether the migration is complete.

    3. When the migration is complete, Structural starts the upsert data generation.

    Required. Structural calls this endpoint to start the migration process specified by the provided URL.

    The request includes:

    • Any custom parameter values that you add.

    • The connection information for the intermediate database.

    The request uses the following format:

    The response contains the identifier of the migration task.

    The response uses the following format:

    Required. Structural calls this endpoint to check the current status of the migration process.

    The request includes the task identifier that was returned when the migration process started. The request URL must be able to pass the request identifier as either a path or a query parameter.

    The response provides the current status of the migration task. The possible status values are:

    • Unknown

    • Queued

    • Running

    The response uses the following format:

    Optional. Structural calls this endpoint to retrieve the log entries for the migration process. It adds the migration logs to the upsert logs.

    The request includes the task identifier that was returned when the migration process started. The request URL must be able to pass the request identifier as either a path or a query parameter

    The response body of the request should be 'text/plain'. It contains the raw logs.

    Optional. Structural calls this endpoint to cancel the migration process.

    The request includes the task identifier that was returned when the migration process started. The request URL must be able to pass the request identifier as either a path or query parameter.

    To enable the migration process, toggle Enable Migration Service to the on position.

    When you enable the migration process, you must configure the POST Start Schema Changes and GET Status of Schema Change endpoints.

    You can optionally configure the GET Schema Change Logs and DELETE Cancel Schema Changes endpoints.

    To configure the endpoints:

    1. To configure the POST Start Schema Changes endpoint:

      1. In the URL field, provide the URL of the migration script.

      2. Optionally, in the Parameters field, provide any additional parameter values that your migration scripts need.

    When you enable upsert, you must provide the connection information for the intermediate database.

    For details, go to the workspace configuration information for the .

    During upsert data generation, when Structural finds inconsistencies between the source and destination database schemas:

    • Where possible, Structural attempts to address the issue so that the data generation can succeed.

    • Structural does not change the schema of the destination database.

    • For constraint-related schema issues, Structural only attempts to address the issues if Warn on Mismatched Constraints is enabled for the workspace. If the setting is turned off, then the job fails.

    Here are some common schema issues that can occur, and how Structural responds to them.

    In this case, a column that is present in the source schema is not present in the destination schema.

    For example, a new column is added to a production source table, but is not in the schema of the de-identified destination database that is used for testing.

    When this occurs, Structural ignores the column. It does not add the column to the destination schema.

    Structural adds a warning to the job logs.

    In this case, a column that is present in the destination schema is not present in the source schema.

    For example, a developer adds a column to the de-identified destination database so that they can test a new feature. The new feature is not yet released, so the source production data doesn't include the column.

    When this occurs:

    1. If the destination column is nullable, then Structural sets the value to NULL.

    2. If the destination column is not nullable, but the column has a default value, then Structural sets the destination value to the default.

    3. If the non-nullable destination column does not have a default value, then Structural attempts to set a value based on the column data type. For example, Structural might set an integer column to 0, or a varchar column to an empty string.

    4. If Structural is unable to set a value, then the data generation fails and Structural returns an error.

    In this case, the same column has different data types in the source and destination schemas.

    For example, a column might be a string in the source schema and a timestamp in the destination schema.

    When this occurs, for each record:

    1. If possible, Structural converts the values. For example, the source column is a string and contains datetime values. The generator also produces datetime values. In that case, Structural should be able to populate a datetime destination column.

    2. If it cannot convert the value, and the column is nullable, then Structural sets the destination column value to NULL.

    3. If it cannot convert the value, and the column is not nullable, then the record is excluded from the upsert.

    For each of these actions, Structural also adds warnings to the job logs.

    If Structural cannot perform any of those actions to work around the issue, then the data generation fails and Structural returns an error.

    In this case, a constraint on a source column is not present in the destination schema.

    For example, a column is required in the source schema but optional in the destination schema.

    If Warn On Mismatched Constraints is enabled for the workspace, then Structural does not have to make any changes to the data. It populates the destination column correctly.

    Structural also adds a warning to the job logs.

    If Warn On Mismatched Constraints is turned off, then the job fails.

    In this case, a constraint on a destination column is not present in the source schema.

    For example, a column has no constraints in the source schema, but has a uniqueness constraint in the destination schema.

    When this occurs, if Warn on Mismatched Constraints is enabled for the workspace, Structural removes any records that fail the constraint. For example, for a uniqueness constraint, Structural removes duplicate records.

    Structural also adds warnings to the job logs.

    If Warn on Mismatched Constraints is turned off, then the job fails.

    In this case, a table in the source schema is not present in the destination schema.

    For example, a new table is added to a production source table, but is not yet in the schema of the de-identified destination database that is used for testing.

    When this occurs, Structural ignores the table. It does not add the table to the destination schema.

    Structural also adds warnings to the job logs.

    In this case, a table in the destination schema is not present in the source schema.

    For example, a developer adds a table to the de-identified destination database so that they can test a new feature. Because the new feature is not yet released, the source production data doesn't include the table.

    When this occurs, Structural ignores the table. It does not attempt to populate the destination table.

    Structural also adds warnings to the job logs.

    Structural cannot detect that a table is renamed.

    From Structural's perspective, the original table is removed, and the table with the new name is added.

    For example, a source and destination schema both contain a table called Users.

    In the source database, the Users table is renamed to People.

    Structural would detect the following schema issues:

    • The source schema contains a People table that is not in the destination schema. For information about how Structural addresses this, go to .

    • The destination schema contains a Users table that is not in the source schema. For information about how Structural addresses this, go to .

    Canceled

  • Completed

  • Failed

  • To configure the GET Status of Schema Change endpoint, in the URL field, provide the URL for the status check.

    The URL must include an {id} placeholder. This is used to pass the identifier that is returned from the Start Schema Changes endpoint.

  • To configure the GET Schema Change Logs endpoint, in the URL field, provide the URL to use to retrieve the logs. The URL must include an {id} placeholder. This is used to pass the identifier that is returned from the Start Schema Changes endpoint.

  • To configure the DELETE Cancel Schema Changes endpoint, in the URL field, provide the URL to use for the cancellation. The URL must include an {id} placeholder. This is used to pass the identifier that is returned from the Start Schema Changes endpoint.

  • Disable Triggers

    Indicates whether to disable any user-defined triggers before the upsert job runs. This prevents duplicate rows from being added to the destination database. By default, this is enabled.

    Automatically Start Upsert After Successful Data Generation

    Indicates whether to immediately run the upsert job after the initial data data generation to the intermediate database. By default, this is enabled. If you turn this off, then after the initial data generation, you must start the upsert job manually. For more information, go to .

    Persist Conflicting Data Tables

    When an upsert job cannot process rows with unique constraint conflicts, as well as rows that have foreign keys to those rows, this setting indicates whether to preserve the temporary tables that contain those rows. By default, this is disabled. Structural only keeps the applicable temporary tables from the most recent upsert job.

    { 
      "parameters": {/* user supplied parameters */ },
      "databaseConnectionDetails": {
            "server": "rds.amazon.com",
            "port": "54321",
            "username": "user",
            "password": "password",
            "databaseName": "tonic_upsert",
            "schemaName": "<Oracle schema to use>",
            "sslEnabled": true,
            "trustServerCertificate": false
      }
    }
    { "id": "<unique-string-identifier>" }
    {
      "id": "a0c5c4c3-a593-4daa-a935-53c45ec255ea",
      "status": "Completed",
      "errors": []
    }

    About the upsert process

    Enabling upsert

    Configuring upsert processing

    Connecting to migration scripts for schema changes

    Required license: Enterprise

    How upsert works with the migration process

    POST Start Schema Changes endpoint

    GET Status of Schema Change endpoint

    GET Schema Change Logs endpoint

    DELETE Cancel Schema Changes endpoint

    Enabling and configuring the migration process

    Connecting to the intermediate database

    How Structural responds to inconsistencies in the source and destination schemas

    Source column is not in the destination schema

    Destination column is not in the source schema

    Source and destination columns have different data types

    Source constraint is not in the destination schema

    Destination constraint is not in the source schema

    Source table is not in the destination schema

    Destination table is not in the source schema

    Source or destination table is renamed

    video tutorial
    table mode
    data connector
    Data generation process with upsert
    Upsert data generation process with migration
    Source column missing from destination schema
    Destination column is missing from source schema
    Flow to address a column missing from the source schema
    Data type mismatch between the source and destination columns
    Flow to address a data type mismatch
    Constraint in source schema is not in the destination schema
    Constraint in destination schema is not in the destination schema
    Source table column is not in the destination schema
    Destination table is not in the source schema
    Source table is not in the destination schema
    Destination table is not in the source schema

    Warn on Mismatched Constraints

    Indicates whether to treat mismatched foreign key and unique constraints between the source and destination databases as warnings instead of errors, so that the upsert job does not fail. By default, this is disabled.

    Starting an upsert job based on the most recent data generation

    Generator summary

    The following table summarizes the available generators. The table includes generator characteristics that you might take into account when you select the generator to use for a column.

    Generator hints and tips also provides some suggestions for generators to use for specific use cases.

    Information in the table

    The generator summary includes the following columns:

    • Generator - The name of the generator, linked to the entry in the .

    • Description - An overview description of the generator.

    Generator
    Description
    Supported features

    Supported features - Includes the following information:

    • The that the generator supports

    • Whether the generator is a or a

    • The generator

    API:

    Generates replacement values for U.S. mailing addresses.

    You select the address component or format for the replacement values.

    For example, the column might only contain a street address or a postal code, or it might contain a full address.

    Consistency - Self and other Linkable

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    API:

    generator reference

    Identifies the algebraic relationship between 3 or more numeric values, including at least one non-integer.

    Based on the relationship, generates new values to match. If there is no relationship, uses the Categorical generator.

    Linkable - linking is required Privacy ranking: 3

    Alphanumeric String Key API: AlphaNumericPkGenerator

    Generates unique alphanumeric strings of the same length as the input.

    For example, for the origin value ABC123, the output value is a six-character alphanumeric string such as D24N05.

    Consistency - Self only

    Primary key generator

    Unique columns allowed

    Format-preserving encryption (FPE) Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Array Character Scramble API: ArrayTextMaskGenerator

    Within an array, replaces letters with random other letters, and numbers with random other numbers.

    Preserves punctuation and whitespace.

    Consistency - Self only Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Array JSON Mask API: ArrayJsonMaskGenerator

    Used to transform array values in JSON.

    To identify values to transform, you provide a list of JSONPaths.

    For each JSONPath, you assign a sub-generator to apply to matching values.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    Array Regex Mask API: ArrayRegexMaskGenerator

    Used to transform values in an array.

    To identify values to transform, you provide a regular expression.

    For each capture group in an expression, you assign a sub-generator to apply to matching values.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    ASCII Key API: AsciiPkGenerator

    Generates unique alpha-numeric strings based on any printable ASCII characters.

    You can optionally exclude lowercase letters from the generated values.

    The replacement value does not preserve the length of the original value.

    Consistency - Self only

    Primary key generator

    Unique columns allowed

    Format-preserving encryption (FPE) Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Business Name API: BusinessNameGenerator

    Generates a random company name-like string.

    Consistency - Self or other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    Categorical API: CategoricalGenerator

    Shuffles the original values for a column to different rows. Maintains the overall frequency of each value.

    For example, a column contains the values Small (3 times), Medium (4 times), and Large (5 times).

    In the transformed data, each value appears the same number of times, but the values are shuffled to different rows.

    Linkable

    Differential privacy is configurable Privacy ranking:

    • 2 with differential privac

    • 3 without differential privacy

    Character Scramble API: TextMaskGenerator

    Replaces letters with random other letters and numbers with random other numbers.

    Preserves punctuation, whitespace, and mathematical symbols.

    Consistency - Self only Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Character Substitution API: StringMaskGenerator

    Replaces characters with other random characters. Preserves punctuation, capitalization, and whitespace.

    A replacement character is always from within the same Unicode Block as the source character.

    A source character is always mapped to the same destination character. For example, M might always map to V.

    Always self-consistent

    Unique columns allowed Privacy ranking: 4

    Company Name (Deprecated) API: CompanyNameGenerator

    This generator is deprecated. Use the Business Name generator instead.

    Generates a random company name-like string.

    Consistency - Self or other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    Conditional API: ConditionalGenerator

    Applies different generators to rows conditionally based on the column value.

    For example, apply the Character Scramble generator for values other than Test.

    You configure a list of conditions. Each condition performs a check against the column value.

    For each condition, you assign a sub-generator to apply to matching values.

    Unique columns allowed

    Composite generator. Other feature support is based on the sub-generators.

    Privacy ranking:

    • If a fallback generator is selected, then the lower of 5 or the fallback generator.

    • 5 if no fallback generator is selected.

    Constant API: ConstantGenerator

    Uses a single specified value to replace all of the values in the column.

    The replacement value must be compatible with the column data type.

    Differential privacy

    Data-free Privacy ranking: 1

    Continuous API: GaussianGenerator

    Generates a continuous distribution to fit the underlying data.

    Can link to other columns to create multivariate distributions.

    Can also be partitioned by other columns.

    Linkable

    Differential privacy is configurable Privacy ranking:

    • 2 with differential privacy

    • 3 without differential privacy

    Cross Table Sum API: CrossTableAggregateGenerator

    Populates the column using the sum of values from a column in another table.

    To select the rows to use, uses a foreign key value that matches the primary key value for the current row.

    For example, to transform the Total_Sales column in the Customers table, from the Transactions table, use the sum of the Amount values for rows where the Customer_ID value matches the primary key value for the current customer.

    Privacy ranking: 3

    CSV Mask API: CsvMaskGenerator

    Used to mask text in a delimited format.

    Parses the text as a row where the columns are delimited by a specified character.

    For each index, you assign a sub-generator to apply to the index value.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    Custom Categorical API: CustomCategoricalGenerator

    Replaces the original column value with a value from list of values that you provide.

    Consistency - Self and other

    Linkable

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    Date Truncation API: DateTruncationGenerator

    Truncates dates or timestamps to a specific date or time component.

    For example, you might truncate a date value to the month or a timestamp to the hour. You can specify a fallback generator to use for values that the generator cannot process.

    Privacy ranking: 5

    Email API: EmailGenerator

    Scrambles characters in an email address.

    Preserves the formatting and keeps the @ and ..

    You can identify specific email domains to not scramble.

    Consistency - Self only Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Event Timestamps API: EventGenerator

    Generates timestamps that fit an event distribution.

    You can link columns to create a sequence of events across multiple columns.

    You can also partition the generator by other columns.

    Linkable Privacy ranking: 3

    File Name API: FileNameGenerator

    Scrambles characters in a file name.

    Preserves the formatting and the file extension.

    Consistency - Self only Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Find and Replace API: FindAndReplaceGenerator

    Replaces all instances of the find string with the replace string.

    For the find string, you can optionally provide a regular expression.

    Privacy ranking: 5

    Finnish Personal Identity Code

    API: FinnishPicGenerator

    Generates a valid Finnish Personal Identity Code (PIC) with the same century indicator as the original value.

    You configure the date range during which the PIC was issued.

    Consistency - Self only

    Data-free if not consistent

    Unique columns allowed

    Format-preserving encryption (FPE)

    Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    FNR API: FnrGenerator

    Transforms Norwegian national identity numbers.

    You can optionally preserve the gender and birthdate portions of the identifier values.

    Consistency - Self and other

    Unique columns allowed Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Geo API: GeoGenerator

    Used to transform columns that contain latitude and longitude values.

    Linkable

    Unique columns allowed Privacy ranking: 3

    HIPAA Address API: HipaaAddressGenerator

    Can be used to generate cities, states, zip codes, and latitude/longitude values that follow HIPAA guidelines for safe harbor.

    Consistency - Self only Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Hostname API: HostnameGenerator

    Generates random host names, based on the English language.

    Consistency - Self and other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    HStore Mask API: HStoreMaskGenerator

    Used to transform values in an HStore column in a PostgreSQL database.

    You specify a list of keys for which to transform the values.

    For each key, you assign a generator to apply to the key value.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    HTML Mask API: HtmlMaskGenerator

    Used to transform columns that contain HTML content.

    To identify the values to transform, you provide a list of path expressions.

    For each path expression, you assign a generator to apply to the matching value.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    IBAN

    API: IbanGenerator

    Generates an International Bank Account Number (IBAN).

    You determine whether to preserve the banking code and country code from the original value.

    You also determine whether to generate replacement values for invalid source values.

    Consistency - Self only

    Unique columns allowed

    Format-preserving encryption (FPE) when either:

    • Assigned to a primary key column.

    • Consistency is enabled.

    Privacy ranking -

    • 3 if not consistent

    • 4 if consistent

    Integer Key API: IntegerPkGenerator

    Generates unique integer values.

    By default, the generated values are within the range of the column’s data type.

    You can also specify a range for the generated values. The source values must be within that range.

    Consistency - Self only

    Differential privacy if not consistent

    Data-free if not consistent

    Primary key generator

    Unique columns allowed

    Format-preserving encryption (FPE) Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    International Address API: InternationalAddressGenerator

    For Canadian mailing addresses, can generate:

    • Street name

    • Postal code

    For United Kingdom (UK) mailing addresses, can generate:

    • City

    • County

    • District

    • Country

    • Postal code

    Consistency - Self only

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    IP Address API: IPAddressGenerator

    Generates a random IP address-formatted string.

    You specify the percentage of IPv4 addresses.

    The remaining addresses are IPv6.

    Consistency - Self or other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    JSON Mask API: JsonMaskGenerator

    Used to transform values in JSON columns.

    To identify values to transform, you provide a list of JSONPaths.

    For each JSONPath, you assign a sub-generator to apply to matching values.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    MAC Address API: MACAddressGenerator

    Generates a random MAC address formatted string.

    Consistency - Self only

    Differential privacy if not consistent

    Data-free if not consistent

    Format-preserving encryption (FPE) Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    Mongo ObjectId Key API: ObjectIdPkGenerator

    Generates unique MongoDB objectId values.

    Can be assigned to text columns that contain MongoDB ObjectId values.

    The column value must be 12 bytes long.

    Consistency - Self only Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Name API: NameGenerator

    Generates a random name string from a dictionary of first and last names.

    You specify the name format.

    For example, a column might contain only a first name, or a full name that is last name first.

    Consistency - Self or other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    Noise Generator API: NoiseGenerator

    Masks values in numeric columns.

    Either adds or multiplies the original value by random noise.

    Consistency - Self or other Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Null API: NullGenerator

    Replaces all of the column values with NULL values.

    Differential privacy

    Data-free

    Unique columns allowed Privacy ranking: 1

    Numeric String Key API: NumericStringPkGenerator

    Generates unique numeric strings of the same length as the input numeric string.

    Consistency - Self only

    Primary key generator

    Unique columns allowed

    Format-preserving encryption (FPE) Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Passthrough API: PassthroughGenerator

    Default generator.

    Does not perform any transformation on the source data.

    Unique columns allowed Privacy ranking: 6

    Phone API: USPhoneNumberGenerator

    Generates a random telephone number that matches the country or region and format of the input telephone number.

    For invalid telephone numbers, either replaces individual numbers or generates a valid replacement number.

    Consistency - Self only Privacy ranking: 3

    Random Boolean API: RandomBooleanGenerator

    Generates a random boolean value.

    You specify the percentage of true values. The remaining values are false.

    Differential privacy

    Data-free Privacy ranking: 1

    Random Double API: RandomDoubleGenerator

    Generates a random double number that is between the specified minimum (inclusive) and maximum (exclusive) values.

    Differential privacy

    Data-free Privacy ranking: 1

    Random Hash API: RandomStringGenerator

    Generates a random hash string.

    Differential privacy

    Data-free Privacy ranking: 1

    Random Integer API: RandomIntegerGenerator

    Returns a random integer that is between the specified minimum (inclusive) and maximum (exclusive) values.

    Differential privacy

    Data-free Privacy ranking: 1

    Random Timestamp API: RandomTimestampGenerator

    Generates random dates, times, and timestamps that fall within a specified range.

    Differential privacy

    Data-free Privacy ranking: 1

    Random UUID API: UUIDGenerator

    Generates a random new UUID string.

    Differential privacy

    Data-free

    Unique columns allowed Privacy ranking: 1

    Regex Mask API: RegexMaskGenerator

    To identify values to transform, you provide a regular expression.

    For each capture group in an expression, you assign a sub-generator to apply to matching values.

    Unique columns allowed

    Composite generator. Other feature support is based on the sub-generators. Privacy ranking: 5

    Sequential Integer API: UniqueIntegerGenerator

    Generates a column of unique integer values that start with specified value, and then increment by 1 for each processed row.

    Linkable

    Unique columns allowed Privacy ranking: 3

    Shipping Container API: ShippingContainerGenerator

    Generates values of ISO 6346 compliant shipping container codes.

    The codes are all in the freight ("U") category.

    Consistency - Self or other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    SIN API: SINGenerator

    Generates a new valid Canadian Social Insurance Number.

    Preserves the formatting from the original value.

    Consistency - Self only

    Data-free if not consistent

    Unique columns allowed

    Format-preserving encryption (FPE) Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    SSN API: SsnGenerator

    Generates a new valid United States Social Security Number.

    For numeric columns, the dashes (xxx-xx-xxxx) are always excluded.

    Otherwise, you can specify the percentage of values for which to include the dashes.

    Consistency - Self or other

    Differential privacy if not consistent

    Data-free if not consistent Privacy ranking:

    • 1 if not consistent

    • 4 if consistent

    Struct Mask API: StructMaskGenerator

    Used to transform StructFields within a StructType in Databricks data.

    To identify the StructField value to transform, you provide a path expression.

    For each path expression, you assign a sub-generator to apply to the matching values.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    Text Composition

    API: TextCompositionGenerator

    Generates a replacement text value that includes values from other columns in the same row.

    Consistent by default with the included columns.

    Privacy ranking: 5

    Timestamp Shift API: TimestampShiftGenerator

    Shifts timestamps by a random amount of a specific unit of time, within a set range.

    The range can start before the original value. You can specify a fallback generator to use for values that the generator cannot process.

    Consistency - Self or other Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Unique Email API: UniqueEmailGenerator

    Generates unique email addresses.

    Replaces the username with a randomly generated GUID, and masks the domain with a character scramble.

    Consistency - Self only

    Unique columns allowed Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    URL API: UrlGenerator

    Used to transform URLs.

    Preserves the formatting. Keeps the URL scheme and top-level domain intact.

    Unique columns allowed Privacy ranking: 3

    UUID Key API: UuidPkGenerator

    Generates UUIDs.

    Consistency - Self only

    Primary key generator

    Unique columns allowed

    Format-preserving encryption (FPE) Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    Valid Address API: ValidAddressGenerator

    Generates real address components for a selected country.

    Currently supports the United States, Brazil, and Mexico.

    Consistency - Self and other

    Linkable

    Privacy ranking:

    • 3 if not consistent

    • 4 if consistent

    XML Mask API: XmlMaskGenerator

    Used to transform values in XML columns.

    To identify the values to transform, you provide XPaths.

    For each XPath, you assign a sub-generator to apply to the matching values.

    Composite generator. Feature support is based on the sub-generators. Privacy ranking: 5

    generator characteristics
    composite generator
    primary key generator
    privacy ranking
    Address
    AddressGenerator
    Algebraic
    AlgebraicGenerator

    Connection testing

    When you provide connection details for a database server, Structural provides a Test Connection button to test the connection. We strongly recommend that you test your database connections.

    Structural also runs connection tests when you open an existing workspace and when you start data generation.

    Structural is currently migrating its data connectors from the legacy connection test to a newer suite of connection tests.

    Legacy connection test

    Used by: Connectors other than file connector, MySQL, Oracle, PostgreSQL, Snowflake, SQL Server

    The legacy connection test verifies whether Structural is able to connect to the database server.

    For some data connectors, the legacy connection test also checks whether the database user has sufficient permissions to perform the required Structural tasks.

    When you run the legacy connection test, Structural indicates whether the test was successful.

    New connection test suite

    Used by: File connector, MySQL, Oracle, PostgreSQL, Snowflake, SQL Server

    For the new connection test suite, the exact tests vary based on the data connector and the connection role, such as source or destination.

    At a high level, the new tests can include:

    • Configuration - Is the connection configuration valid?

    • Connectivity - Can Structural connect to the server?

    • Schema queries - Can Structural retrieve the data schema?

    • Data queries - Can Structural retrieve the data?

    When you run the new connection test suite, Structural:

    • Lists the tests that it ran.

    • Indicates whether each test was successful.

    • For failed tests, the results:

      • Indicate whether the failure is fatal, a warning, or informational. Fatal issues must be resolved. Other types of failures might not block functionality, but might have unexpected side effects.

    To configure the timeouts for the connection tests, set the following . You can set these settings from the Environment Settings tab on Structural Settings.

    • TONIC_TEST_CONNECTION_TIMEOUT_IN_SECONDS - Used by both the legacy and the new connection tests. The number of seconds before a general connectivity test times out. By default, the connection test times out after 15 seconds.

    • TONIC_DATASOURCE_TEST_TIMEOUT_SECONDS - Used by the new connection tests only. The number of seconds before an individual test times out. By default, the test times out after 300 seconds.

    • TONIC_DATASOURCE_TEST_SUITE_TIMEOUT_SECONDS - Used by the new connection tests only. The number of seconds before a connection test suite times out. By default, the suite of tests times out after 600 seconds.

    For the following data connectors, if Structural is able to connect to the database, then it also tests and reports on the connection speed.

    • MySQL

    • Oracle

    • PostgreSQL

    • SQL Server

    Data generation - Can Structural start and run a data generation job? Can Structural write generated data?

    When possible, provide a hint as to how to resolve the failure. You can also use the Ask AI option to ask an LLM to troubleshoot the issue and suggest next steps.

    Configuring timeouts for the connection tests

    Testing the connection speed

    environment settings

    Constant

    Uses a single value to mask all of the values in the column.

    For example, you can replace every value in a string column with the value String1. Or you can replace every value in a numeric column with the value 12345.

    Characteristics

    Consistency

    No, cannot be made consistent.

    To configure the generator, in the Constant Value field, provide the value to use.

    The value must be compatible with the field type. For example, you cannot provide a string value for an integer column.

    If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Linking

    No, cannot be linked.

    Differential privacy

    Yes

    Data-free

    Yes

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    1

    Generator ID (for the API)

    How to configure

    Structural data encryption
    ConstantGenerator
    Allowing Structural to retrieve information about the latest version

    Algebraic

    The algebraic generator identifies the algebraic relationship between three or more numeric values and generates new values to match. At least one of the values must be a non-integer.

    If a relationship cannot be found, then the generator defaults to the Categorical generator.

    This generator can be linked with other Algebraic generators.

    Characteristics

    Consistency

    No, cannot be made consistent.

    To configure the generator, from the Link To dropdown list, select the columns to link this column to. You can select other columns that are assigned the Algebraic generator.

    You must select at least three columns.

    The column values must be numeric. At least one of the columns must contain a value other than an integer.

    If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

    Linking

    Yes, can be linked.

    Differential privacy

    No

    Data-free

    No

    Allowed for primary keys

    No

    Allowed for unique columns

    No

    Uses format-preserving encryption (FPE)

    No

    Privacy ranking

    3

    Generator ID (for the API)

    How to configure

    Structural data encryption
    AlgebraicGenerator

    Alphanumeric String Key

    Generates unique alphanumeric strings of the same length as the input.

    For example, for the origin value ABC123, the output value is a six-character alphanumeric string such as D24N05.

    You can configure the generator to:

    • Preserve a value prefix. For example, if the origin value always starts with acct_, you might want to preserve that in the output value, and only mask the unique part of the identifier.

    • Preserve a specified number of characters from the end of each source value. For example, you might want the last 3 characters of each destination value to be the same as the last 3 characters of the source value.

    • Only replace numerical characters, and keep other types of characters as is.

    To configure the generator:

    1. To preserve a prefix value, in the Preserve prefix field, type the prefix string to preserve in the output values. For example, if you set Preserve prefix to acct_, then any source values that start with acct_ will also start with acct_ in the destination data.

    2. To preserve a set of characters from the end of the original value, in the Preserve suffix length field, type the the number of characters to preserve at the end of each value. For example, if you set Preserve suffix length to 3, then the destination values all have the same final 3 characters as the source values. For example, for the source value abc123efg, the destination value ends with

    efg
    .
  • To only replace the numerical characters, toggle Mask digits only to the on position.

  • Toggle the Consistency setting to indicate whether to make the generator self-consistent. By default, the generator is not consistent.

  • If is enabled, then to use it for this column, in the advanced options section, toggle Use data encryption process to the on position.

  • Consistency

    Yes, can be made self-consistent.

    Linking

    No, cannot be linked.

    Differential privacy

    No

    Characteristics

    How to configure

    Data-free

    No

    Allowed for primary keys

    Yes

    Allowed for unique columns

    Yes

    Uses format-preserving encryption (FPE)

    Yes

    Privacy ranking

    • 3 if not consistent

    • 4 if consistent

    Generator ID (for the API)

    Structural data encryption
    AlphaNumericPkGenerator

    Using Collection View

    For MongoDB, Collection View replaces Database View and Table View. From Collection View, you can view the fields in a selected collection. You can then assign a collection mode to the collection, and assign generators to fields.

    Collection View for a MongoDB workspace

    Selecting the collection to view

    From the Collection dropdown list, select the collection to view.

    You can also use the Structural Agent to navigate between views. For example, Display the companies collection.

    Assigning a collection mode to the collection

    Collection mode is the term used for table mode. The collection mode determines at the collection level how Structural uses the collection data to generate the destination database.

    Available collection modes

    By default, the collection mode is De-Identify. In this mode, Structural uses the assigned generators to transform the source database into the destination database.

    For , the only other options are Truncate and Preserve Destination.

    • Truncate means that only the collection structure is included in the destination database. The collection has no data in the destination database.

    • Preserve Destination means that Tonic does not change the data that is currently in the destination database.

    To assign the collection mode:

    1. Click the Collection Mode dropdown list.

    2. On the panel, click the current collection mode.

    3. From the drop-down list, select the mode to use.

    You can view a collection either as a hybrid document or as single documents. From the View dropdown list, select the view to use.

    The default view is Hybrid Document. For the hybrid document view, the key list reflects all of the permutations of every field from every document. For example, a field might sometimes be a datetime value and sometimes a string. Hybrid document view lists both types.

    Single Document view displays a single document at a time. You can then page through up to 100 documents. Single Document view displays the structure for each document.

    For each field, Collection View always displays:

    • The field name and type.

    • For fields that you configured as primary or foreign keys, a key icon.

    • The assigned generator.

    • An example value. For the hybrid view, you can use the magnifying glass icon to display additional example values.

    For the hybrid document view, there is also a Field Freq column. Field Freq shows the percentage of documents that contain that permutation of field and type.

    For example, a field might be Null 33% of the time and contain a numeric value 67% of the time. Or a field value might be an Int32 value 3% of the time and an Int64 value 6% of the time. The percentages apply to the first 100 documents.

    The Preview toggle at the top right of Collection View allows you to choose whether to display original source data or the transformed data. You can switch back and forth to determine exactly how Tonic Structural transforms the data based on the collection and field configuration.

    By default, the Preview toggle is in the on position, and the displayed data reflects the selected collection mode and the assigned generators. For collections that use Truncate mode, the preview data is empty. Truncated collections do not have data in the destination database.

    To display the original source data, toggle Preview to the off position.

    In the single document view, you can filter the fields by either the field name or the field value.

    In the hybrid document view, you can filter the fields based on either the field name or field properties.

    You can filter single document view to only display fields that have specific text in either the field name or the field value.

    To filter by value, toggle Search by Value to the on position.

    After you select the filter type, in the search field, type text that is in the field name or value. As you type, Structural filters the list to only include fields that contain the filter text.

    To filter hybrid view by field name, in the search field, begin to type text that is in the field name. As you type, Structural filters the list to only include fields with names that include the filter text.

    From the hybrid document view, you can filter the fields based on field properties.

    To display the Filters panel, click Filters.

    To search for a filter or a filter value, in the search field, start to type the value. The search looks for text in the individual settings.

    To add a filter, depending on the filter type, either check the checkbox or select a filter option. As you add filters, Structural applies them to the field list.

    Above the list, Structural displays tags for the selected filters.

    To clear all of the currently selected filters, click Clear All.

    The Filters panel in hybrid view includes the following fields.

    An at-risk field:

    • Is marked as sensitive

    • Is assigned the Passthrough generator.

    To only display at-risk fields, on the Filters panel, check At-Risk Field.

    When you check At-Risk Field, Structural adds the following filters under Privacy Settings:

    • Sets the sensitivity filter to Sensitive.

    • Sets the protection status filter to Not protected.

    You can filter the fields based on the field sensitivity.

    On the Filters panel, under Privacy Settings, the sensitivity filter is by default set to All, which indicates to display both sensitive and non-sensitive fields.

    • To only display sensitive fields, click Sensitive.

    • To only display non-sensitive fields, click Not sensitive.

    Note that when you check At-risk Field, Structural automatically selects Sensitive.

    You can filter the fields based on whether they have any generator other than Passthrough assigned.

    On the Filters panel, under Privacy Settings, the field protection filter is by default set to All, which indicates to display both protected and not protected fields.

    • To only display fields that have an assigned generator, click Protected.

    • To only display fields that do not have an assigned generator, click Not protected.

    Note that when you check At-Risk Field, Structural automatically selects Not protected.

    When Structural detects that a field is sensitive, it can also determine a recommended generator.

    For example, when it detects a name value, it also recommends the Name generator.

    You can filter the fields to display the fields that have recommended generators.

    On the Filters panel, under Recommended Generators, check the checkbox next to the recommended generator for which to display the fields that have that recommendation.

    You can filter the fields by the field data type. For example, you might only display columns that contain either numeric or integer values.

    To only display fields that have specific data types, on the Filters panel, under Database Data Types, check the checkbox for each data type to include.

    The list of data types only includes data types that are present in the currently displayed fields and that are compatible with other applied filters.

    To search for a specific data type, in the Filters search field, begin to type the data type.

    When the source database schema changes, you might need to update the configuration to reflect those changes. If you do not resolve the schema changes, then the data generation might fail. The data generation fails if there are unresolved conflicting changes, or if you configure Structural to always fail data generation when there are any unresolved changes.

    For more information about schema changes, go to .

    To only display fields that have unresolved schema changes, on the Filters panel, check Unresolved Schema Changes.

    For detected sensitive fields, the sensitivity type indicates the type of data that was detected. Examples of sensitivity types include First Name, Address, and Email.

    To only display fields that contain specific sensitivity types, on the Filters panel, under Sensitivity Type, check the checkbox for each sensitivity type to include.

    The list of sensitivity types only includes sensitivity types that are present in the currently displayed fields.

    To search for a specific sensitivity type, in the Filters search field, type the sensitivity type.

    When the Structural sensitivity scan identifies a value as belonging to a sensitivity type, it also determines how confident it is in that determination.

    You can filter the columns based on the confidence level.

    To only display columns that have a specific confidence level, on the Filters panel, under Sensitivity confidence, check the checkbox next to each confidence level to include.

    You can filter the column list to indicate whether to include:

    • Columns that are not primary or foreign keys.

    • Columns that are foreign keys.

    • Columns that are primary keys.

    On the Filters panel, under Field Type:

    • To display fields that are neither a primary key nor a foreign key, check Non-keyed.

    • To display fields that are primary keys, check Primary key.

    • To display fields that are foreign keys, check Foreign key.

    You can add comments to fields. For example, you might use a comment to explain why you selected a particular generator or marked a field as sensitive or not sensitive.

    Note that you can only add comments from Hybrid view. You cannot add comments from Single view.

    If a field does not have any comments, then to add a comment:

    1. Click the comment icon.

    2. In the comment field, type the comment text.

    3. Click Comment.

    When a field has existing comments, the comment icon is green. To add comments:

    1. Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user and timestamp.

    2. In the comment field, type the comment text.

    3. Click Reply.

    On the field configuration panel, the sensitivity toggle at the top right indicates whether the field is marked as sensitive.

    To mark a field as sensitive, toggle the setting to the Sensitive position.

    To mark a field as not sensitive, toggle the setting to the Not Sensitive position.

    You can also use the Structural Agent to set sensitivity. For example, Mark the occupation field as not sensitive.

    You can assign a generator to each combination of field and type. For example, depending on the document, the data type for a field might be either string or integer. You can indicate to use the Character Scramble generator when the field type is a string and the Random Integer generator when the field type is integer.

    In hybrid document view, the Null type reflects when the column value is Null. You do not assign a generator to it.

    To assign a generator:

    1. Click the generator value for the field.

    2. On the configuration panel, from the Generator Type dropdown list, select the generator.

    3. Configure the generator options. For details about the available configuration options for each generator, go to the .

    You can also use the Structural Agent to apply generators to fields. For example:

    • Apply the recommended generators to all name fields.

    • Apply the Timestamp Shift generator to the renewal_date field.

    In addition to assigning generators to individual fields, you can assign generators to generic paths. The paths use JSONPath syntax.

    For more information, go to .

    By default, Structural retrieves 100 documents. It then uses the data in these documents to populate example values in the hybrid document.

    For sparsely populated collections, where less common fields are not present in those 100 documents, Structural retrieves extra documents until it has example values for all fields. For very sparsely populated collections, this might cause the collection view to load slowly, because it must retrieve many documents.

    To disable examples for sparse collections, set the TONIC_MONGO_DISABLE_EXTRA_EXAMPLES to true. You can add this setting manually to the Environment Settings list on Structural Settings.

    When this setting is true, fields that do not have a retrieved value use a dummy default value that is based on the data type.

    Assigning the collection mode

    Required workspace permission: Assign table modes

    Selecting the type of view

    Hybrid document view

    Single document view

    Information on the field list

    Toggling between source and preview data

    Required workspace permission:

    • Source data: Preview source data

    • Destination data: Preview destination data

    Filtering collection fields

    Filtering single document view by field name or value

    Filtering hybrid view by field name

    Filtering hybrid view by field properties

    Searching for a filter

    Adding a filter

    Clearing the selected filters

    Filters panel filters

    At-risk fields

    Sensitivity

    Protection status

    Recommended generators

    Field data type

    Unresolved schema changes

    Sensitivity type

    Sensitivity confidence

    Primary or foreign keys

    Commenting on fields

    Adding a new comment

    Replying to an existing comment

    Indicating whether a field is sensitive

    Required workspace permission: Configure column sensitivity

    Assigning a generator to a field and type

    Required workspace permission: Configure column generators

    Assigning generators to fields that match JSONPath expressions

    Disabling examples for sparse collections

    MongoDB
    Viewing and resolving schema changes
    Generator reference
    Assigning generators to path expressions
    environment setting
    Hybrid Document view of Collection View
    Single Document view of Collection View
    Filter field and Search by Value toggle for single document view
    Filter field and Filters button for hybrid view
    Filters panel for hybrid view on Collection View

    Structural support for languages and international values

    Supported character sets

    Structural can work with the following character sets:

    • ASCII

    • UTF-8

    • UTF-16

    Sensitivity detection in other languages

    Structural's rule-based sensitivity detection is limited to English values.

    To detect values in other languages requires a connection to an LLM. For more information, go to .

    Structural currently supports the following generators to identify and replace government identifiers:

    • - United States Social Security Numbers.

    • - Canadian Social Insurance Number.

    • - Norwegian national identity number.

    To detect other national identifiers requires a connection to an LLM. For more information, go to .

    The Structural generator is currently based on addresses in the United States.

    The generator is limited to specific values in Canada and the United Kingdom.

    For other international address values, consider the following workarounds:

    • Use the generator, and use the LLM option to prompt the generator to create a set of realistic values. For example, Generate a list of 40 cities and towns in France.

    • Use the generator to reuse the existing values in the source data.

    The and generators support a wide range of datetime formats.

    Detection of national identifiers

    Address value replacement

    Datetime value replacement

    AI in Structural
    SSN
    SIN
    FNR
    Finnish Personal Identity Code
    AI in Structural
    Address
    International Address
    Custom Categorical
    Categorical
    Random Timestamp
    Timestamp Shift

    Getting started with the Structural free trial

    If you are a user who wants to set up an account in an existing Tonic Structural Cloud or self-hosted organization, go to Creating a new account in an existing organization.

    About the Structural free trial

    The Structural 14-day free trial allows you to explore and experiment in Structural Cloud before you decide whether to purchase Structural.

    Structural tracks and displays the amount of time remaining in your free trial. You can request a demonstration and contact support.

    When the free trial period ends, you can continue to use Structural to configure workspaces. You can no longer generate data. Contact Tonic.ai to discuss purchasing a Structural license.

    Signing up for the free trial

    To start a new free trial of Structural:

    1. Go to app.tonic.ai.

    2. Click Create Account.

    On the Create your account dialog, to create an account, either:

    • To use a corporate Google email address to create the account, click Create account using Google.

    • To create a new Structural account:

      1. Enter your email address. You cannot use a public email address for a free trial account.

    Structural sends an activation link to your email address.

    After you complete the activation, Structural displays the workspace details view to allow you to create your first workspace. For more information, go to .

    The workspace details view includes an option to create a sample workspace that uses data that we provide.

    For more information about workspace settings, go to .

    After you create the workspace, Structural runs a sensitivity scan to identify sensitive values in the workspace source data.

    The summarizes the results. Use the Agent to ask questions about Structural and to help you to apply to transform the sensitive values. You can also explore other features such as , which produces a smaller set of output data.

    When the generator configuration is complete, to create output data.

    During the trial, Structural prompts you to chat with our sales team or to schedule a demo.

    If your free trial has expired, to get an extension, you can reach out to us using either the in-app chat or an email message.

    Create and confirm a Structural password.
  • Click Create Account.

  • Creating your first workspace

    Next steps for free trial users

    Workspace configuration settings
    Workspace configuration settings
    Structural Agent
    generators
    subsetting
    run a data generation
    New Workspace view with the Create Sample Workspace option to create a workspace that uses sample data