Viewing and resolving schema changes

A database schema can evolve over time. For example, a table is added, a column is removed, or a column data type changes.
It's important that you are aware of these changes and that you update your data generation configuration to address these changes.
In some cases, if you don't update the configuration, then sensitive data might be leaked. For example, when a new column is added, by default the generator is Passthrough. If you do not assign a different generator, then the next time you generate data, the source data is copied to the destination database without being masked.
In other cases, the data generation fails if you don't update the configuration. For example, a column changes its data type from integer to string. If the column is assigned the Random Integer generator, the data generation fails.
Tonic monitors your source database to look for changes to the data schema. It alerts you to those changes, and allows you to acknowledge or resolve the changes. You can also configure your workspace so that you cannot generate data when there are unacknowledged or unresolved schema changes.
Schema changes that Tonic detects
Tonic detects the following schema changes.
Conflicting schema issues
Conflicting schema issues can cause the data generation to fail if they are not resolved.
Tonic detects the following conflicting schema issues:
Table is removed from the schema
Column is removed from the schema
Column changes data type
Column changes nullability, for columns that are assigned the NULL generator
Non-conflicting schema changes
Non-conflicting schema changes are only visible with a Professional or Enterprise license.
Non-conflicting schema changes do not cause the data generation to fail. However, to prevent leakage of sensitive data, you should address these changes before you generate data.
Tonic detects the following non-conflicting schema changes:
Table is added to the schema
Column is added to the schema
Schedule for schema change scans
When you navigate to a workspace in Tonic, Tonic always runs a scan to check for schema changes.
Tonic can also run a periodic schema change detection scan in the background.
For databases other than Databricks, Snowflake on AWS, Snowflake on Azure, and MongoDB, Tonic by default runs a background scan every two hours.
For Databricks, Snowflake on AWS, Snowflake on Azure, and MongoDB, Tonic does not run any periodic scans. The data structure for these databases makes it expensive to run them. Instead, you can enable a daily schema change detection scan.
For information on how to configure whether and when Tonic runs the periodic or daily detection scans, see Configuring schema change detection.
Configuring schema change detection
For data connectors other than Databricks, Snowflake on AWS, Snowflake on Azure, and MongoDB, you use the following environment variables to configure the periodic schema change detection. You set the variables in the web server container:
TONIC_ENABLE_QUICK_PERIODIC_SCHEMA_CHANGE_SCANS - Boolean to indicate whether to enable the periodic background schema change scan.

Default is true.
TONIC_PERIODIC_QUICK_SCHEMA_CHANGE_SCAN_INTERVAL_IN_MINUTES - If periodic background schema change detection is enabled, the number of minutes between scans.

The default value is 120, which indicates to run the schema change detection every two hours.
For Databricks, Snowflake on AWS, Snowflake on Azure, and MongoDB, use the following environment variables to enable and configure the daily schema change detection scan.
TONIC_ENABLE_DAILY_EXPENSIVE_SCHEMA_CHANGE_SCANS - Boolean to indicate whether to enable the daily schema change detection scan.

Default is false.
TONIC_DAILY_EXPENSIVE_SCHEMA_CHANGE_SCANS_HOUR - If the daily schema change detection is enabled, this sets the hour at which to run the scan.

The value is an integer between 0 and 23, where 0 is midnight and 23 is 11:00 PM.

For example, a value of 14 indicates to run the job at 2:00 PM every day. Default is 0.
Preventing data generation when there are schema changes
Conflicting schema issues always prevent data generation. By default, non-conflicting schema changes do not block data generation.
However, you can configure Tonic to always prevent data generation whenever there are any unacknowledged or unresolved schema changes.
To block data generation for any schema changes, on the Edit Workspace page, under Source Server, toggle the Block data generation if schema changes detected setting to the on position.
Workspace setting to block data generation if there are schema changes
Viewing the unaddressed schema changes
The Workspaces view provides a summary of the unaddressed schema changes for each workspace. The Schema Changes view contains the complete list.
Schema change information on the Workspaces view
On the Workspaces view, the Schema Changes column shows the number of conflicting and non-conflicting schema changes.
To display a more detailed summary of the schema change detection, hover over the column. The summary includes the timestamp of the last schema scan, and a link to the Schema Changes view.
Schema changes overview on the Workspaces view
Displaying the Schema Changes view
To display the Schema Changes view, either:
On the workspace management view, in the workspace navigation bar, click Schema Changes.
On Workspaces view, from the dropdown menu in the Name column, select Schema Changes.
Resolving conflicting schema issues
The Conflicting Schema Issues list contains the schema changes that make your current workspace configuration invalid and that you have not yet resolved.
Conflicting Schema Issues list on the Schema Changes view
An issue is resolved when either:
You resolve the issue from the Conflicting Schema Issues list.
For columns that have nullability or data type changes, you change the assigned generator in Privacy Hub, Database View, or Table View.
If there are any unresolved conflicting schema issues, then data generation is blocked. If there are no conflicting schema issues, then the Conflicting Schema Issues section is not displayed.
For parent and child workspaces, for removed tables and columns, when a child workspace overrides the parent workspace configuration for the table or column, you must resolve the change in the child workspace.
If there is a conflicting change for the removed table or column in the parent workspace configuration, then regardless of whether the configuration is inherited, you must resolve the change in the parent workspace before the change is resolved for the child workspace.
For changes to column nullability or data type, you resolve the change separately in the child and parent workspaces. Depending on the configuration, the conflict might only exist in one of the workspaces.
Information in the issues list
For each issue, the list includes:
Table name
Column name, if the change affects a specific column
Description of the schema change
For changes to columns data type or nullability, a link to Database View. The link filters Database View to display only that column.
Resolve button or Select dropdown list. For changes to column data type or nullability, the Select dropdown list allows you to either resolve the issue or update the column configuration.

For a child workspace, if the issue must be resolved in the parent workspace, the button is Go to Parent. If you do not have access to the parent workspace, then the button is disabled.
The list does not include changed or removed columns for which the assigned generator is Passthrough.
Resolving a single issue
For the following types of issues, you can only resolve the issue. Resolving the issues allows Tonic to do the required cleanup to reflect the removal. See How Tonic resolves conflicting issues.
Removed table
Removed column
For these issues, to resolve the issue, click Resolve.
For a column that changed nullability or data type, you can use the Select dropdown list to either:
Resolve the issue. See How Tonic resolves conflicting issues.
Assign a different generator to the column and then resolve the issue.
For these issues:
1.Click Select.
2.To have Tonic resolve the issue:
1.Select Reset to Passthrough.
2.On the confirmation dialog, click Resolve.
3.To select a different generator for the column:
1.Select Apply New Generator.
2.On the generator configuration panel, select and configure the generator.

For detailed configuration options for each generator, see the Generator reference.
3.When you change the generator configuration, the Mark Resolved button is enabled.

To close the panel and also resolve the issue, click Mark Resolved.
For a child workspace, if the issue must also be resolved in the parent workspace, then the button changes to Go to Parent.
Resolving all of the issues
To resolve all of the issues:
Confirmation dialog to resolve all issues
1.Click Resolve All Issues.
2.On the confirmation dialog, click Resolve All.
See How Tonic resolves conflicting issues.
For a child workspace, for issues that must also be resolved in the parent workspace, the button changes to Go to Parent.
How Tonic resolves conflicting issues
To resolve conflicting issues, other than for the columns that you assign a new generator to, Tonic takes the following actions:
Removes the configuration for the affected table or column.
For a column that has a changed data type or nullability, Tonic resets the generator to Passthrough.
Removes the links to the affected columns.

The columns that were linked to the affected columns otherwise keep their current configuration.
Dismissing non-conflicting schema changes
The Notifications list requires a Professional or Enterprise license.
For Basic license users, if you know that there are non-conflicting changes, you can run a new sensitivity scan to get the protection status of the new columns.
The Notifications list contains schema changes that do not make the current configuration invalid. These changes are new tables and new columns.
Notifications list on the Schema Changes view
If there are non-conflicting schema changes, then data generation is blocked only if you configured your workspace to block data generation for all unaddressed schema changes.
If there are no non-conflicting schema changes, then the Notifications list is not displayed.
Tonic automatically dismisses a notification when:
You assign Truncate or Preserve Destination table mode to a new table.
You assign a generator other than Passthrough to a new column.
Tonic does not automatically dismiss non-conflicting schema changes in a child workspace, even if the parent workspace configuration is updated. You always dismiss the changes separately in the parent and child workspaces.
Dismissed notifications are removed from the list. Dismissing a notification does not change your workspace configuration configuration.
Information in the notifications list
For each notification, the list includes:
Table name
Column name, for new columns
Description of the schema change
A link to Database View. The link automatically filters Database View to only display the affected table or column.
Dismiss button or Select dropdown list.

For new columns, the Select dropdown list allows you to either dismiss the notification or assign a generator to the column.
Dismissing an individual notification
For a new table, the only option is to dismiss the notification. To dismiss the notification, click Dismiss.
For a new column, you can use the Select dropdown list to either:
Dismiss the notification.
Assign a different generator to the column and then dismiss the notification.
For a new column:
1.Click Select.
2.To dismiss the notification, select Dismiss Notification.
3.To assign a generator for the column:
1.Select Apply New Generator.
2.On the generator configuration panel, select and configure the generator.

For detailed configuration options for each generator, see the Generator reference.
3.When you change the generator configuration, the Dismiss button is enabled.

To close the panel and also dismiss the notification, click Dismiss.
Dismissing all of the notifications
To dismiss all of the notifications:
Confirmation dialog to dismiss all notifications
1.Click Dismiss All Notifications.
2.On the confirmation dialog, click Dismiss All.
Rerunning the sensitivity scan to check for new sensitive data
Whenever there are schema changes, especially new tables and columns, it is important to determine whether those new tables and columns contain sensitive data.
By default, Tonic copies all rows from a table. The column generator is set to Passthrough, meaning that the source data is copied as is to the destination database.
From Privacy Hub, you can run a new sensitivity scan. You can then use the updated results to guide the table and column configuration.
Last modified 1mo ago