V556 - V601

A new entry is added each week, and contains the release notes for all of the Tonic versions that were released during that week.

V594 - V601

October 14, 2022

Updates

Improved the performance of Table View, particularly when scrolling.

Reworked data generation to better group generator errors.

On the Subsetting view:

• Improved accuracy of the row counts for small tables.

• The table list shows whether each table is in the subset and the number of pre-subset and post-subset rows even when subsetting is not enabled for data generation.

The health check for the PyML container can now use HTTP instead of HTTPS.

MongoDB

• Collection View now updates after the first scan without having to be refreshed.

MySQL

• Improved performance when loading workspaces.

Oracle

• Fix to improve query performance.

• Added permission checks when testing database connections.

PostgreSQL

• Tonic now applies the source database check constraints on the destination database after data generation is complete.

Snowflake

• Fixed an issue where data generation failed when there were tables that used Preserve Destination table mode.

V588 - V593

October 7, 2022

Updates

Reverted an update from v580 that caused slow performance when retrieving the list of tables for a workspace. In some cases, this caused Tonic to indicate that no tables were available.

Fixed an issue where custom value processors interfered with the display of configuration options for the SSN generator.

Fixed an issue where the dropdown lists to select a column generator periodically scrolled to the top of the screen.

Improved performance when navigating among Tonic views.

Improved the list of suggested sub-generators for the Conditional generator.

Improved the data generation process to prevent jobs from hanging when an error occurs.

Fixed an issue where the post-subset generation row count always returned 0.

Updated the Tonic logging framework.

Google BigQuery

• Improved handling of timestamp and datetime fields.

MongoDB

• Fixed an issue where workspaces crashed when running data generation on new collections.

MySQL

• Corrected the handling of the BIT data type.

• Improved performance when loading workspaces.

Oracle

• The Constant generator now works correctly for RAW columns.

• Stopped using source database constraints that were in the recycle bin to populate the destination database.

• Improved performance when retrieving schema information and loading workspaces.

• Added the OracleDriverAnalyzer tool to analyze Oracle performance.

PostgreSQL

• Display an error when an excluded or included schema does not exist in the source database.

• Fixed a permissions error that occurred when creating a new workspace using Azure PostgreSQL for the destination database.

Spark

• Fixed an issue with the workspace configuration view that prevented users from creating and updating Spark-based workspaces.

SQL Server

• Fixed an issue that caused a data generation error when XML indexes had dependencies on other indexes.

• Upgraded SqlClient to 5.0.0.

V583 - V587

September 30, 2022

Enhancements

Assigning generators from Schema Changes view - On Schema Changes view, new columns, changes to column data type, and changes to column nullability have a Select dropdown list that includes an option to assign a different generator to the column and then resolve the issue. See Resolving conflicting schema issues and Dismissing non-conflicting schema changes.

Complete list of blocking issues for data generation - When data generation is blocked, the generation panel now displays all of the blocking issues. This allows you to correct all of the blocking issues before you attempt to run data generation again.

Other updates

Made some visual updates to the Tonic navigation pane and the Tonic login panel.

On the Job History page, the details popover for queued jobs now points to the correct job.

In Database View, the generator list for the Applied Generators filter is now correctly alphabetized.

Fixed an issue where the dropdown arrow for the table mode selector was not always clickable.

When importing a workspace, Tonic now validates that columns do not have multiple generators assigned to them.

Updated to provide clearer error messages when there is an issue with an assigned sub-generator.

Fixed an issue where Privacy Hub sometimes did not reload after a new sensitivity scan.

The database type filter now includes all of the available Spark database types.

Corrected the link to the Tonic privacy policy.

Fixed an issue where jobs failed when multiple tables with the same name in different schemas were assigned Preserve Destination table mode.

Timestamp Shift is now the recommended generator for Date and Timestamp columns.

Corrected the display of available buttons on the Tonic application.

MongoDB

• Improved error handling for database summary queries.

• Added a warning about subsetting performance for percentage subsetting target tables on MongoDB versions before 4.4.2.

• Fixed a memory usage issue with loading Collection View when the data contains large arrays.

Oracle

• A new environment variable, TONIC_ORACLE_DATA_PUMP_PARALLELISM, allows you to choose the maximum number of threads for parallelization for Oracle Data Pump.

• Optimized queries for better performance.

Spark

• Added the ability to specify the proxy user when using Livy.

• Fixed an issue where users could change the configuration type when editing a Spark workspace.

V572 - V582

September 23, 2022

Enhancements

Subsetting results - The Subsetting view now displays the results of the most recent subsetting data generation run. See Subsetting data generation results. The information includes:

• When the job began and ended.

• The percent reduction from the original source data to the subset destination data.

• The percentage of the source data that is included in the subset destination data.

• The volume of data in the source data and the subset destination data.

Schema changes filter on Database View - On Database View, the advanced filters now include an option to only display columns that have unresolved schema changes. This filter is not combined with other filters. When you filter for unresolved schema changes, the other column filters are disabled. See Unresolved schema changes.

Larger where clause editor for subsetting - For where clause target tables, you can now display an editor with a larger text area for entering the where clause. This provides better support for longer, more complex where clauses. See Configuring a WHERE clause target table.

Other updates

Fixed an issue that prevented users from deleting more than one tag from the Edit Workspace view.

Added the ability to run Tonic workers, the Tonic web server, and Tonic notifications on Heroku.

Updates to improve handling of canceled jobs, both when users cancel jobs and when jobs fail.

Improvements to data generation memory handling and performance.

Improved the Synthesis Report for AI Synthesizer.

Fixed an issue where adding constraints to a destination database resulted in deadlocks.

Oracle

• Update to allow the JSON Mask generator to be used on user-defined types (UDTs).

PostgreSQL

• Fixed a regression that caused sequence fetching errors in v9.6 and earlier.

Spark

• Improved performance for the Continuous generator.

• Enabled partition filter validation on Hive.

SQL Server

• Added support for full text catalogs in Server Management Objects (SMOs).

• Fixed an issue to enable the correct handling of schema, table, and column names that contain single quotes.

• Made a fix to correctly display error messages.

V571 - V576

September 16, 2022

Enhancements

Workspace inheritance - For enterprise customers, the workspace inheritance function allows you to create child workspaces that automatically inherit source data and Tonic configuration from their parent workspace. Changes to the parent workspace configuration are copied to the child workspace. You can override the subsetting configuration and post-job scripts in a child workspace. For more information about workspace inheritance and how it works, see About workspace inheritance.

Links from Schema Changes to Database View - Added links from Schema Changes entries to Database View. The links automatically filter Database View to only include the affected column or table. The links only display for columns and tables that are in the source database. Removed columns or tables do not have links to Database View.

Schema filtering for PostgreSQL - For PostgreSQL workspaces, when you create or edit the workspace, you can specify a list of schemas to either include or exclude from the source database.

User profile pictures - From the User Settings view, Tonic users can now upload a user image for their account. For SSO providers that support user images, the image from the SSO is used by default. See Choosing your user image.

Zip code configuration for HIPAA Address generator - A new configuration setting for the HIPAA Address generator allows you to determine how the generator sets the zip code.

• If the setting is off, the generator uses the current process, which replaces the last two digits of the zip code with zeros. For low population areas, the zip code is all zeros.

• If the setting is on, then the generator selects a real zip code that starts with the same three digits as the original zip code. For low population areas, if a state is provided in the data, the generator selects a random zip code from that state. Otherwise it selects a random zip code from the United States.

Other updates

Minor updates to the Subsetting view. New icons for the subsetting summary and the inbound and outbound relationship counts. Added a Use subsetting toggle to indicate to use the subset configuration for data generation. This toggle is synchronized with the same toggle on the Confirm Generation panel.

Fixed an issue where Tonic logged users out between browser sessions more often than expected.

Improved performance for the JSON Mask generator.

Fix to ensure that the AI Synthesizer is canceled when a data generation job that includes AI Synthesizer is canceled.

Fixed an issue where a data generation job would hang if there was a failure.

Fix to address an issue where after a one-click update, the Tonic version unexpectedly regressed to an earlier version.

Fix to ensure that cross-table commands do not run in parallel, which could cause deadlocks in the database.

MongoDB

• Fixed an issue that prevented the Regex Mask generator from being applied to fields in arrays.

• Updated to require 3.6 as the minimum server version for MongoDB.

PostgreSQL

• Sequence values are now copied over correctly.

Spark

• Fixed a data generation error in Hive and Livy that occurred when a source table had non-lowercase column names.

V567 - V570

September 9, 2022

Enhancements

Filtering workspaces by tags - On the Workspaces view, you can filter the workspaces by the assigned tags. In the Tags column heading, click the filter icon to display the list of applied tags. Check the checkbox for each tag to include. The list is filtered to include workspaces that have at least one of the selected tags. See Filtering the workspace list.

Automatically resolving schema changes - Schema changes are now resolved or dismissed automatically when you update the table or column configuration.

• For a new table, the schema change is dismissed when you set the table mode to Truncate or Preserve Destination.

• For a new column, the schema change is dismissed when you assign a generator to the column.

• For a column that has a conflicting data type or nullability change, the schema change is resolved when you assign a different generator to the column.

Other updates

Corrected the number of rows for out-of-subset tables. No longer show 0 when out-of-subset tables are processed.

Improved error messaging when a where clause for a subset target table is invalid.

Corrected an issue where the subsetting table configuration was not handled correctly.

The job types filter on the Job History view now only shows valid types for the workspace.

Improved performance for data generation.

Improved performance for the AI Synthesizer.

Improved reporting of subsetting progress when parallelism is enabled.

MongoDB

• You can now assign the Preserve Destination mode to a collection.

• Subsetting and foreign key management is now enabled for all customers.

Spark

• For workspaces that use Livy, you can now assign the Continuous generator and Noise Generator as sub-generators for a mask generator.

• Improved performance for the Categorical generator.

• Improved performance for the HIPAA Address generator.

V563 - V566

September 2, 2022

Enhancements

You can now enable parallel processing for subsetting. The TONIC_SUBSETTING_PARALLELISM environment variable sets the number of steps to process in parallel. See Enabling parallel processing for subsetting.

Other updates

Made a couple of small improvements to the AI Synthesizer generator.

Fixed an issue where users could not change the percentage on a subsetting target table.

Improved how we sort tables for parallel processing to improve efficiency.

MongoDB

• The Categorical, Current Date, Date Truncation, HIPAA Address, and Unique Email generators now work on Mongo array fields.

• Linking for the Continuous generator now works correctly.

Spark

• Added support for Spark 2.3 on Livy.

V557 - V562

August 26, 2022

Enhancements

On Privacy Hub, the term "Unprotected Sensitive" is replaced with "At-Risk". The protection status counts exclude columns that are not included in the destination database. See Viewing the current protection status.

On Database View, added an option to filter by whether a column is included in the destination database. A new At-Risk toggle provides a shortcut to filter for columns that are included, marked as sensitive, and not assigned a generator. The Column Type filters, which filter columns based on whether they are a primary or foreign key, are changed from checkboxes to toggles. See Filtering the column list.

In the Privacy Report, added Not Included as a value for ColumnPrivacyStatus, to identify columns that are not included in the destination database. The value Protected replaces the current values Masked and Anonymized, which are moved to a new ProtectionType column. See Using the Privacy Report to verify data protection.

Other updates

Improved the display of the Database View advanced filter for smaller screens.

Fixed an issue with the generation of the API reference.

Began to log latencies each hour for source and destination databases.

MongoDB

• In Collection View, improved the display of key columns on smaller screens.

• Corrected an issue that resulted in duplicate schema change results.

• Fixed the collection scan status when a new scan is started.

• Single document view no longer reloads when you apply a generator.

• Corrected an issue where selecting a different collection to view applied updates from the previous collection.

• Fixed an issue where the Null generator could not be removed from a field.

MySQL

• Fixed an issue where the Tonic application could not load when table and row size estimates were not available.

• Increased the connection resiliency for write operations.

• Improved handling of different character sets in MySQL.

V552 - V556

August 19, 2022

When you create a sample workspace, it now includes a tag called Sample Tag.

Fixed an issue where the Update Tonic button was not displayed correctly.

Improved the user interface for activating new hosted accounts.

Improved message to notify users that the subset configuration changed since the last subsetting data generation.

The HIPAA Address generator now works correctly for US addresses.

MongoDB

• You can now link fields in MongoDB collections.

• Conditional generators can now be applied to array elements.

• The Random Timestamp generator now works correctly on datetime columns.

Oracle

• Data generation can now work without DBLink.

PostgreSQL

• Completed additional updates to support cross-schema types.