V609 - V652

A new entry is added each week, and contains the release notes for all of the Tonic versions that were released during that week.

V646 - V652

December 9, 2022

Enhancements

From the Subsetting view, you can now view the table configuration and results summary for previous subsetting data generation runs. On the Previous Runs tab, you select the run to view the details for. Note that you can only view the details for jobs that you run after you upgrade.

Other updates

Corrected an issue for instances using a Docker version before 20.10.10 that prevented Tonic processes from starting after an upgrade from version 642 or below to version 643 through 647.

Corrected an issue where estimated data generation time was logged as a negative number.

Improved the performance and resiliency of the subset WHERE clause validation.

Improved the cleanup of database connections after they are used.

Improved error message when the source database version is newer than the destination database version.

Google BigQuery

  • Fixed a user interface issue that prevented a new workspace from being created.

  • Tonic now supports post-job scripts in BigQuery workspaces. Post-job scripts run inside of transactions. They are limited to statements that are supported within transactions.

Oracle

  • The minimum supported version for the Oracle data connector is now Oracle 12c. Tonic no longer supports 11g. As a result of this change, the Oracle Helper is no longer needed. After upgrading, you should remove Oracle Helper from the list of containers to fetch.

Snowflake

  • For database connections, the server name should not include https://. Tonic now detects and removes https:// if it is entered.

Spark

  • Eliminated a vulnerability that could allow users to see the results of statistics jobs that they did not have access to.

SQL Server

  • Fixed an issue with loading tables in SQL Server 2012 and below

V639 - V645

December 2, 2022

Enhancements

Overriding table mode and generator configuration in child workspaces - The workspace inheritance feature now allows child workspaces to override the table mode and generator configuration from the parent workspace. Database View and Table View indicate when the parent configuration is overridden, and provide options to reset the configuration.

Other updates

You can now assign the SIN generator to fields that have uniqueness constraints.

The Address generator now only allows you to link columns that contain the following types of address values - City, City State, Country, Country Code, State, State Abbreviation, Zip Code, Latitude, Longitude.

Improved memory handling for subset processing of downstream tables.

On the Subsetting view, when the database returns an unrealistic value for the subset size, we replace the value with a warning message.

When using Helm to deploy Tonic via Kubernetes, integer environment variable values that are longer than 6 digits might be converted to scientific notation. To avoid this issue, Helm recommends that you enter values in quotes. Tonic can now parse scientific notation to better handle this behavior for values that are not in quotes.

Fixed an issue where linking columns that have the same name as columns in another table caused the generators to be removed from those other columns.

Made minor memory improvements throughout the Tonic application.

Corrected an issue where Tonic displayed a "subsetting in progress" message before the processing started.

Amazon Redshift

  • Fixed an issue where data generation failed when the name of one table was part of the name of another table (for example, Test and Test1).

MongoDB

  • Generators are now applied correctly to documents that are nested in arrays.

Oracle

  • Removed the permissions check for reading truncated tables.

  • Fixed an issue where DBLinks created for data generation were not removed when the generation failed or was canceled.

Snowflake

  • The Categorical generator no longer returns errors when applied to binary, time, and date columns.

  • Improved the performance for loading the user interface when the data contains a large number of foreign keys.

Spark

  • The Audit Trail entries on Privacy Hub now display the correct table and column names.

  • Corrected an issue where sub-generators for the Conditional generator were not applied correctly.

  • Improved error messages when testing the connection to a Databricks database.

V637 - V638

November 23, 2022

Updates

Improved user signup experience for the hosted version of Tonic.

Made some small styling updates to Database View. Removed the dropdown to uncheck or check all of the tables. Removed the highlighting on the table mode dropdown for tables that contain columns with assigned generators. Changed the Filter button labels to Filters.

For subsetting, verify that primary key fields are not assigned a non-primary key generator.

Amazon Redshift

  • Improved error messaging for permissions for data generation.

  • Improved performance for data generation.

MongoDB

  • Fixed an issue that caused errors when applying the Timestamp Shift generator.

Snowflake

  • Improved error messaging for permissions for data generation.

  • Improved performance for data generation.

Spark

  • Improved error messaging for unsupported versions.

SQL Server

  • Improved handling of alphanumeric sequences.

  • Case-sensitive databases now connect correctly to Tonic.

  • Corrected how indexing recommendations for subsetting are determined.

V630 - V636

November 18, 2022

Enhancements

Other updates

Tonic no longer offers DB2 as a data connector type.

On the Job History list, changed the job type "Privacy Scan" to "Sensitivity Scan".

Corrected a display issue in Privacy Hub where the column details panel extended past the bottom of the page.

For the JSON Mask generator, the path expression selection tool now works for arrays and for keys that contain spaces or special characters.

Tonic now provides a more meaningful error when Preserve Destination mode is assigned to a table in a workspace that does not have a defined destination database.

Added a message to notify users when queries used to validate a subset WHERE clause run for a long time.

Tonic now continues to record logs when a job fails.

Improved memory usage when running data generation.

MongoDB

  • Corrected a subsetting issue that occurred when multiple foreign keys in one collection referred to the same primary key in another collection.

MySQL

  • Updated the MySqlConnector driver to 2.2.0.

Oracle

  • Downgraded the Oracle drivers to 3.21.70 after encountering issues with 3.21.80.

  • Corrected an issue with the Event Timestamps generator.

PostgreSQL

  • When executing psql and pgdump commands, we properly enforce the requirement to use SSL for connections.

  • Corrected connection issues by escaping special characters in password files.

Redshift

  • Fixed an issue where Tonic failed to display an error for an invalid generator assignment.

Snowflake

  • Fixed an issue where Tonic failed to display an error for an invalid generator assignment.

  • Fixed an issue where large tables caused data generation to fail with "Error: Authentication token has expired. The user must authenticate again."

Spark

  • You can now apply the Timestamp Shift generator to string fields that contain datetime values.

V626 - V629

November 11, 2022

Enhancements

Tonic can now integrate with Keycloak for SSO authentication.

Other updates

Fixed an issue where users received the error HTTP/1.1 415 Unsupported Media Type from the /api/GenerateData/start endpoint.

Improved performance for the /api/job endpoint. As part of this update, the endpoint only returns the most recent 100 jobs.

Improved loading time for the Job History view, which now displays only the 100 most recent jobs.

Updated how Tonic determines whether to use SSL for connections.

The Add Foreign Key Relationships tab on the Foreign Keys view now expands to fill the height of the browser.

Fixed an issue where a failed cleanup step incorrectly marked a Tonic update as failed.

MongoDB

  • When performing a schema scan, Tonic now displays a progress indicator.

  • Improved subsetting performance.

  • Fixed an issue where subsetting failed because of duplicate keys.

  • Corrected how Tonic handles views.

Redshift

  • Improved memory management.

  • The Lambda role now requires the s3:ListBucket permission.

Snowflake

  • Fixed an issue where the Tonic user interface could not load when the database included foreign keys.

  • Improved memory management.

  • The Lambda role now requires the s3:ListBucket permission.

V619 - V625

November 4, 2022

Enhancements

If data generation is not blocked on all schema changes, Tonic now displays a dismissible warning when there are non-conflicting schema changes. Conflicting schema changes always block data generation.

For the GenerateData API endpoint, added an optional clientResourceId query parameter. When you provide a value, then jobs that have the specified clientResourceId run serially instead of in parallel. The check applies to all jobs across the instance, regardless of whether they belong to the same workspace.

Other changes

Fixed an issue that made it difficult to click the Cancel Job button in the Job History list.

On the workspace configuration view, the source and destination database details are now populated correctly when you refresh the page.

When granting access to a workspace, improved how we display long names and email addresses.

For tables that use Scale mode, removed Passthrough from the generator selection dropdown list. Previously, the option displayed in the dropdown list even though it couldn't be selected.

Improved performance for looking up tables and columns.

Corrected an issue where the same environment variable with different cases caused Tonic to crash.

When logging in using single sign-on (SSO), when the email address uses a different case from an existing username-password account, it now resolves to the same user.

For instances deployed using Docker Compose, Tonic now cleans up old, unused images.

Improved performance when running generators on de-identified tables.

Corrected an issue where the Event Timestamps generator produced unexpected values for linked columns.

Improved error handling for account creation.

MongoDB

  • Updated MongoDB.Driver and MongoDB.Bson to 2.18.0.

  • Subsetting jobs with foreign keys can now run on collections that have lower permissions. To enable this, set the environment variable TONIC_BYPASSDOCUMENTVALIDATION_ON_DOWNSTREAM_KEY_MERGES to false. The default value is true, which means that the destination database requires the dbAdmin role in addition to readWrite.

Oracle

  • Tonic now automatically uses the optimal table prefix for schema queries. The TONIC_ORACLE_DICTIONARY_TABLE_PREFIX environment variable is removed.

  • Improved error handling for missing permissions for table size queries.

SQL Server

  • Improved performance for table queries.

V610 - V618

October 28, 2022

Enhancements

The Subsetting view now shows for each table the percentage of data that is included in the destination database.

Other updates

The workspaces view no longer briefly flashes a message indicating that the workspace cannot be found.

Added the ability to display trace information in the log files. To enable the trace information, set the environment variable TONIC_LOG_TRACES to true.

Fixed an issue that caused an incorrect warning to display when linking columns that were assigned the Custom Categorical generator.

Fixed a data type error in AI Synthesizer for models that only contain categorical data.

For the Event Timestamps generator, Tonic now prevents the generator from being assigned to a time-only value.

Amazon Redshift

  • Added ServerCompatibilityMode to the Redshift connection string to prevent connection errors.

Databricks

  • Added support for Databricks 10.x.

MongoDB

  • Fixed an issue where Tonic generated collection names in Preserve Destination mode that exceeded the maximum length limit.

MySQL

  • The MySQL data connector now supports the ability to use the DELIMITER command in post-job scripts.

Oracle

  • Improved performance for schema queries.

  • Improved performance for databases without database links.

  • Restored missing information for loader errors.

  • Improved performance for applying indexes and constraints.

  • Added the environment variable TONIC_ORACLE_REDO_LOG_ENABLED, which by default disables recovery information writes to REDO LOG files.

  • Improved performance for retrieving tables and columns.

  • Fixed an issue with loading custom types.

  • Fixed an issue where multiple indexes on the same column caused data generation to fail. Corrects a regression from V611.

  • When fetching constraints, Tonic can now support primary key indexes that have the same name.

  • Excluded views from table queries.

PostgreSQL

  • Updated the process of loading tables to improve performance and eliminate an error indicating that PostgreSQL types could not be found.

SQL Server

  • Fixed an issue where Preserve Destination and Incremental table mode could not be used in tables with foreign key relationships.

  • Tonic now provides a more informative error when data generation fails because the database is not accessible.

  • Tonic now properly handles identifiers with single quotes.

  • Fixed an issue that caused a worker process to crash when using Azure to connect to the database.

  • Added additional logging of the available databases.

  • Improved handling of schema and table names in privacy scans.

V602 - V609

October 21, 2022

Enhancements

You can now generate DEBUG level logs for the Tonic API. To do this, set the environment variable TONIC_CONSOLE_LOG_LEVEL to DEBUG.

Tonic now supports logging for long-running queries. The environment variable TONIC_LONG_RUNNING_QUERY_LOGGING_INTERVAL provides the interval in minutes for logging queries. By default, Tonic generates a log entry for a long-running query every 10 minutes. To see this information, TONIC_CONSOLE_LOG_LEVEL must be set to DEBUG.

Other updates

Terminology change - In the Tonic documentation, we have changed the term "mask generator" to "composite generator".

Fixed an issue that prevented the use of Preserve Destination and Truncation modes on tables and collections that had names that were close to the maximum length limit.

You can now assign the Integer Key generator as a sub-generator for string values in composite generators.

For Google SSO, fixed an issue where users who did not have a group membership could not access Tonic.

Improved performance for format-preserving encryption (FPE), which is primarily used for key generators.

Improved generation performance for tables that use De-Identify table mode.

Improved error display when reading data from the source database.

Fixed an issue where values for columns that were assigned the Passthrough generator were being masked with 1s and 0s.

The User Settings page no longer displays the password change option for SSO users.

For the JSON Mask generator, fixed an issue where a large matching value caused the buttons to move off of the configuration dialog.

Fixed an issue where data connection pooling caused data generation to fail because of colliding queries.

Google BigQuery

  • Fixed an issue where source data was not loading properly into the Tonic user interface.

MongoDB

  • Fixed an issue where the generator configuration panel closed unexpectedly after making edits.

Oracle

  • Added a warning when a generator produces a value that is truncated in the destination database.

  • When Tonic retrieves table sizes, the results now include the size of the associated large objects (LOBs).

  • To help with troubleshooting, Tonic can now determine which Oracle patches were applied to a database.

  • Fixed an issue that caused errors when reading a large object array.

  • Added additional checks to the data connection tests.

  • Updated our Oracle Driver to 3.21.80.

  • Fixed an issue where primary keys could not be copied to another index.

  • Additional performance improvements.

  • Corrected how we check the current user's database privileges.

PostgreSQL

  • Added an option to not refresh materialized views. To turn off the refresh, set the environment variable TONIC_POSTGRES_REFRESH_MATERIALIZED_VIEWS to false.

  • Fixed an issue with parsing PostgreSQL schemas in Windows environments.

  • Fixed an issue where data generation failed when the data contained read-only arrays.

Snowflake

  • Fixed an issue where data generation failed when the schema definition (DDL) contained row access policies.

SQL Server

  • Added permission checks to the data connection tests.

  • Fixed an issue where Tonic processes failed.

Last updated