V653 - V703

A new entry is added each week, and contains the release notes for all of the Tonic versions that were released during that week.

Releases V684 through V695 were removed from quay because of a regression that was fixed in later releases.

V692 - V703

February 3, 2023

Update to PII detection - PII container removed

As of V697, updated how Tonic performs PII detection. The name detection process now also scans for international names with origins from 103 different countries.

The PII detection process now runs from the Tonic worker instead of in a separate container.

To avoid errors when you upgrade, future versions up to V999 will have a placeholder PII container. The container is not active and is not used to run Tonic.

Before you upgrade to V1000 or later, you must remove the PII container from your instance. To remove the container:

  • For a Docker deployment, in docker-compose.yaml, remove the tonic_pii_detection section. You can see the relevant section in this pull request that removed the section from the file that we distribute.

  • For a Kubernetes deployment:

    • Remove the file tonic-pii-detection-deployment.yaml.

    • Remove the file tonic-pii-detection-service.yaml.

    • Update tonic-worker-deployment.yaml to remove the entry for TONIC_PII_DETECTION_URL.

    • Update values.sample.yaml to remove the entry for pii_detection.

You can see the updates in this pull request that made the updates to the files that we distribute.

If you do not remove the PII container before you upgrade to V1000 or later, then when you upgrade Tonic, you will encounter image pull errors.

Other updates

Improved error messaging on the Tonic UI for failed data generation jobs.

Reduced the telemetry logging for PII detection during sensitivity scans to reduce the performance impact on self-hosted instances with machines that do not have internet access.

Fixed an issue that prevented users from configuring the Random Timestamp generator Date Format when the generator was selected as a sub-generator of the JSON Mask and other composite generators.

Made a minor correction to a data generation log message.

Reduced Tonic slowdowns caused by telemetry.

MySQL

  • Fixed an issue that caused data connections to fail with the error The ConnectionString property has not been initialized.

  • Fixed an issue where chained foreign keys caused subsetting jobs to fail.

Oracle

  • Fixed an issue where long column names caused data generation to return the error ORA-00972: identifier is too long.

  • Fixed an issue where chained foreign keys caused subsetting jobs to fail.

PostgreSQL

  • Fixed an issue where chained foreign keys caused subsetting jobs to fail.

SQL Server

  • Users can now assign generators to timestamp and rowversion columns.

  • Fixed an issue where a database ID mismatch in the master and database tables prevented data generation from starting.

  • Fixed an issue where chained foreign keys caused subsetting jobs to fail.

V686 - V691

January 27, 2023

Enhancements

In a Databricks workspace, you can now choose to write all of the output tables to one of the following formats: Avro, JSON, Parquet, Delta, CSV, ORC. This setting replaces the previous option to write all of the output tables to Databricks Delta. See Connecting to the destination server.

When sharing a workspace, free trial users can now invite other users with the same corporate email domain to start their own free trial.

Other updates

Added messaging to the Tonic application about changes to the Tonic license plans. For details about the Tonic license plans, see #tonic-license-plans.

Fixed an issue with the JSON Mask generator where when users deleted a sub-generator, a different sub-generator was deleted.

Fixed cases where data generation jobs remained in the queued state indefinitely.

Fixed a performance regression that affected workspace loading.

Oracle

  • Fixed an issue where a data generation fails when there are index-organized tables.

V679 - V685

January 20, 2023

Enhancements

When creating new virtual foreign keys, you can now use the top level field name check box to select or deselect all of the fields that have that name.

Other updates

Fixed an issue where a warning did not display correctly when a subset table was configured with an invalid table mode for subsetting.

For the Custom Categorical generator, we no longer treat newlines as empty strings on numeric columns. Newlines are still treated as empty strings on string columns.

Fixed an issue where an error occurred when a column is assigned the Custom Value Processor generator.

Fixed an issue where data generation with subsetting failed with the following error: Could not load type from assembly 'Allos.Generators'.

Redesigned the generator selection dropdown to better separate the suggested generators from the other applicable generators.

Oracle

  • Improved error logging during data generation.

SQL Server

  • Fixed an issue where views that reference preserved tables (table mode is Preserve Destination) were not created.

  • Added verification to ensure that all of the required tables and objects are created in the destination database before data generation.

V673 - V678

January 13, 2023

Enhancements

Tonic now displays warnings at 30, 15, and 7 days before a Tonic license expires.

Other updates

After a one-click update of Tonic, containers for Docker Compose customers now include the version number in the name.

Addressed an issue for Docker Compose customers where Tonic did not restart properly after a one-click update.

Tonic can now download SAML IDP metadata from a URL. To configure the URL, set the environment variable TONIC_SSO_SAML_IDP_METADATA_XML_URL.

Added helper text to indicate the value format for a database server.

Added the ability to configure the SAML request issuer. To configure the issuer, set the environment variable TONIC_SSO_SAML_ENTITY_ID.

Improved error messaging for WHERE clause validation in subsetting configuration.

Moved some of the temporary files used for data generation from /tmp to /tmp/tonic.

MySQL

  • Fixed an error where data generation occasionally failed with the error "Allos.Core.Exceptions.TonicException: No databases selected to overwrite".

Oracle

  • Updated to ensure that system tables are ignored.

PostgreSQL

  • Fixed an issue with data generation for partition tables.

V668 - V672

January 6, 2023

Enhancements

New command-line tool for Tonic installation - Tonic now offers the Tonic Installation Manager (TIM), a command-line tool to deploy Tonic on either Amazon EKS or a VM. For details, see .

Other updates

Updated how Tonic performs PII detection. Added additional name values to the information that the detection process looks for.

Fixed an issue where navigating to the workspace edit page sometimes threw an error.

Fixed an issue where SSO account creation erroneously returned errors even though the account was created successfully.

Improved our generator recommendations that are based on the security scan to prevent memory issues for larger databases.

Improved error messaging when Tonic-hosted users attempt to connect to a database on a local network.

Fixed an issue where job log timestamps displayed the wrong month value.

Amazon Redshift

  • Corrected a permissions check issue that caused connection tests to fail.

MySQL

  • Upgraded MySqlConnector to version 2.2.5.

  • Fixed an issue where subsetting jobs failed with a "No database selected" error.

Oracle

  • Fixed an issue where data generation jobs did not stop when they were canceled.

  • Fixed an issue with applying the Event Timestamps generator to timestamps that have timezone values.

  • Update to prevent Tonic from running out of space during subsetting.

  • Fixed an issue where materialized views and materialized view logs are not torn down after data generation.

  • When TONIC_ORACLE_SKIP_CREATE_DB is true, we now properly truncate tables.

PostgreSQL

  • Upgraded the Npgsql library to version 7.0. This is used for both workspace data and the Tonic application database.

Snowflake

  • Corrected a permissions check issue that caused connection tests to fail.

V659 - V667

December 23, 2022

Enhancements

New endpoints and expanded API documentation for generators - For generators, the generated Swagger documentation now contains descriptions for each endpoint and for all of the model properties. You can send a request to retrieve a list of available generators. You can now update the generator configuration for a single column, instead of having to provide the configuration for an entire table. There is also a new endpoint to revert a column to Passthrough.

On the Subsetting page, you can now sort the tables based on whether they are in or out of the subset.

Other updates

Improved UI support for timeouts when loading schema information for a source database.

Updated the subsetting process to ensure that a small percentage-based target table contributes at least one row to the subset.

The environment variable TONIC_SUBSET_PARALLELISM is deprecated. Tonic now uses the environment variable TONIC_TABLE_PARALLELISM to control parallel processing for subsetting.

When a data preview or workspace loading process is no longer needed, such as when there are network issues or the user leaves the application, Tonic now attempts to cancel the process.

MySQL

  • Improved error messaging when a database connection times out.

Oracle

  • Fix to ensure that data generation fails when drop table statements fail.

PostgreSQL

  • Improved error messaging when a database connection times out.

SQL Server

  • Fixed an issue where some tables were not created in the destination database.

  • Improved error messaging when a database connection times out.

V653 - V658

December 16, 2022

Enhancements

Tonic now supports SAML SSO providers.

Other updates

Fixed an issue where containers do not start and the following error message is returned: "The directory named as part of the path /var/run/supervisor/supervisord.pid does not exist"

Corrected an issue where in some cases when the job failed because of a source connection issue, the job status was not set to Failed.

In the subsetting configuration, you can now configure a target table percentage value with up to 3 decimal places.

Resolved a potential database connection leak when canceling data generation jobs.

Improved error messaging when a deleted SSO user tries to log into Tonic.

Improved error display for subsetting WHERE clause validation.

Fixed an issue where running a query to validate a subsetting WHERE clause could cause the application to slow down.

Fixed an issue where jobs could fail when subsetting parallelism was enabled.

The Conditional generator now allows you to configure a selected sub-generator to be consistent with other columns. The selected generator must support consistency with other columns.

Databricks

  • Updated the Databricks job names to use the format tonic_workspaceName/workspaceId + jobID.

  • Improved performance for page loads.

Google BigQuery

  • Added support for the JSON data type.

MongoDB

  • Fixed a minor flickering issue on the Collection view.

MySQL

  • Improved logging of errors that produce data generation warnings.

Oracle

  • Corrected an issue where we displayed foreign keys that referenced a schema outside of the workspace.

PostgreSQL

  • Fixed an issue with handling HStore types during data generation.

  • Improved logging of errors that produce data generation warnings.

  • Improved management of partition tables.

Spark

  • Improved logging for data generation.

SQL Server

  • Fixed authentication handling for Active Directory. Tonic supports NTLM v2 authentication.

Last updated