V899 - V951
A new entry is added each week, and contains the release notes for all of the Tonic versions that were released during that week.
Releases V899 - V901 were removed from quay because of a regression that was fixed in later releases.
V947 - V951
August 25, 2023
You can now export individual topics from the Tonic documentation to PDF files. To export a topic to PDF, click the actions menu next to the topic title, then click Export as PDF.
Fixed an issue with removing unique constraint conflicts in upsert where rows that didn’t have a conflict were excluded from the upsert process.
File connector
For Amazon S3 and Google Cloud Storage, the permission to list all buckets is no longer required. However, if that permission is not present, users must manually type in the bucket name where the file is located.
When you copy a file connector workspace, Tonic now copies the file groups to the new workspace.
V942 - V946
August 18, 2023
Upsert data generation (beta feature)
Previously, the data generation process always replaced the entire destination database. The new upsert data generation option (currently in beta) allows you to add new records and update existing records without touching any of the other records in the destination database. For example, you might have a regular set of records that you use for testing that you want to maintain.
Upsert requires a connection to an intermediate database. When you run data generation with upsert, the initial data generation writes the transformed data to the intermediate database. It replaces the intermediate database, similar to regular data generation. After the generation to the intermediate database, the upsert process identifies the records to add to or update in the destination database. It ignores other records in the destination database.
Upsert requires a Professional or Enterprise license, and is only supported for the following data connectors:
MySQL
Oracle
PostgreSQL
SQL Server
New AI-enhanced documentation search option
The Tonic documentation now provides access to Lens, an AI-based search option. Instead of searching for specific words, you can ask questions such as "How do I create a workspace?". Lens searches the documentation for the answer. It generates a response that includes links to the topics that contain the information it used.
To use the Lens search, click the search field. At the top right of the search panel, click Lens. Then ask your question.
Other updates
The Custom Categorical generator now supports consistency with other columns. Previously, the generator only supported self-consistency.
Tonic now prevents you from starting data generation for a workspace that does not have a destination database specified.
Fixed a display issue where the generator preset details panel briefly showed the occurrences for the previously selected generator preset.
Tonic now suggests the Name generator for columns that Tonic detects as containing names, when the detection is based on the sampled data in the column. By default, the Name generator uses the First Last format.
A new configuration option allows webhooks to bypass SSL certificate validation and trust the server certificate.
File connector
You cannot start data generation for a file connector workspace that has no source files specified.
The Table View data preview for file groups that contain JSON or XML files no longer displays above the real data records an extra row that contains the value 0.
Fixed an issue where generators such as the Categorical generator unexpectedly could not be used as sub-generators.
Improved error handling when a file group is incorrectly configured as having a header row.
Fixed an issue that caused data generation to fail for XML files because of missing metadata.
Fixed an issue where Database View displayed duplicate columns.
Table View no longer displays an extra data row.
MongoDB
Fixed an issue that caused unscanned collections to not display on Collection View.
The sensitivity scan no longer marks Null fields as sensitive.
PostgreSQL
Improved performance when refreshing materialized views in PostgreSQL v11, v12 and v13.
V936 - V941
August 11, 2023
On the generator configuration panel, changed the label of the Save As menu to Preset Options. The menu contains options related to configuring generator presets.
Free trial users now have access to the file connector.
Tonic now displays the error that occurs when an Algebraic generator configuration does not include any floating-point values.
For composite generators, the generator preset details panel now provides a clearer explanation that presets for composite generators must be configured from within a workspace.
Improved performance for the Address generator and the HIPAA Address generator.
Amazon Redshift
Fixed an issue with clearing temporary tables.
File connector
Fixed how Tonic handles EOF characters.
MongoDB
Corrected the order of the available generator presets for a field.
Snowflake
Fixed an issue where data generation returned the error
The specified bucket does not exist.
SQL Server
Database connections can now use the
MultiSubnetFailoveroption.Improved error messaging when a database cannot be created because of permissions issues.
V926 - V935
August 4, 2023
Removed support for TIM - The Tonic Installation Manager (TIM) command-line tool to install and configure Tonic is no longer available.
Free trial users can now use a public email address to create the free trial account. Users with public email addresses cannot invite other users or share workspaces. Public accounts are only allowed for free trials.
Users on a Professional instance can now share the Manager workspace permission set with users and groups.
Improved error handling and validation messages for the foreign key file upload process.
Counts of generator preset occurrences no longer include occurrences in deleted workspaces.
On the bulk update panel in Database View, the consistency and differential privacy options now display correctly.
Fixed an issue where you could not select Passthrough as a sub-generator in a composite generator.
Fixed an issue where custom presets could not be deleted.
Fixed a display issue where long post-job action names overflowed into the next column.
Fixed an issue where you could not assign Random Timestamp as a sub-generator for the Conditional generator.
Fixed an issue where the generator configuration panel displayed the generator preset options when the user did not have the Manage generator presets global permission.
Fixed an issue where when a constraint failed to be applied, data generation failed.
Improved display when users who do not have the Manage generator presets permission try to display the Occurrences tab on the preset details panel.
Improved how we handle unavailable options for workspace actions in Workspaces view and in the Tonic navigation options.
For the Conditional generator, Tonic now correctly compares MySQL date values.
Databricks
Tonic cluster initialization scripts are now uploaded as workspace files instead of DBFS files. The new, optional Workspace Path setting for Databricks workspaces controls the parent directory where Tonic uploads initialization scripts. The default value is
/Shared.
File connector
The file connector can now support .txt files that contain CSV, XML, or JSON content.
Fixed an issue where Tonic incorrectly identified how a file connector file was encoded.
Improved error messages when uploading files for the file connector.
Fixed an issue when configuring a file group from Amazon S3 where users saw the error "Failed to fetch files from S3. The continuation token provided is incorrect." but could still see the list of files in the S3 bucket.
Tonic now correctly updates the file configuration for file groups. Previously, users could not add files that did not match the default configuration.
Tonic now displays an error when it is unable to read files from Amazon S3.
The file explorer for Amazon S3 can now list the files in folders that have names that contain special characters.
Improved encoding detection and file parsing.
Tonic now correctly handles EOF characters in .csv files.
Tonic now preserves the encoding of .csv files.
MongoDB
Fixed an issue where the protection status information at the top of Privacy Hub did not update correctly after a new sensitivity scan.
V923 - V925
July 28, 2023
Custom generator presets
Earlier this year, for Enterprise instances, we introduced the concept of generator presets. A generator preset is a saved configuration of a generator. You can assign generator presets to columns.
The initial release only included built-in generator presets, which allowed you to set the default configuration for Tonic generators.
This update in V924 introduces custom generator presets, which allow you to set up multiple configurations of the same generator. You can create custom generator presets from Generator Presets view. From a generator configuration panel, you can also save the current configuration as a new custom generator preset.
Generator preset occurrences
From Generator Presets view, you can see how often each preset was used in a workspace configuration.
The Occurrences column of the generator presets list shows:
The number of times the baseline configuration was used
The number of times the baseline configuration was overridden, meaning that a user selected the generator preset and then made a change to the generator configuration
On the generator preset details panel, the Occurrences tab displays both the number of occurrences and the specific workspaces and columns where the generator preset was used. You cannot see workspace and column details for workspaces that you do not have access to.
Other updates
Tonic can now integrate with GitHub for SSO authentication.
To manage generator presets, users must now have the Manage generator presets global permission. Previously, you could also manage generator presets if you had the Manager or Editor workspace permission set for any workspace.
Fixed an issue where the table data in Table View was not updated correctly when switching the table mode to or from Scale mode.
Improved performance for the Regex Mask and Conditional generators.
MongoDB
Fixed an issue where the subsetting Graph View did not display virtual foreign key relationships.
You can now add collections to a subsetting rule before the sensitivity scan completes.
PostgreSQL
Fixed an issue where certain database constraints were not handled correctly, which resulted in job warnings about the failure to add those constraints.
V921 - V922
July 26, 2023
Permissions and permission sets
As of V922, Tonic now uses permissions and permission sets to manage access to Tonic features and functions.
A permission controls access to a single feature or function. A permission set is a saved collection of permissions.
Tonic provides built-in global and workspace permission sets. You cannot change the configuration of built-in permission sets. Enterprise instances can create custom permission sets.
Global permission sets contain global permissions, which control access to actions outside the context of a specific workspace. The built-in Admin global permission set grants access to all global permissions. Users and groups configured in the TONIC_ADMINISTRATORS environment variable are granted the Admin (Environment) global permission set, which also grants access to all global permissions. These global permission sets replace the previous admin user concept.
The built-in General User global permission set is assigned to all Tonic users, and grants access to create workspaces. You can also designate a different global permission set to assign to all Tonic users.
Workspace permission sets are assigned in the context of a specific workspace. They provide access to workspace permissions, which are associated with workspace management functions. The built-in workspace permission sets (Manager, Editor, Viewer Auditor) mirror the previous workspace roles. Similar to the previous Owner role, the Manager workspace permission set is granted access to all workspace permissions. However, unlike the Owner role, the Manager workspace permission set can be assigned to any user or group. You use the workspace sharing function to assign workspace permission sets within a workspace.
Each workspace has a single owner. The user who creates the workspace is the initial owner. All owners are by default granted the Manager workspace permission set. You can also designate a different workspace permission set to assign to workspace owners. You use the transfer ownership function to select a different owner for a workspace.
On Tonic Settings view, the User Management tab is replaced by the Access Management tab. From the Access Management tab, you can:
If you use SSO, view a list of SSO groups
View, configure, and manage access to global permission sets
API endpoint to track user access and permissions
A new API endpoint to track the following events related to user access and permissions:
A user account is created.
A user account is removed.
A user logs in to Tonic.
A permission is added to or removed from a permission set.
A permission set is assigned to or removed from a user. This might be a global permission set, or a workspace permission set in the context of a specific workspace.
A generator preset is updated.
The endpoint is:
GET /api/audit-events/search
Other updates
You can now assign the Business Name generator as a sub-generator for the Regex Mask generator.
For subsetting, Graph View and the table details panel now display information about cycle breaks, when the subsetting process needs to break a circular dependency.
Databricks
Fixed an issue that prevented the use of partition filters in Databricks Unity Catalog workspaces.
File connector
When a file connector workspace is deleted, Tonic now deletes files that were uploaded from a local file system.
Spark
Fixed an issue with data generation for workspaces that use Hive.
V910 - V920
July 21, 2023
The Admin Panel is renamed to Tonic Settings.
Tonic now provides a more meaningful error when Preserve Destination mode is assigned to a table in a workspace that does not have a defined destination database.
Fixed an issue where Tonic opened too many connections to the application database.
Fixed an issue with timestamps in the Tonic API specification.
Fixed a Tonic Cloud issue where using a different email domain to update a Tonic license caused issues with Tonic logging.
Enhanced the performance of the HIPAA Address generator.
The Data Pipeline V2 data generation process now respects the TONIC_PROCESS_PARALLELISM environment variable.
Improved performance for subsetting, especially for data that contains a large number of foreign key relationships.
Made a small performance improvement to primary key generators.
File connector
Improved error messages when uploading files for the file connector.
The file connector now supports extended ASCII-encoded files.
Fixed an issue where the file preview omitted the first row of the file.
MongoDB
On the Jobs view, you can now filter for the Collection Schema Scan job type.
Fixed an issue where sensitivity scans failed on large collections.
Oracle
Tonic now provides a more meaningful error when it loads Table View for a table that the database account does not have access to query data from.
PostgreSQL
Corrected the job history entries for subsetting jobs that run using the Data Pipeline V2 process.
Spark
Fixed an issue that caused jobs to fail when an invalid Repartition or Coalesce value was specified.
V899 - V909
July 14, 2023
Tonic Data Pipeline V2 for PostgreSQL ends beta
During the first half of 2023, Tonic has run a beta program for PostgreSQL for our new Data Pipeline V2. The beta program is now ending. Thank you to all of those who provided feedback.
Starting with version V905, Tonic.ai will progressively enable Data Pipeline V2 for all customers. To ensure a smooth transition for all our PostgreSQL customers, Tonic.ai controls the rollout remotely.
The remote rollout mechanism is controlled by an HTTPS request from your instance to https://feature-flag.tonic.ai. A JSON payload with a unique identifier for your deployment is sent, and the status of Data Pipeline V2 is returned. This request happens at the start of each data generation. If your Tonic server cannot reach https://feature-flag.tonic.ai, then the check is skipped.
What to expect for the enrollment:
Before your instance is enrolled in Data Pipeline V2, Tonic Customer Success will contact you to confirm your enrollment.
After your instance is enrolled, the Data Pipeline V2 toggle on the Confirm Generation panel is removed. All PostgreSQL jobs run using Data Pipeline V2.
For jobs that run on V2:
The job type is Data Pipeline Generation instead of Data Generation.
Jobs should run faster. Data Pipeline V2 has better resource handling, and can provide more parallel execution, especially for large tables and to apply constraints. Not all jobs will be faster, but for some jobs there should be a significant improvement.
We will continue to improve Data Pipeline V2 as we expand coverage to other data connectors.
Expanded Graph View for subsetting
The subsetting Graph View is expanded to use more of the available screen space. The Configure Subset panel, which includes the Options and Latest Results tabs, no longer displays on Graph View. It only displays on Table View.
Other updates
Fixed an issue where when a data generation job failed, tables that used Preserve Destination table mode were not restored.
The generated Tonic API documentation now includes the endpoints for managing file groups for file connector workspaces.
Fixed an issue that caused jobs for some workspaces to fail with the exception "Cannot modify workspace whose schema is not the latest version.".
Fixed an issue where the job details view displayed incorrect information.
Updated the /api/DataSource endpoint to not contain secure data such as the API key.
Updated our OpenAPI documents to ensure that all values of operationId are unique.
Improved error messages for failed data generation.
Databricks
The Test Connection button on the workspace details view now works correctly.
MongoDB
Fixed an issue where schema scans time out even when a timeout is not configured.
Fixed an issue where MongoDB workspaces did not support MongoDB versions below v4.4.
MySQL
Tonic data generation now supports generated columns in de-identified tables.
Oracle
To improve the resiliency of Oracle commands during data generation, fixed the retry logic.
PostgreSQL
Tonic no longer waits to process a job cancellation until after it finishes the constraint application that was in progress when the job was canceled.
Fixed an issue where Tonic returned an "insufficient space" exception when writing numeric values to a destination database.
Snowflake
Improved performance for Snowflake on Azure.
Spark
Job tracking URLs now display correctly on the job details page.
SQL Server
Fixed an issue where Tonic did not retrieve all of the compound keys from the source data.
Last updated
