Structural Cloud API docs
Other product guides
Tonic.ai

V1054 - V1103

A new entry is added each week, and contains the release notes for all of the Tonic versions that were released during that week.

The following releases were removed from quay because of regressions:

  • V1054 - V1061

  • V1074

V1098 - V1103

February 16, 2024

Fixed an issue with the Name generator where capitalization was not preserved if consistency was disabled.

For Table View, fixed an issue where the delete button to remove the generator assignment was sometimes hidden.

Oracle

  • Tonic now supports Oracle Advanced Queues. Queue tables and queues are created in the destination database. Messages are not copied from the source database to the destination database. Queue subscribers are not currently supported. You must add them to the destination database manually.

PostgreSQL

  • Tonic can now de-identify the ltree data type.

SQL Server

  • Fixed an issue where indexes were not created on schema bound views.

V1090 - V1097

February 9, 2024

Redesigned Database View

We redesigned Database View to improve the display and the filtering.

In the updated columns list, the Column column contains the schema, table, and column name, and the column data type. It provides access to the data preview option.

The Applied Generator column shows the applied generator. Applied Generator indicates when a column is unprotected, when the column is a primary or foreign key, and when the configuration overrides the parent workspace. If the table mode is not De-Identify, it shows the table mode. It provides access to the commenting option.

Filters other than the column name filter are moved under the Filters option. There are also new filters for the sensitivity type (the type of sensitive data that Tonic detected in the column) and whether the column has a recommended generator.

Privacy Report updates

In the Privacy Report, new column, Column Privacy Rank, indicates the privacy ranking for a column based on the assigned generator and generator configuration. The generator summary and generator reference include the possible privacy ranking values for each generator.

Added a new column, Tonic Detected Sensitivity, that indicates whether the Tonic sensitivity scan identified the column as sensitive. Renamed the Is Sensitive column to Current Sensitivity. Current Sensitivity indicates whether the column is currently marked as sensitive.

Also corrected an earlier issue with the order of the columns.

Other updates

Fixed an issue that caused all subset runs to record the percentage of rows in the subset as 100%. Subset runs that occur after updating to this version display the correct percentage.

The option to write output data to a container repository is out of beta.

Databricks

  • For Databricks 13+, writing output to delta tables now uses the table’s liquid clustering configuration to cluster the insertion.

  • Fixed an issue with loading tables into the Tonic application.

Google BigQuery

  • Improved batch size calculations to reduce memory pressure during data generation jobs.

  • Fixed an issue with applying the Timestamp Shift and Date Truncation generators on datetime or timestamp data when TONIC_GRPC_ENABLED was false.

  • For the Timestamp Shift and Date Truncation generators, improved support for time values that include fractional seconds.

  • Fixed an issue where the Character Substitution generator failed when assigned to a NUMERIC or BIGNUMERIC column.

Oracle

  • Tonic now supports IDENTITY columns. Before this change, IDENTITY columns caused errors during destination database creation.

V1082 - V1089

February 2, 2024

For the Custom Categorical generator, you can now add a NULL value to the available custom category values. To indicate a NULL value, use the keyword {NULL}.

Made the following API updates to better accommodate users of the previous version of the API:

  • jobs/{id}/workspace_snapshot now returns the WorkspaceDataModel object.

  • Fixed an issue where workspace_snapshot could return empty replacements.

  • Added a new endpoint, GET jobs/{id}/workspace_snapshot?api-version=v2023_07_00, that returns V17WorkspaceDataModel

Databricks

  • When writing output to delta tables, the destination tables should now retain the TBLPROPERTIES from the source delta table, including 'delta.feature.allowColumnDefaults'.

V1076 - V1081

January 26, 2024

Redesigned data model for generator assignments - The new version of the Tonic API includes a redesign of the data model for generator assignments. To use the previous version of the generator assignment data model, make sure that your API calls specify version 2023.07.0.

The generator assignment data model redesign includes the following changes:

  • For all generators, moved the following fields to the metadata object in the link object:

    • presetId

    • generatorId

    • customValueProcessor

    • encryptionProcessor

  • For the <value type> Mask composite generators:

    • Moved pathExpression to the metadata object in the link object.

    • Removed the following fields from the link object:

      • subPresetId

      • subGeneratorId

      • customSubGeneratorValueProcessor

    • Added the following fields to the subGeneratorMetadata object under metadata:

      • presetId

      • generatorId

      • customValueProcessor

Other updates

Updated the sensitivity scan to better identify company and organization names and suggest the appropriate generator.

Databricks

  • The Conditional Generator is now supported on version 11.3+.

Google BigQuery

  • Fixed some cases where a copied View did not have its definition updated to reference resources within the destination dataset. This could result in failures when attempting to copy the View into the destination dataset.

  • Added support for Preserve Destination table mode.

  • Fixed an issue where sub-queries in table filter clauses did not work if TONIC_GRPC_ENABLED was set to true.

MySQL

  • You can now enable diagnostic logging for a MySQL data generation job.

PostgreSQL

  • Improved the destination database permissions check on Google Cloud SQL to handle additional superuser authentication setups.

SQL Server

  • Data generation now preserves hidden properties of columns.

V1072 - V1075

January 19, 2024

If your instance of Tonic is deployed on Docker, you can now use an external Kubernetes cluster to enable the option to write destination data to container artifacts.

You can now assign the Integer Key generator to a column with a decimal data type. The actual column values must still be integers.

Fixed an issue in Table View where an error displayed if you changed the selected table while the data was loading.

Databricks

  • For cluster versions 11.3 and above, Tonic now displays accurate lists of available and suggested generators.

File connector

  • Fixed an issue that prevented users from adding files to existing local file groups.

SQL Server

  • Fixed an issue that caused errors when creating inline functions in the destination database.

V1069 - V1071

January 12, 2024

The Enable Diagnostic Logging global permission is now granted to the built-in Account Admin permission set.

Databricks

  • CREATE CATALOG or CREATE SCHEMA permissions are no longer required if the destination catalog or schema already exists.

  • For Databricks 11.3 and higher, added support for the Random Double generator.

  • For Databricks 11.3 and higher, added support for the IP Address generator.

V1063 - V1068

January 5, 2024

Diagnostic logging for data generation - By default, Tonic now redacts sensitive data in data generation log files.

When users start a data generation or upsert job, if they have the new global permission Enable diagnostic logging, they can choose to enable diagnostic logging, which does not redact the logs. The Enable diagnostic logging permission is also required to download the diagnostic logs. By default, the permission is only granted to the Admin and Admin (Environment) global permission sets.

In addition to the option for individual jobs, there are environment settings that enable diagnostic logging for specific data connectors.

Other updates

In the Release Candidate version of the API, the response model for the GET /api/workspace/minimal endpoint has been updated for more straightforward de-serialization.

Fixed an issue where a non-unique composite primary key column could only be assigned unique generators.

Users can now press Enter to finish copying a workspace or a generator preset, instead of having to click Copy.

File connector

  • Added support for the Conditional generator for file groups that contain CSV files.

Google BigQuery

  • Post-job scripts now support DDL and EXPORT.

Oracle

  • Fixed an issue with the permissions check that prevented connecting to Amazon RDS for Oracle databases.

SQL Server

  • In Data Definition Language (DDL) that applies to the destination database, Tonic now strips WITH INLINE clauses from definitions of user-defined functions (UDF). Inlining does not require these clauses. WITH_INLINE clauses in UDF definitions that do not meet the requirements for inlining can prevent the UDF from being restored properly in the destination database.

  • Fixed an issue where the order of XML columns was changed in the destination database.

V1062

December 29, 2023

For the OpenID Connect (OIDC) SSO integration, Tonic now supports authentication by client secret that uses HTTP basic authentication (client_secret_basic). To provide the client secret, configure the TONIC_SSO_CLIENT_SECRET environment setting.

SQL Server

  • A new environment setting, TONIC_SQL_SERVER_SKIP_CREATE_DB, indicates whether to skip schema creation for the destination database. If true, then Tonic does not create the schema. It uses the existing schema to populate the destination database. The default is false. You can configure this environment setting from the Environment Settings tab on Tonic Settings.

V1054 - V1061

December 22, 2023

During free trial signup, the data connector options now include an option to use local files for the source data. This creates a file connector workspace for local files, and displays the File Groups view to allow the free trial user to start to add file groups to the workspace.

Added an environment setting, Tonic Test Connection Timeout In Seconds (TONIC_TEST_CONNECTION_TIMEOUT_IN_SECONDS), that you can set from the Environment Settings tab on Tonic Settings. This setting configures the timeout for testing a database connection. Previously, connection test attempts timed out after 5 seconds. The new default is 15 seconds.

When you configure a workspace to write the output to container artifacts, you can now specify custom resources for the Kubernetes pod, including the ephemeral storage, memory, and CPU millicores.

Improved performance when marking a large number of columns as not sensitive.

Fixed an issue that caused Tonic workers that are deployed on Docker to crash unexpectedly.

For numeric columns that support arbitrary precision and scale, when the scale is 0 (for example, NUMERIC(N,0)), or when the underlying values are all integers, these columns are now supported as primary keys for the purpose of subsetting.

Amazon EMR

  • You can now use the Timestamp Shift generator as a sub-generator within the Struct Mask generator.

Amazon EMR and Databricks

  • When files are saved to non-job-specific file destinations, the new environment setting TONIC_WORKSPACE_DEFAULT_SAVE_MODE indicates the mode to use. If set to a value other than null (Ignore, Append, Overwrite), this setting takes precedence over TONIC_WORKSPACE_DEFAULT_ERROR_ON_OVERRIDE.

Google BigQuery

  • Fixed an issue introduced in v982 where data generation failed with an HTTP 404 Not Found error for the destination table when the source and destination are in different BigQuery projects.

MongoDB

  • Added an environment setting, TONIC_DOCUMENT_MAX_DEPTH, to configure the maximum depth of JSON document that can be handled. The default value, which is also the recommended minimum value, is 32.

  • For bulk apply, fields with the same selection path but different data types no longer share a selection state.

SQL Server

  • Fixed an issue to properly restore schema-bound views in dependency order.

Last updated