TOИIC
Search…
Release notes

V507

July 1, 2022
Refreshed the Audit Trail user interface on Privacy Hub. The new Protection Audit Trail provides a paginated list of the updates to the sensitivity designation and generator assignments.
Deep links now work correctly when you use Google SSO to authenticate.
Error messages from Oracle are now displayed in response to invalid where clauses in subset configuration.
Made minor memory improvements to the Address generator.
Snowflake
  • Reduced the frequency of schema change detection on Snowflake databases. This can result in cost savings on Snowflake clusters, because the clusters can sleep more often.
  • Snowflake generation now works correctly when there are foreign key constraints.
SQLServer
  • Added support for security policies, sequences, check constraints, and system versioned temporal tables.

V502

June 24, 2022
Improved cross-tab support for automatic logouts when you configure an inactivity period.
The Update option in the actions menu now takes you directly to the System tab on the Admin Panel instead of the Users tab.
Corrected the password length requirement to be 12 characters or greater instead of greater than 12 characters.
Improved the estimated row progress for scaled tables.
Eliminated a race condition that occurred when applying constraints.
SQL Server:
  • The Categorical generator can now support more than 2 billion rows in a category.
Databricks:
  • Can now run concurrent jobs that use different versions of Databricks.
  • Tonic now supports ORC and Hive tables in Databricks.

V500

June 17, 2022
When users create a new password, Tonic now displays a panel with the password requirements, and indicates whether the password meets those requirements.
Improved the parallelization and concurrency for processing foreign key constraints.
Databricks and Spark EMR
  • Improved the performance of the Noise generator.
MongoDB
  • Improved display of longer key values in the Key column. Widened the column and added truncation.
  • Optimized queries against the Tonic database.

V495

June 10, 2022
Features/Enhancements
  • For Postgres databases, Tonic now supports name and char data types.
  • For Tonic single sign-on, Tonic now supports Azure Active Directory.
  • From the administration screen, administrators for customers that run Tonic on Kubernetes and Docker can now download logs from all containers that run Tonic.
  • For Spark-powered integrations, Tonic now supports the Address generator as a sub-generator.
Bugs
  • Improved performance for:
    • Random Hash generator on Spark and Databricks
    • UUID Generator on Spark
    • Subsetting
  • Improved the browsing experience on low resolution displays.

V485

May 27, 2022
Features/Enhancements
  • Skip batch instead of failing generations on Postgres in some cases when values fail to be inserted
  • Improvement to Synthesis Reports for AI Synthesizer generated data
  • Changing Lambda deployment from ECR to ZIP for integrations using Lambda functions. Images no longer need to be manually deployed to ECR.
Bugs

V482

May 25, 2022
Features/Enhancements
  • Support for arrays with Struct Mask generator on EMR/Spark and Databricks
Bugs
  • Fix issue causing privacy scans on Dremio to fail
  • Minor UI fixes to workspace page
  • Fix for data generation issue on Db2 iSeries when destination is empty
  • More robust handing of Databricks host URLs

V481

May 24, 2022
Features/Enhancements
  • Adds data types to all keys in Mongo Collection view and tooltip popovers to key names
  • Updates to data generation UI
Bugs
  • Fix shaky dialog when table mode resizes

V480

May 23, 2022
Features/Enhancements
  • Add data type advanced search for Oracle, Redshift, and BigQuery
  • Performance improvements for constraint restoration
  • Improved performance of Random Integer generator on Spark
Bugs
  • Fix issue on SQL Server preserve destination mode when table names contain a ".".
  • Postgres: Fix issue copying arrays with trailing slashes

V479

May 20, 2022
Features/Enhancements
  • Allow the console log level to be configured on the Web Server and Worker
  • User passwords can now be reset by an Admin in the Admin Panel

V478

May 19, 2022
Features/Enhancements
  • Added compatibility with SQL Server 2012
  • Spark SDK: Support for validation of workspace when processing a dataframe
Bugs
  • Minor fixes for MongoDB integration

V467

May 17, 2022
Features/Enhancements
  • Support graceful cancellation of privacy metric computation for AI Synthesizer
  • Java SDK documentation available from the SDK Setup dialog
Bugs
  • Only show SSO and Enterprise licensing information to users when appropriate
  • Minor UI fixes for scrolling on the workspace view table
  • Minor UI fixes for the Edit Workspace page

V466

May 17, 2022
Features/Enhancements
  • For S3/EMR data source, support separately testing each component of the connection (Glue catalog, EMR cluster, and S3 bucket)
  • The Additive Noise Generator is renamed to "Noise Generator" with two noise options "Additive" (existing) and "Multiplicative" (new)
  • Improve Javadocs for Spark SDK and other enhancements
Bugs
  • Fixes for the SQL beautifier on post-job scripts, including scenarios which may have crashed the app on Safari
  • Skip processing Temporal Types when not supported (SQL Server 2012) to prevent job failures

V465

May 16, 2022
Features/Enhancements
  • Add hourly heartbeat/status log message for web server and worker
  • Group database/integrations on the create workspace screen by type

V464

May 13, 2022
Features/Enhancements
  • Add support for Azure Datalake Storage Gen 2 as an output destination for Databricks
  • Don't include statistics for Oracle schema copy
  • MongoDB: Support uploading foreign key file, support deleting configured foreign keys
Bugs
  • Do not stop generation in SQL Server if schema pre-fetch fails
  • Improved deletion of old logs to prevent timeouts

V463

May 13, 2022
Features/Enhancements
  • Adds support for Spark 3.1
  • Improvements to Dremio for data type searching, Struct support, and a new getVersion method on SDK
Bugs
  • Fix workspace name not updating in dropdown after rename
  • Don't allow non-editor/owner to open table mode selector

V462

May 12, 2022
Features/Enhancements
  • Update subsetting tooltip help text
  • Don't check destination recovery mode on Azure SQL
  • Improve performance of table constraint application when subsetting by applying single and cross table constraints in parallel
Bugs
  • Fix logo navigation redirect when signed in as a user with no workspaces
  • Fix issue with setting a partition filter on Spark
  • Fix issue with collection picker on MongoDB showing removed collections
  • Fix excessive memory usage of Hostname generator

V461

May 10, 2022
Features
  • Improves performance of Custom Categorical generator on Spark and Databricks
  • Add support for Azure Databricks
  • Display SQL Server schema generation progress as a percentage
Bugs
  • Do not fetch column data on configuration popover open to reduce queries against the source database
  • Reduce the number of queries to the source database for table/column details during generation

V460

May 10, 2022
Features
  • Spark 2.4 and Databricks Library Improvements
  • Add ability to remove users in the Admin Panel
Bugs
  • Fixed issue with the displayed port when connecting to MongoDB via connection string
  • Fix for data table column resizing not rendering properly
  • Support random latitude and longitudes with HIPAA Address generator

V459

May 9, 2022
Features
  • New design for the workspace dropdown selector
  • Add warning when previewing AI Synthesizer on text which does not appear to be categorical
Bugs
  • MongoDB: Collections no longer uncollapse when a generator is applied in hybrid view

V458

May 6, 2022
Features
  • Add support for Spark SDK on Spark v2.4
Bugs
  • Fix UI crash when selecting incremental as Table Mode
  • Fix AI synthesizer when no categorical columns present
  • Ensure AI Synthesizer model is not trained if only one column is selected
  • Render HTML Mask in the generator model panel on table view
  • Fix issue with not being able to set incremental table mode column if you refresh after setting it to incremental

V457

May 5 2022
Features
  • Added a pre-job check to guard against cycles in applied generators
  • Better support for environments where websockets are not supported
    • Add environment flag to disable websockets on API
    • Gracefully fail if websockets can't connect

V456

May 3, 2022
Bugs
  • Fixed issue preventing connecting to Postgres databases using an SSH tunnel
  • Fixed nullability error in US Phone generator

V455

May 3, 2022
Bugs
  • Fix issue where collection loads were attempted for non-Mongo DB's in some instances
  • Write pg_dump results directly to file instead of reading them via stdout to prevent process locks
  • Fixed isssue where copying a workspace on the Workspace View doesn't update the table of workspaces

V454

May 2, 2022
Features
  • Support applying generator values for PK fields in MongoDB to their FK fields
Bugs
  • Reduce Frequency of Upstream table filtering logs
  • Remove double scroll bar on foreign key page when viewing foreign keys
  • Improved database introspection queries on IBM Db2 iSeries

V453

April 29, 2022
  • Consistently clear local storage on authorization failures
  • Aggregate dropped row exceptions for more efficient logging

V452

April 28, 2022
Features
  • Improved performance for schema scans on MongoDB
Bugs
  • Minor fixes to workspace view
  • Fix for concurrent lambda function update issues
  • Fix brief non-ideal state while loading collections on MongoDB

V451

April 27, 2022
Features
  • Allow skipping of privacy scan on MongoDB using TONIC_TABLE_SKIP_REGEX environment variable

V450

April 27, 2022
Features
  • Improve usability of processing with Java SDK
Bugs:
  • Close connection to application database when not being used
  • Removed excess querying of database version

V449

April 26, 2022
Features
  • Improves performance of Name Generator on Spark / Databricks
  • Change Dremio schema input to Tag Input and add an informational popover
Bugs:
  • Do not null FKs between reference tables
  • Allow foreign keys to be added by editors

V448

April 26, 2022
Features
  • Improves performance of the SSN Generator on EMR Spark / Databricks
Bugs:
  • Keep the password field empty when copying from source database settings to destination database
  • Improved exception handling when masking and preserving destination tables
  • More quickly alert user when a cycle has been found in their relationships
  • Handle Postgres interval data types with a null boundary
  • Disable Add Foreign Keys tab for non-workspace owner users

V447

April 25, 2022
Features
  • The Workspaces view now supports bulk actions. Share, transfer, delete, and leave multiple workspaces by checking the boxes at the right end of each row, and choosing the desired action from the Actions menu.
Bugs:
  • Fixed an issue where Safari would fail to load the application
  • Fixes to constraint application parallelism
Breaking Changes
  • The response for the Workspace deletion endpoint (/api/workspace/{workspaceId}) has been modified slightly

V446

April 22, 2022
Features
  • Enable Timestamp Generator To Work Without Statistics For Spark Object Masks
  • When a mongo foreign key field has a primary key with a generator applied, replace the example data with a link to the collection with the primary key
Bugs
  • Fix for MySql generation hanging with empty password

V445

April 21, 2022
Features
  • Improves performance of Name Generator on EMR and Databricks
Bugs
  • Fix Display Bug in AI Synthesizer Configuration Panel
  • Fix application of constraints in Redshift

V444

April 20, 2022
Features
  • Add the version of the worker that ran a job to the job details
  • On the Mongo Collection View for foreign/primary keys: Added a hover tooltip for key icons, replaced the generator with a label
  • Refresh the last collection visited when a new foreign key file is uploaded and optimize single_doc foreign key
Bugs
  • Disable date truncation generator as a subgenerator for Spark
  • Fixes Notification service consuming all disk space when crashing
  • Show the appropriate Conditional sub-generator label instead of passthrough
  • Fixed in memory table query for SQL Server

V443

April 19, 2022
Features
  • Teardown and database creation performance improved for SQL Server
  • Skip preserve destination tables (just like truncated) for privacy scan
Bugs
  • Fix not allowing generators allowed on arrays that were scanned prior to v439

V442

April 18, 2022
Features
  • Adds support for events (dependent rows) to AI Synthesizer (formerly known as Smart Linking)
  • Allow wildcard (%) in schema name for Dremio
Bugs
  • Support Date Truncation and Timestamp Shift on Snowflake TIMESTAMP_TZ and Redshift TIMESTAMPTZ columns
  • Disable the generator dropdown in the UI for columns with both primary and foreign keys
  • MongoDB - Fix errors when linking Categorical Generators
  • Bug fixes for synthesis report
  • Fix data type mappings for Dremio Integer and Varchar

V441

April 14, 2022
Features
  • Display whether webhooks were used in job details
  • Terminology changes
    • Masked is changing to De-identify
    • Synthesized is changing to Scale
    • Truncated is changing to Truncate
    • Generator name change: Smart Linking is changing to AI Synthesizer
Bugs
  • Fixes to SQL script beautifier

V440

April 14, 2022
Features
  • Move single connections on connection pools behind a feature flag
  • Do not drop indexes on auto increment columns in MySQL to improve performance
  • Add type filter to JSON Mask generator
  • Remove unique constraint on workspace names and change default workspace name to "Untitled Workspace"
  • Display key icons for user uploaded foreign/primary key fields in Mongo Single document view
Bugs
  • Fix issue where 0 results in Workspaces table sometimes let you navigate to a negative page number
  • Fix issue where workspace permissions didn't update in the UI until a refresh
  • Fixes authentication error on Postgres when username has special characters
  • Fixed reopening open connections when TONIC_ENABLE_SINGLE_CONNECTION is false
  • Remove owner section from exported workspace body
  • Improve readability of workspace tags in small-width window sizes

V439

April 13, 2022
Features
  • Apply foreign key constraints serially on a different thread to avoid deadlocks & improve performance
Bugs
  • Clear bulk column search query on filters reset
  • Memory improvements to Mongo schema serialization
  • Better handling of schema name in Java SDK

V438

April 12, 2022
Features
  • Allow changes to order of JSON Mask sub-generators
  • Running a job is now disabled when the license expiration date has passed
  • Separate Workspace Sharing and Workspace Role-based Workspace access into different license features
  • Improve SQL Server performance by prefetching all tables & views
Bugs
  • When using Java SDK only show Java supported generators

V437

April 11, 2022
Features
  • Improves performance of Email Generator on EMR and Databricks
  • Add support from Dremio with Spark
Bugs
  • Allowing Kubernetes ImagePullBackOff for up to 5 minutes before throwing error when updating through UI
  • Fixes inability to run a data generator on Spark with default database as the source

V436

April 8, 2022
Features
  • Display estimated time remaining on row-based tasks in job details page
  • Display key icons for user uploaded foreign/primary key fields in Mongo hybrid documents
Bugs
  • Copy Generator in Mongo: Fix bug where if generator was copying a field that wasn't in the document it would throw exception
  • Add exception handling for SQL Server datetime columns to drop invalid records, better SQL Server XML type exception handling
  • Create source pool just before processing tables to avoid any timeouts in SQL Server

V435

April 8, 2022
Features
  • Display row based progress on jobs whenever possible
  • Statistics Jobs for the Java Spark SDK now appear in Jobs Table/Jobs Detail Page
  • Skip pre-job health check for PyML service when not needed

V434

April 7, 2022
Features
  • Added database metric sharing for AWS users
  • Don't null self-referential foreign keys on reference tables when subsetting

V433

April 7, 2022
Features
  • Improves performance of Email Generator on EMR and Databricks
Bugs
  • Better small-width responsiveness of tables in UI

V432

April 6, 2022
Features
  • Add synthesis report
  • Support for update via UI with Docker
Bugs
  • Prevent unexpected behavior of progress tracker when the system clock shifts
  • Fixes error when creating partition function with null range on SQL Server
  • Fix issues with loading screen showing when not needed

V431

April 5, 2022
Features
  • Add Remove Whitespace Transformer to Copy generator
  • Add ability to generate JSONPath expression for JSON Mask generator by clicking on preview JSON
Bugs
  • Improved support for large decimal numeric types in Postgres

V430

April 4, 2022
Features
  • Add exception handling for invalid SSH Tunnel private key format and list valid formats in tooltip

V429

April 4, 2022
Features
  • Apply constraints in parallel to data generation in Postgres
Bugs
  • Ensure workspace tag input closes when focus is lost from outside click
  • Remove foreign key checks when applying constraints in MySQL

V428

April 1, 2022
Features
  • Invalidate refresh tokens on logout, log in or out all open sessions (tabs) on login or logout, cross-tab inactivity timeout support
  • Improve load times on workspace pages
Bugs
  • Fix linking of non-consistent generators in Spark

V427

March 31, 2022
Features
  • Include workspace ID in URLs
  • Parallelize writes on MySQL passthrough tables
  • Allow decimals in subsetting percentages
Bugs
  • Don't allow duplicate entries of tables in table mode settings for workspace
  • Handle null values with Copy generator on MongoDB
  • Database query performance improvements

V426

March 30, 2022
Features
  • Improve MySQL generation performance by applying constraints in parallel
Bugs
  • Fix issue where indexes fail to be applied to converted columns on SQL Server
  • Fix issue with SSO account creation failing

V425

March 29, 2022
Features
  • Support Copy generator on MongoDB
  • Support parallelism of MongoDB schema scan
  • Support truncation mode on MongoDB
Bugs
  • Set Command Timeout in Postgres to prevent timeouts

V424

March 25, 2022
Features
  • Change password requirements
    • Length >= 12 characters
    • Must include number
    • Must include lowercase letter
    • Must include uppercase letter
    • Must include non-alphanumeric character
Bugs
  • Fixes to transfer workspace text
  • Allow opening Admin Panel when user has no workspaces

V423

March 25, 2022
Features
  • Improve constraint restoration performance on MySQL
  • Allow incremental tables in MySQL where all columns are keys
Bugs
  • Prevent app from crashing when admin status of user is unknown

V422

March 24, 2022
Features
  • Add login rate limitin3
Bugs
  • Fix issue with functions and procedures not being created in the destination on Oracle databases

V421

March 23, 2022
Features
  • On EMR and Databricks, write JSON files to JSON instead of Parquet
  • Improved handling of Latitude/Longitude with Smart Linking Generator
  • Add support for enum arrays columns in Postgres
Bugs
  • Fix issue with EMR cross account data generation failing to resolve the Glue database

V420

March 17, 2022
Features
  • Workspace Updates - Admin View All, Leave, Share & Transfer V2, Owner Filter, and more
  • Parse as bigint if decimal fails
  • Oracle Whitelist support in BuildDb
  • Initialize after log sink reset
  • Replace slashes with underscores in build versioning
  • Add 'No generators' filter for bulk table view
  • Add progress tracker for SQL server post-data script generation
  • Allow support for views without fully qualified tables in Redshift
  • Improve performance of listing workspaces
  • Improve performance of the Constant Generator on EMR and Databricks
  • Sensitive columns now appear in the privacy hub for Mongo workspaces
  • Allow processing of decimals with a scale of 1 up to 9223372036854775807 in Postgres
  • Check for cloudsqlsuperuser permission on GCP Cloud SQL destination postgres databases
Bugs
  • Address Lambda Function Setup bug
  • Improve Regex Mask Display of Empty Matches and values with new lines
  • Do not fail completely when SSO groups do not resolve
  • Misc Redshift View fixes
  • Fix issue where column can't be in schema changes and show as sensitive
  • Fix issue where some downstream tables are missing their foreign keys in the destination database when using subsetting
  • Fix issue with shifting table size on data preview in database view
  • Prevent error from occurring when MongoDB collection size is too large

V412

March 10, 2022
Features
  • Added support for user defined enum types as keys in subsetting
  • Simplify schema diff changes error toast on generation
  • Show more unprotected columns button mongo privacy hub
  • Added filtering on custom value generator dropdown
  • Add ConsistentOn to AdditiveNoiseGenerator
  • Enabling Timeshift Generator on Compound Primary Keys
  • Regex Mask Generator supports matching empty capture groups
  • Accommodate OFFSET in where clauses in SQL Server subsetting
  • Miscellaneous workspace config improvements
  • Add support Azure Sql with SqlServerDatabaseCreator
  • Add progress tracking for mongo schema scans
  • Add syntax highlighting for where clauses in subsetting
  • Add support for integer primary keys on smallint
Bug fixes
  • Fixed error message for MySQL data type not found
  • Fix Postgres Test Connection bug
  • Fix Workspace patches after rename or modify tags
  • Fix Unix Timestamp empty column bug
  • Throw OperationCanceledException instead of CancelTonicJobException on cancellation request
  • Fix Redshift view replication missing schema

V401

February 24, 2022
Features
  • Oracle perf improvements
  • Add new range options for integer pk generator
  • CNPJ and CPF generator performance improvements
  • Updates Regex Mask Generator to support replacing all matches
  • Start the progress token for passthrough tables when there are no passthrough tables with references
  • Add support for Timestampshift for spark
  • Add warning system to Privacy Scan
  • Add ability to update via UI for Admin users for Kubernetes
  • Regex mask generator with Custom Value Processor
  • Check for cancellation and report progress on text copy in Postgresql
  • Miscellaneous perf improvements
Bug fixes
  • Start the progress token for passthrough tables when there are no passthrough tables with references
  • Treat non-standard oracle error code 5 not as an error
  • Fix grabbing workspace users for comments
  • Dispose the source connection instead of just closing in Postgres
  • Fix for test discovery exception
  • Fix postgres sequences behavior
  • Object mask bulk view fix
  • Miscellaneous typos fixed
  • Fixed database type for BigQuery test connection

V391

February 15, 2022
Features
  • Relax unique index constraint on username
  • Added tonic version in containers as an environment variable
  • Adding logic to bypass Fallback generators for Spark databases
  • Support case insensitive name consistency
  • Disable dropping replacements on missing columns for mongo
  • Update Oracle driver
  • Enable additive noise generation for strings in mongo
  • Add support for unsigned 64-bit integer columns in integer key generator
  • Add random timestamp and timestamp shift generators as options for integer columns (for unix timestamp columns)
  • Add checks for min/maxTime on timestamp and min/maxDate on dates
  • Adding null ref check for fallback generation option check during preview
  • Revamped workspace configuration screen
  • Support for Decimal on Spark (when using C# UDFs)
  • Allow Regex Generator on PKs
  • Added new environment variable to filter tables ingested by UI for Oracle
  • Misc perf improvements
Bug fixes
  • Fix mongo view more values issues
  • Disable dropping replacements on missing columns for mongo
  • Fix Workspace specified via API without DBs can be modified in UI
  • Fixing Random Timestamp generator on Java
  • Date truncation generator bug fixes

V383

February 8, 2022
Features
  • Better error exception handling for number conversions in Mongo
  • Added loading indicator when viewing single Mongo document
  • Improved Mongo performance and logging
  • Added table filter validation for EMR
  • Handling numbers as doubles instead of integers in Snowflake
  • Added aiven.io system schema to Postgres
  • Added the Geo Generator for latitude and longitude data
  • Added support for materialized view in Redshift
Bug fixes
  • Fixed array regex mask generator with correct subgenerator options
  • Fixed conditional generator by adding binding

V378

February 4, 2022
Features
  • Updated to dotnet 6
  • Moved FK import to FK page
  • Added support for tinyint in MySQL as a primary key
  • Added table partition filtering for EMR
  • Enabled Regex Mask generator on EMR
  • Added support decimals with precision and scale for EMR
  • Moved Tonic version number to hamburger menu
  • Added job sorting by start time
  • Ability to retrieve Kubernetes logs via API
  • Improved behavior of upstream reference tables for subsetting
  • Improved logging for subset filtering
  • Updated sensitivity scan to better reflect job status
  • Improved privacy scan performance for Mongo
  • Introduced automated schema checks (not for Mongo)
  • Added HTML Mask generator
  • Added Additive Noise Generator
  • Added CNPJ generator
  • Added Array Regex Mask generator
Deprecated endpoints
Updates to the foreign key feature impact the API endpoints used to fetch, upload, and delete the Foreign Key file. Two previous API endpoints are now marked as deprecated:
  • DELETE /api/datasource/delete_fkupload has been marked as deprecated and replaced by DELETE /api/workspace/{workspaceId}/foreign-key/
  • GET /api/datasource/download_fkupload/ has been marked as deprecated and replaced by GET /api/workspace/{workspaceId}/foreign-key/
The main change between both APIs is the switch to using workspaceId to fetch the Foreign Key file instead of the datasourceId.
Both are slated to be removed after 2022-06-01.
Bug fixes
  • Fix number parsing issues in JSON mask generator
  • Fix MySQL parsing of auto_increment columns
  • Eliminated re-querying of data type on EMR Spark