Release Notes

Search…
Release Notes
Could not load image
V467
May 17, 2022
Features/Enhancements
Support graceful cancellation of privacy metric computation for AI Synthesizer
Java SDK documentation available from the SDK Setup dialog
Bugs
Only show SSO and Enterprise licensing information to users when appropriate
Minor UI fixes for scrolling on the workspace view table
Minor UI fixes for the Edit Workspace page
V466
May 17, 2022
Features/Enhancements
For S3/EMR data source, support separately testing each component of the connection (Glue catalog, EMR cluster, and S3 bucket)
The Additive Noise Generator is renamed to "Noise Generator" with two noise options "Additive" (existing) and "Multiplicative" (new)
Improve Javadocs for Spark SDK and other enhancements
Bugs
Fixes for the SQL beautifier on post-job scripts, including scenarios which may have crashed the app on Safari
Skip processing Temporal Types when not supported (SQL Server 2012) to prevent job failures
V465
May 16, 2022
Features/Enhancements
Add hourly heartbeat/status log message for web server and worker
Group database/integrations on the create workspace screen by type
V464
May 13, 2022
Features/Enhancements
Add support for Azure Datalake Storage Gen 2 as an output destination for Databricks
Don't include statistics for Oracle schema copy
MongoDB: Support uploading foreign key file, support deleting configured foreign keys
Bugs
Do not stop generation in SQL Server if schema pre-fetch fails
Improved deletion of old logs to prevent timeouts
V463
May 13, 2022
Features/Enhancements
Adds support for Spark 3.1
Improvements to Dremio for data type searching, Struct support, and a new getVersion method on SDK
Bugs
Fix workspace name not updating in dropdown after rename
Don't allow non-editor/owner to open table mode selector
V462
May 12, 2022
Features/Enhancements
Update subsetting tooltip help text
Don't check destination recovery mode on Azure SQL
Improve performance of table constraint application when subsetting by applying single and cross table constraints in parallel
Bugs
Fix logo navigation redirect when signed in as a user with no workspaces
Fix issue with setting a partition filter on Spark
Fix issue with collection picker on MongoDB showing removed collections
Fix excessive memory usage of Hostname generator
V461
May 10, 2022
Features
Improves performance of Custom Categorical generator on Spark and Databricks
Add support for Azure Databricks
Display SQL Server schema generation progress as a percentage
Bugs
Do not fetch column data on configuration popover open to reduce queries against the source database
Reduce the number of queries to the source database for table/column details during generation
V460
May 10, 2022
Features
Spark 2.4 and Databricks Library Improvements
Add ability to remove users in the Admin Panel
Bugs
Fixed issue with the displayed port when connecting to MongoDB via connection string
Fix for data table column resizing not rendering properly
Support random latitude and longitudes with HIPAA Address generator
V459
May 9, 2022
Features
New design for the workspace dropdown selector
Add warning when previewing AI Synthesizer on text which does not appear to be categorical
Bugs
MongoDB: Collections no longer uncollapse when a generator is applied in hybrid view 
V458
May 6, 2022
Features
Add support for Spark SDK on Spark v2.4 
Bugs
Fix UI crash when selecting incremental as Table Mode
Fix AI synthesizer when no categorical columns present
Ensure AI Synthesizer model is not trained if only one column is selected
Render HTML Mask in the generator model panel on table view
Fix issue with not being able to set incremental table mode column if you refresh after setting it to incremental
V457
May 5 2022
Features
Added a pre-job check to guard against cycles in applied generators
Better support for environments where websockets are not supported
Add environment flag to disable websockets on API
Gracefully fail if websockets can't connect
V456
May 3, 2022
Bugs
Fixed issue preventing connecting to Postgres databases using an SSH tunnel 
Fixed nullability error in US Phone generator
V455
May 3, 2022
Bugs
Fix issue where collection loads were attempted for non-Mongo DB's in some instances 
Write pg_dump results directly to file instead of reading them via stdout to prevent process locks
Fixed isssue where copying a workspace on the Workspace View doesn't update the table of workspaces
V454
May 2, 2022
Features
Support applying generator values for PK fields in MongoDB to their FK fields
Bugs
Reduce Frequency of Upstream table filtering logs
Remove double scroll bar on foreign key page when viewing foreign keys
Improved database introspection queries on IBM Db2 iSeries
V453
April 29, 2022
Consistently clear local storage on authorization failures
Aggregate dropped row exceptions for more efficient logging
V452
April 28, 2022
Features
Improved performance for schema scans on MongoDB
Bugs
Minor fixes to workspace view
Fix for concurrent lambda function update issues
Fix brief non-ideal state while loading collections on MongoDB
V451
April 27, 2022
Features
Allow skipping of privacy scan on MongoDB using TONIC_TABLE_SKIP_REGEX environment variable
V450
April 27, 2022
Features
Improve usability of processing with Java SDK
Bugs:
Close connection to application database when not being used
Removed excess querying of database version
V449
April 26, 2022
Features
Improves performance of Name Generator on Spark / Databricks
Change Dremio schema input to Tag Input and add an informational popover
Bugs:
Do not null FKs between reference tables
Allow foreign keys to be added by editors
V448
April 26, 2022
Features
Improves performance of the SSN Generator on EMR Spark / Databricks
Bugs:
Keep the password field empty when copying from source database settings to destination database
Improved exception handling when masking and preserving destination tables
More quickly alert user when a cycle has been found in their relationships
Handle Postgres interval data types with a null boundary
Disable Add Foreign Keys tab for non-workspace owner users
V447
April 25, 2022
Features
The Workspaces view now supports bulk actions. Share, transfer, delete, and leave multiple workspaces by checking the boxes at the right end of each row, and choosing the desired action from the Actions menu.
Bugs:
Fixed an issue where Safari would fail to load the application
Fixes to constraint application parallelism
Breaking Changes
The response for the Workspace deletion endpoint (/api/workspace/{workspaceId}) has been modified slightly
V446
April 22, 2022
Features
Enable Timestamp Generator To Work Without Statistics For Spark Object Masks
When a mongo foreign key field has a primary key with a generator applied, replace the example data with a link to the collection with the primary key
Bugs
Fix for MySql generation hanging with empty password
V445
April 21, 2022
Features
Improves performance of Name Generator on EMR and Databricks
Bugs
Fix Display Bug in AI Synthesizer Configuration Panel
Fix application of constraints in Redshift
V444
April 20, 2022
Features
Add the version of the worker that ran a job to the job details
On the Mongo Collection View for foreign/primary keys: Added a hover tooltip for key icons, replaced the generator with a label
Refresh the last collection visited when a new foreign key file is uploaded and optimize single_doc foreign key
Bugs
Disable date truncation generator as a subgenerator for Spark
Fixes Notification service consuming all disk space when crashing
Show the appropriate Conditional sub-generator label instead of passthrough
Fixed in memory table query for SQL Server
V443
April 19, 2022
Features
Teardown and database creation performance improved for SQL Server
Skip preserve destination tables (just like truncated) for privacy scan
Bugs
Fix not allowing generators allowed on arrays that were scanned prior to v439
V442
April 18, 2022
Features
Adds support for events (dependent rows) to AI Synthesizer (formerly known as Smart Linking)
Allow wildcard (%) in schema name for Dremio
Bugs
Support Date Truncation and Timestamp Shift on Snowflake TIMESTAMP_TZ and Redshift TIMESTAMPTZ columns
Disable the generator dropdown in the UI for columns with both primary and foreign keys
MongoDB - Fix errors when linking Categorical Generators
Bug fixes for synthesis report
Fix data type mappings for Dremio Integer and Varchar
V441
April 14, 2022
Features
Display whether webhooks were used in job details
Terminology changes
Masked is changing to De-identify
Synthesized is changing to Scale
Truncated is changing to Truncate
Generator name change: Smart Linking is changing to AI Synthesizer
Bugs
Fixes to SQL script beautifier
V440
April 14, 2022
Features
Move single connections on connection pools behind a feature flag
Do not drop indexes on auto increment columns in MySQL to improve performance
Add type filter to JSON Mask generator
Remove unique constraint on workspace names and change default workspace name to "Untitled Workspace"
Display key icons for user uploaded foreign/primary key fields in Mongo Single document view
Bugs
Fix issue where 0 results in Workspaces table sometimes let you navigate to a negative page number
Fix issue where workspace permissions didn't update in the UI until a refresh
Fixes authentication error on Postgres when username has special characters
Fixed reopening open connections when TONIC_ENABLE_SINGLE_CONNECTION is false
Remove owner section from exported workspace body
Improve readability of workspace tags in small-width window sizes
V439
April 13, 2022
Features
Apply foreign key constraints serially on a different thread to avoid deadlocks & improve performance
Bugs
Clear bulk column search query on filters reset
Memory improvements to Mongo schema serialization
Better handling of schema name in Java SDK
V438
April 12, 2022
Features
Allow changes to order of JSON Mask sub-generators
Running a job is now disabled when the license expiration date has passed
Separate Workspace Sharing and Workspace Role-based Workspace access into different license features
Improve SQL Server performance by prefetching all tables & views
Bugs
When using Java SDK only show Java supported generators
V437
April 11, 2022
Features
Improves performance of Email Generator on EMR and Databricks
Add support from Dremio with Spark
Bugs
Allowing Kubernetes ImagePullBackOff for up to 5 minutes before throwing error when updating through UI
Fixes inability to run a data generator on Spark with default database as the source
V436
April 8, 2022
Features
Display estimated time remaining on row-based tasks in job details page
Display key icons for user uploaded foreign/primary key fields in Mongo hybrid documents
Bugs
Copy Generator in Mongo: Fix bug where if generator was copying a field that wasn't in the document it would throw exception
Add exception handling for SQL Server datetime columns to drop invalid records, better SQL Server XML type exception handling
Create source pool just before processing tables to avoid any timeouts in SQL Server
V435
April 8, 2022
Features
Display row based progress on jobs whenever possible
Statistics Jobs for the Java Spark SDK now appear in Jobs Table/Jobs Detail Page
Skip pre-job health check for PyML service when not needed
V434
April 7, 2022
Features
Added database metric sharing for AWS users
Don't null self-referential foreign keys on reference tables when subsetting
V433
April 7, 2022
Features
Improves performance of Email Generator on EMR and Databricks
Bugs
Better small-width responsiveness of tables in UI
V432
April 6, 2022
Features
Add synthesis report
Support for update via UI with Docker
Bugs
Prevent unexpected behavior of progress tracker when the system clock shifts
Fixes error when creating partition function with null range on SQL Server
Fix issues with loading screen showing when not needed
​
V431
April 5, 2022
Features
Add Remove Whitespace Transformer to Copy generator
Add ability to generate JSONPath expression for JSON Mask generator by clicking on preview JSON
Bugs
Improved support for large decimal numeric types in Postgres
V430
April 4, 2022
Features
Add exception handling for invalid SSH Tunnel private key format and list valid formats in tooltip
V429
April 4, 2022
Features
Apply constraints in parallel to data generation in Postgres
Bugs
Ensure workspace tag input closes when focus is lost from outside click
Remove foreign key checks when applying constraints in MySQL
V428
April 1, 2022
Features
Invalidate refresh tokens on logout, log in or out all open sessions (tabs) on login or logout, cross-tab inactivity timeout support
Improve load times on workspace pages
Bugs
Fix linking of non-consistent generators in Spark
V427
March 31, 2022
Features
Include workspace ID in URLs
Parallelize writes on MySQL passthrough tables
Allow decimals in subsetting percentages
Bugs
Don't allow duplicate entries of tables in table mode settings for workspace
Handle null values with Copy generator on MongoDB
Database query performance improvements
V426
March 30, 2022
Features
Improve MySQL generation performance by applying constraints in parallel
Bugs
Fix issue where indexes fail to be applied to converted columns on SQL Server
Fix issue with SSO account creation failing
V425
March 29, 2022
Features
Support Copy generator on MongoDB
Support parallelism of MongoDB schema scan
Support truncation mode on MongoDB
Bugs
Set Command Timeout in Postgres to prevent timeouts
V424
March 25, 2022
Features
Change password requirements
Length >= 12 characters
Must include number
Must include lowercase letter
Must include uppercase letter
Must include non-alphanumeric character
Bugs
Fixes to transfer workspace text
Allow opening Admin Panel when user has no workspaces
V423
March 25, 2022
Features
Improve constraint restoration performance on MySQL
Allow incremental tables in MySQL where all columns are keys
​
Bugs
Prevent app from crashing when admin status of user is unknown
V422
March 24, 2022
Features
Add login rate limitin3
Bugs
Fix issue with functions and procedures not being created in the destination on Oracle databases
V421
March 23, 2022
Features
On EMR and Databricks, write JSON files to JSON instead of Parquet
Improved handling of Latitude/Longitude with Smart Linking Generator
Add support for enum arrays columns in Postgres
Bugs
Fix issue with EMR cross account data generation failing to resolve the Glue database
V420
March 17, 2022
Features
Workspace Updates - Admin View All, Leave, Share & Transfer V2, Owner Filter, and more 
Parse as bigint if decimal fails 
Oracle Whitelist support in BuildDb 
Initialize after log sink reset 
Replace slashes with underscores in build versioning
Add 'No generators' filter for bulk table view 
Add progress tracker for SQL server post-data script generation 
Allow support for views without fully qualified tables in Redshift 
Improve performance of listing workspaces
Improve performance of the Constant Generator on EMR and Databricks 
Sensitive columns now appear in the privacy hub for Mongo workspaces 
Allow processing of decimals with a scale of 1 up to 9223372036854775807 in Postgres 
Check for cloudsqlsuperuser permission on GCP Cloud SQL destination postgres databases
Bugs
Address Lambda Function Setup bug
Improve Regex Mask Display of Empty Matches and values with new lines
Do not fail completely when SSO groups do not resolve
Misc Redshift View fixes
Fix issue where column can't be in schema changes and show as sensitive
Fix issue where some downstream tables are missing their foreign keys in the destination database when using subsetting
Fix issue with shifting table size on data preview in database view
Prevent error from occurring when MongoDB collection size is too large
V412
March 10, 2022
Features
Added support for user defined enum types as keys in subsetting
Simplify schema diff changes error toast on generation
Show more unprotected columns button mongo privacy hub
Added filtering on custom value generator dropdown
Add ConsistentOn to AdditiveNoiseGenerator
Enabling Timeshift Generator on Compound Primary Keys
Regex Mask Generator supports matching empty capture groups
Accommodate OFFSET in where clauses in SQL Server subsetting
Miscellaneous workspace config improvements
Add support Azure Sql with SqlServerDatabaseCreator
Add progress tracking for mongo schema scans
Add syntax highlighting for where clauses in subsetting
Add support for integer primary keys on smallint
Bug fixes
Fixed error message for MySQL data type not found
Fix Postgres Test Connection bug
Fix Workspace patches after rename or modify tags
Fix Unix Timestamp empty column bug
Throw OperationCanceledException instead of CancelTonicJobException on cancellation request
Fix Redshift view replication missing schema
V401
February 24, 2022
Features
Oracle perf improvements
Add new range options for integer pk generator
CNPJ and CPF generator performance improvements
Updates Regex Mask Generator to support replacing all matches
Start the progress token for passthrough tables when there are no passthrough tables with references
Add support for Timestampshift for spark
Add warning system to Privacy Scan
Add ability to update via UI for Admin users for Kubernetes
Regex mask generator with Custom Value Processor
Check for cancellation and report progress on text copy in Postgresql
Miscellaneous perf improvements
Bug fixes
Start the progress token for passthrough tables when there are no passthrough tables with references
Treat non-standard oracle error code 5 not as an error
Fix grabbing workspace users for comments
Dispose the source connection instead of just closing in Postgres
Fix for test discovery exception
Fix postgres sequences behavior
Object mask bulk view fix
Miscellaneous typos fixed
Fixed database type for BigQuery test connection
V391
February 15, 2022
Features
Relax unique index constraint on username
Added tonic version in containers as an environment variable
Adding logic to bypass Fallback generators for Spark databases
Support case insensitive name consistency
Disable dropping replacements on missing columns for mongo
Update Oracle driver
Enable additive noise generation for strings in mongo
Add support for unsigned 64-bit integer columns in integer key generator
Add random timestamp and timestamp shift generators as options for integer columns (for unix timestamp columns)
Add checks for min/maxTime on timestamp and min/maxDate on dates
Adding null ref check for fallback generation option check during preview
Revamped workspace configuration screen
Support for Decimal on Spark (when using C# UDFs)
Allow Regex Generator on PKs
Added new environment variable to filter tables ingested by UI for Oracle
Misc perf improvements
Bug fixes
Fix mongo view more values issues
Disable dropping replacements on missing columns for mongo
Fix Workspace specified via API without DBs can be modified in UI
Fixing Random Timestamp generator on Java
Date truncation generator bug fixes
V383
February 8, 2022
Features
Better error exception handling for number conversions in Mongo
Added loading indicator when viewing single Mongo document
Improved Mongo performance and logging
Added table filter validation for EMR
Handling numbers as doubles instead of integers in Snowflake
Added aiven.io system schema to Postgres
Added the Geo Generator for latitude and longitude data
Added support for materialized view in Redshift
Bug fixes
Fixed array regex mask generator with correct subgenerator options
Fixed conditional generator by adding binding
V378
February 4, 2022
Features
Updated to dotnet 6
Moved FK import to FK page
Added support for tinyint in MySQL as a primary key
Added table partition filtering for EMR
Enabled Regex Mask generator on EMR 
Added support decimals with precision and scale for EMR 
Moved Tonic version number to hamburger menu
Added job sorting by start time
Ability to retrieve Kubernetes logs via API
Improved behavior of upstream reference tables for subsetting
Improved logging for subset filtering
Updated sensitivity scan to better reflect job status
Improved privacy scan performance for Mongo
Introduced automated schema checks (not for Mongo)
Added HTML Mask generator
Added Additive Noise Generator
Added CNPJ generator
Added Array Regex Mask generator
Deprecated endpoints
Updates to the foreign key feature impact the API endpoints used to fetch, upload, and delete the Foreign Key file. Two previous API endpoints are now marked as deprecated:
DELETE /api/datasource/delete_fkupload has been marked as deprecated and replaced by DELETE /api/workspace/{workspaceId}/foreign-key/
GET /api/datasource/download_fkupload/ has been marked as deprecated and replaced by GET /api/workspace/{workspaceId}/foreign-key/
The main change between both APIs is the switch to using workspaceId to fetch the Foreign Key file instead of the datasourceId.
Both are slated to be removed after 2022-06-01.
Bug fixes
Fix number parsing issues in JSON mask generator
Fix MySQL parsing of auto_increment columns
Eliminated re-querying of data type on EMR Spark
Fixed issue viewing Mongo workspace when there's an empty collection
Fixed for TruncatedDateGenerator when used as SubGenerator
Fixed data passed to SubGenerator in RegexMaskGenerator
V363
January 21, 2022
Features
Show tooltip for sidebar menu items when collapsed
Allow clicking of Queue Status and Download Job logs buttons on Jobs page
Added privacy scans to Jobs page
Added quick navigation to Jobs page from Workspaces tables
Added support for random timestamp generator for EMR
Improved Hstore generation performance
Added job-specific URLs
Webhooks are triggered when jobs are manually cancelled
Added warning for missing use statements in MySQL post-job scripts
Make JSON mask generator respect ordering with multiple subgenerators
​
Bug fixes
Fix Postgres binary timeout issues
Improved error handling & logging when applying Foreign Keys on subsetted SQL Server
Filter out constraints with same name as foreign key
V355
January 7, 2022
Features
Added Privacy Report CSV download for Preview (to Privacy Hub) and Generation Report (to Job Details).
​
​
​
​
Added support for preserve destination in Oracle. 
Added support for temporal tables on Sql Server. 
Added support for read parallelism via paginated queries. 
Updated fallback generators to filter by column details and to remove text-mask generator. 
Disable page parallelism when pageSize too small. 
Add styles for signup divider.
Bug Fixes 
Fix AuroraMySql hang. 
Clarify Subsetting UI. 
SQL Server Constraint Performance Improvements. 
Fixes for PII Scan. 
Fix Jobs Foreign Key Migration Issues. 
Log missing env var once per process. 
Improvements to JSON Mask Performance using JToken creation to improve data type handling. 
Text mask performance improvements. 
Fix for Polling TS error.
V350
December 23, 2021
Features 
Ability to specify a fallback generator on XML and JSON generators 
Ability to exclude multiple email domains in the email generator 
Update column schema to preserve dropped-date and isDropped status 
Change job details checkmarks to green Prevent idle session timeout in Postgres 
Drop rows if decimal parse fails in Postgres Add pre-job check logging
Ability to connect to MongoDB via connection string
Allow incremental mode on tables where the PK is also an FK
Retry polling job updates and cancel job checks when there's connectivity issues with the Tonic DB
Add warning message when fails to restore sequence on destination data
Always show job status warning icon when applicable
Bug Fixes 
Fixed importing workspace json caused schema change alerts
Fixed Timestamp Shift for DateTimeOffset
Fixed for how we count the number of partitions on tables in SQL Server
Properly escaping MySQL Username/Password encoding
V344
December 9, 2021
Features 
Update Workspace Users/Shares via web sockets
Provide job summary updates via web sockets
Make jump to page in Workspaces table more discoverable
Display which Tonic tier the customer is on
Wrap DNS resolution in try-catch block to avoid errors upstream
Expose ports 443 and 80 in Dockerfiles for applicable containers
Update session variables to enforce autocommit for MySQL
Parallelization of row processing for improved job performance
Bug Fixes 
Removed random sampling in ConsistentOn generators
Fixed newly edited Tags of active workspace not being exported
Updated Oracle writer path for job improvement
Improved error message when test connection fails
Made error message more clear for generators that perform frequency samples
Fixed DB2 connection spinning indefinitely via timeout
V341
December 3, 2021
Features 
Added incremental mode support for MySQL