Structural Cloud API docs
Information portal
Tonic.ai

V52 - V97

V97

July 15th, 2020

Features

  • Tables on Sql Server can now be updated in an incremental fashion where only the changes since the last generation are processed. More details here.

  • Added support for customer categorical generator in synthesize mode.

  • Date truncation generator has been added so you can truncate dates to a specified date part.

  • Added support for timestamp ranges in Postgres.

  • Phone number generator now supports multinational phone numbers. The output phone number will match the country/region of the input phone number.

  • Integer key generator can now operate on an Int64/Long column.

Bugs

  • Fixed issue where a specifically sized payload could cause the API request to fail.

  • Geo generator properly passes through null values.

  • Added back remove table API.

  • Tables containing unicode characters now work in preserve destination mode on Sql Server.

  • Improved handling of transaction scopes for Sql Server.

  • Fixed issue where generators applied through the conditional generator sometimes used their default options during generation.

  • Generators added through autodetect are now able to be marked consistent/deferentially private.

  • Resolved multithreading issue using the key generators on tables with linked foreign keys.

  • Sql server date columns no longer show time in the table view.

V93

July 1st, 2020

Features

  • Added ability for full name generators to be consistent with partial name generators.

  • Generator popovers now scroll after reaching a certain height.

  • Add Random Double generator.

  • Add ability to delete Tonic account.

  • Improved job progress tracking for Oracle.

  • Allow workspaces to be imported and exported.

  • Added support for conditional generator on Sql Server image columns.

  • Negative numbers can now be used in the constant generator with numeric columns.

  • Added support for timezone arrays in Postgres.

Performance Improvements

  • Significant performance increase when using the continuous generator.

Bug Fixes

  • No longer displaying out of date conditional generator configuration in the collapsed view.

  • Fixed issue with not being able to backspace the constant value on a constant generator applied to a numeric column.

  • User's can no longer duplicate subset targets on the subset configuration page.

  • Removed conditional generator as a sub-generator choice when using the xml or json mask generators.

  • Fixed issue where consistency could not be applied to a generator inside the conditional generator.

  • Loading indicators on the table view are now more consistent.

  • Continuous generator now works with smaller partitions when generating statistics.

  • Job details page now scrolls when number of steps is too large for the screen area.

  • Fixed issue where adding a custom categorical generator inside the JSON mask generator could cause an error.

  • Removing a table from the Schema Changes page now properly removes it from the subsetting configuration.

  • Disabled including tables out of subset was enabled when subsetting was not.

  • Fixed broken link for API documents on Job Details page.

  • Fixed issue where sequential integer generator didn't reset itself between generations.

  • Forced browser to get new version of assets after each release.

  • Add sql injection safety to generators that partition.

  • Fixed issue with switching between workspaces on the jobs view.

  • Moved popovers in database view to not be blocking other columns.

  • Fix issue with attribute info generator failing on empty attributes.

  • Stopped job progress from updating after job is complete.

  • Fixed issue with canceled jobs still running if they were cancelled before they could run.

V90

June 17th, 2020

Features

  • Added the ability to specify a domain and enable consistency on the unique email generator

  • Conditional generator can now be used with the unique email generator

  • Added confirmation step for canceling a generation.

  • Added support for JSON and JSONB arrays in Postgres.

Performance Improvements

  • Updates to your workspace are now done through Json Patch.

Bug Fixes

  • Fixed issue where a table was unable to be switched to synthesized mode.

  • Fixed issue where min and max on the random integer generator were not editable.

  • Fixed issue where multiple changes might not be saved properly if executed within a few milliseconds of each other.

  • Fixed issue where constraints failed to apply to a large Sql server database.

  • Fixed display issue where the password input appeared to be filled in on the destination database connection screen.

  • Swagger docs no longer report enum fields as integers.

  • Fixed issue where edit, copy, and delete workspace buttons where still clickable even when disabled.

  • Decreased clickable area on consistency and differential privacy switches.

  • Fixed issue with NaN values in Postgres double fields.

V87

May 29th, 2020

Features

  • A new subsetting option allows you to process tables that are not included in your subset.

  • Several Conditional Generator improvements:

    • Conditional Generator can now filter rows by a regular expression.

    • Conditional Generator now supports the following additional generators: Categorical, Custom Categorical, Alphanumeric Key, Numeric String Key, Integer Key.

    • 'Is null' and 'is not null' operators added to conditional generator

  • Custom Categorical Generator now supports numeric types.

  • Random timestamp generator can now be added to text columns.

  • Password managers are now prevented from interfering with the database connection form.

  • Improved logging for constraint application on Sql server.

  • Renamed 'private' to 'sensitive' when referring to a column with data that needs to be protected.

  • Column headers are now red when a column is sensitive but not protected.

Performance Improvements

  • Workspace updates no longer happen on keystroke and will now wait until you exit the field or popup.

  • Changed the way the application loads workspaces to increase performance for large workspaces.

Bug Fixes

  • Fix display issue with Custom Categorical Generator when there were no categories.

  • Fixed error message when applying random timestamp generator.

  • Fixed issue with column editor size changing when marking a column as sensitive.

  • Fixed an issue with xml columns on Sql server

V84

May 12th, 2020

Features

  • Tables can now be filtered by their current mode in the database view.

  • Added support for the 'contains' operator on the conditional generator.

  • Company name generator now supports consistency.

  • Added version check when editing a workspace to better support multitab/multibrowser use. Multiple users can now edit the same workspace without worrying about race conditions. Note: this is a first step, more multiuser features are in development.

Performance Improvements

  • Improved speed of tables in mask mode by fixing issue introduced in v76.

Bug Fixes

  • Fixed issue with applying a large number of constraints with Sql Server.

  • Random boolean, IP address, and random integer generators now work correctly with the conditional generator

  • Tasks that complete immediately now display correctly in the job details page.

  • Fixed display issue with data preview button on privacy hub.

V83

May 8th, 2020

Features

  • Added a generator for shipping container codes.

  • Custom Categorical Generator now works on json fields.

  • Destination database names can now differ from the source for Sql Server.

  • Job completion time has been added to the Job Details page.

  • Additional diagnostic tools added to docker containers.

Performance

  • Column search performance in bulk editor significantly improved.

  • Removed Intercom

Bug Fixes

  • Preserving a partitioned table on Sql Server no longer causes an error.

  • Fixed issue with the Categorical Generator when the table is empty.

  • Fixed issue with the last line break sometimes being removed from the category list on the Custom Category Generator.

  • Fix preserve tables for non-Oracle databases.

  • Styling fixes for Conditional Generator when used in Firefox.

  • Fixed issue where insert into an XML column can fail on Sql Server when the payload has a large attribute, too many nodes, or too much nesting.

V80

Apr 28, 2020

Features

  • You can now apply generators conditionally based on the values within the column.

  • Added support for array types in Postgres

  • Added a custom categorical generator, which allows you to specify a list of categories for a given column

  • Starting a data generation via the API will now return the ID of the job

  • Primary key generators can now be applied in the bulk edit view.

  • Added additional information on the job details page and in the logging, especially around subsetting.

  • Return existing refresh tokens if they exist to better enable multiple open sessions for the same account.

Performance

  • Improved the speed of the integer primary key generator

  • Improved subsetting performance when subset targets are referenced by large tables

Bug Fixes

  • Tonic will now ignore tables created by unsupported database add-ons that previously would require the user to truncate.

    • You may see these tables show up as a schema change alert. Most common ones are spatial_ref_sys and sysdiagrams.

  • Skip some the relationship checks needed for synthesis when table mode is not set to synthesize.

  • Fixed display issue where primary key would show as needing a generator for synthesis mode.

  • Fixed issue where the table drop down on the subset configuration view sometimes rendered in the wrong location.

  • Json path generators will no longer overwrite objects or arrays.

  • Fixed issue where identity sequence was not being set correctly on the destination table.

  • Fix issue when empty strings are present in a column with the numeric string or alphanumeric string primary key generators attached

V74

Apr 14, 2020

Features

Bug Fixes

  • Replaced browser alert with a toast notification when a job fails to be scheduled

  • Oracle now returns columns in the order of their ordinal position

  • Adjusted sizing on subset dropdown

  • Fixed issue where the progress tracker could fail to write events near the end of a generation.

V73

Apr 6, 2020

Features

  • Full Oracle support is out of beta

  • Users can now generate API tokens to more easily authenticate with Tonic's APIs.

  • Completion times now show on job details.

  • Subset configuration area now scrollable.

  • Added a check to ensure destination version is not older than source version for MS Sql Server.

Security

  • Upgraded Java version on docker images to resolve security issues.

Bug Fixes

  • Fixed issue where consistency did not work across multiple workers.

  • Ensure that subset configuration is validated when the subset configuration page has not been loaded yet.

  • Fixed issue where Privacy Hub could continue to refresh even after a scan was complete.

  • Incorrect generators are no longer showing in the MS Sql Server image data type.

  • Fixed magnifying glass placement on Privacy Hub on field hover.

  • Fixed various issues with toast notifications. Reset scrollbar position on table selection change in bulk editor.

V72

Mar 25, 2020

Features

  • Improved logging when testing a connection

  • Added MySql 8 compatibility for our collated queries.

  • Complete overhaul of progress tracking and the job details page. Jobs Details now has progress bars, spinners, and various icons to present the current status of a job.

Performance

  • Index restoration parallelized on MySql.

  • Foreign keys added to a MySql table are now added with 'foreign_key_checks=0'.

  • Tables marked as 'Preserve Destination' now complete significantly faster on SQL Server.

Security

Bug Fixes

  • Fixed issue in MySql Restore Foreign Key query

  • Fixed issue when user changes a JsonMask or XmlMask generator to other generator types

  • Tiny Int not treated as Boolean in MySQL

  • Fixed MySql issue with preserved tables having the same name in different schemas

  • PII Scanner now picks up on zip code data by column name.

  • PII detection using lookup tables now removes leading and trailing whitespaces and converts all values to lowercase. This fixes our PII detection for cities and states.

  • Test Connection button no longer holds on to table locks on MySql databases.

  • Workspace Name field now receives focus when workspace editor is opened.

  • Enter key can now be used to save Workspace Editor.

  • Allow UUID primary keys as downstream type in subsetting mode.

  • Fix issue with GUID primary keys when subsetting is enabled.

  • Long columns names now render properly on Privacy Hub.

  • Fixed issue with bulk adding consistent generators.

  • Account creation no longer hangs when server has no internet connection.

  • Fixed spacing issue with JSON Mask Generator interface.

V66

Mar 10, 2020

Features

  • Bulk operations on multiple columns in the Database View:

    • Add and remove generators

    • Change privacy status

  • UI now distinguishes between primary (gold) and foreign keyed (black) columns

  • Advanced search for columns in the Database view.

  • Generator select popover on database view is now at feature parity with same popover in Table View.

  • Ability to manually mark columns as private

  • Columns with private data that are not protected are flagged as red (Database View only) and have a warning in the generator selector

  • tonic_worker now has a /health endpoint

  • Phone Number Generator supports integer columns

  • Differential Privacy warnings now show for Categorical Generator when being used on JSON and XML columns

  • Ability for user to provide their own secret to be used for encrypting both JWTs and database connection info

  • Ability to change password (under User Settings in the hamburger menu located on the right of the menu bar)

Performance

  • Preserved tables in MySQL are no longer copied into the preserve schema via SELECT INTO. Instead they are RENAME'd

  • No longer drop and restore primary keys and indexes in MySQL

Bug Fixes

  • Better support for 3 byte unicode characters when writing data to masked tables in Postgres

  • Fixed rare concurrency exception being thrown in progress tracker when jobs are running

  • Fixed malformed query in Sql Server which is used when subsetting

  • MySQL handling of character

  • Additional logging of MySQL index handling

V62

Feb 19, 2020

Features

  • Bulk operations on multiple tables in the Database View

  • Where clause in subset - in addition to a target %, you can now specify a custom where clause as the target for subsetting

  • URL Generator is more private

  • PII type is shown in Privacy Hub

Bugs

  • Styling issues for Chrome and Firefox

  • Event Ordering defaults corrected

  • Columns with nothing put nulls gave bad data previews

V61

Feb 11, 2020

Features

  • Health Endpoints /health added for web server and PII detection server

  • Non-null values are no longer shown during data preview

  • Sort by column name in privacy hub

  • Ability to delete a workspace

  • Columns part of truncated tables are now labelled as such in the Database view

  • PII Detection now uses the Character Scramble on suspected phone numbers

  • Support for Noda time in PostgresSQL and throughout the product

  • Other columns no longer shows in the database view

  • Email Generator now has an excluded domain filter

  • Subset with Foreign Key Upload File

Performance

  • Database View UI rendering improvements

  • UI improvements for MySQL databases with 1000s of tables

  • Ability to process tables in parallel across all databases. This setting is controlled by the TONIC_TABLE_PARALLELISM environment variable

  • Faster reads from source database. This affects SQL Server, MySQL, and Postgres and should see improvements for reads from the source database across the board

  • Optimization for SQL Server to improve writes

  • Better distribution of load to workers

Bug Fixes

  • Fixed bug in XML generation for Postgres

  • Reduce rate of false positive when detecting zip codes

  • Subsetting now properly supports self-referential foreign keys

  • SQL Server uniqueidentifier columns now properly work with the UUID generator

  • Privacy hub has better styling for long column names

  • Fixed issues related to change of focus on Target % input on subset configuration page

V57

Jan 2, 2020

Features

  • Support consistency for the email generator on a unique column

  • Rename and consolidation of docker images. Please contact support@tonic.ai before upgrading for a new docker-compose file

  • Massive refactor of subsetting, mostly behind the scenes. User facing improvements:

    • Support for all databases (not just Postgres and MySQL)

    • Heroku no longer requires a temporary database

    • Performance boost

  • Categorical generator now supports differential privacy (https://www.tonic.ai/post/differential-privacy-comes-to-tonic/)

Bug Fixes

  • Fix for certain unicode characters in table/schema names

  • Fix for character scramble on numerical columns

  • Several fixes for air gapped deployments

  • Significant performance improvements for the privacy scan

V55

Dec 27, 2019

Features

  • Suggested Generators now show privacy scan suggested generator when applicable

Bug Fixes

  • Better support for SSH Tunneling over private IPs

V52

Nov 15, 2019

Features

  • Generator improvements:

    • Email Generator can be used on columns with a unique constraint

    • Character scramble now supports consistency

    • Filename and Email generators make use of character scramble as opposed to character substitution (more secure). They both also now support consistency.

  • More improvements to PII Detection, specifically speed, resilience, and better name detection

  • Ability to duplicate workspaces (see screenshot below)

  • Show generators applied on column in bulk view when table is truncated

  • Support for SSH Bastion when connecting to source and output database

  • Added new ENV variable for controlling parallelism for MySQL generation

Bug Fixes

  • Fixed bad description on FK Relationships upload help text

  • Fixed layout issue in Replacement Panel

  • Fixed issue with ICD10 PII detection

  • Fixed routing issues on Heroku

Last updated