V52 - V97
July 15th, 2020
Features
- Tables on Sql Server can now be updated in an incremental fashion where only the changes since the last generation are processed. More details here.
Incremental Updates
- Added support for customer categorical generator in synthesize mode.
- Date truncation generator has been added so you can truncate dates to a specified date part.
- Added support for timestamp ranges in Postgres.
- Phone number generator now supports multinational phone numbers. The output phone number will match the country/region of the input phone number.
- Integer key generator can now operate on an Int64/Long column.
Bugs
- Fixed issue where a specifically sized payload could cause the API request to fail.
- Geo generator properly passes through null values.
- Added back remove table API.
- Tables containing unicode characters now work in preserve destination mode on Sql Server.
- Improved handling of transaction scopes for Sql Server.
- Fixed issue where generators applied through the conditional generator sometimes used their default options during generation.
- Generators added through autodetect are now able to be marked consistent/deferentially private.
- Resolved multithreading issue using the key generators on tables with linked foreign keys.
- Sql server date columns no longer show time in the table view.
July 1st, 2020
Features
- Added ability for full name generators to be consistent with partial name generators.
- Generator popovers now scroll after reaching a certain height.
- Add Random Double generator.
- Add ability to delete Tonic account.
- Improved job progress tracking for Oracle.
- Allow workspaces to be imported and exported.
- Added support for conditional generator on Sql Server image columns.
- Negative numbers can now be used in the constant generator with numeric columns.
- Added support for timezone arrays in Postgres.
Performance Improvements
- Significant performance increase when using the continuous generator.
Bug Fixes
- No longer displaying out of date conditional generator configuration in the collapsed view.
- Fixed issue with not being able to backspace the constant value on a constant generator applied to a numeric column.
- User's can no longer duplicate subset targets on the subset configuration page.
- Removed conditional generator as a sub-generator choice when using the xml or json mask generators.
- Fixed issue where consistency could not be applied to a generator inside the conditional generator.
- Loading indicators on the table view are now more consistent.
- Continuous generator now works with smaller partitions when generating statistics.
- Job details page now scrolls when number of steps is too large for the screen area.
- Fixed issue where adding a custom categorical generator inside the JSON mask generator could cause an error.
- Removing a table from the Schema Changes page now properly removes it from the subsetting configuration.
- Disabled including tables out of subset was enabled when subsetting was not.
- Fixed broken link for API documents on Job Details page.
- Fixed issue where sequential integer generator didn't reset itself between generations.
- Forced browser to get new version of assets after each release.
- Add sql injection safety to generators that partition.
- Fixed issue with switching between workspaces on the jobs view.
- Moved popovers in database view to not be blocking other columns.
- Fix issue with attribute info generator failing on empty attributes.
- Stopped job progress from updating after job is complete.
- Fixed issue with canceled jobs still running if they were cancelled before they could run.
June 17th, 2020
Features
- Added the ability to specify a domain and enable consistency on the unique email generator
- Conditional generator can now be used with the unique email generator
- Added confirmation step for canceling a generation.
- Added support for JSON and JSONB arrays in Postgres.
Performance Improvements
- Updates to your workspace are now done through Json Patch.
Bug Fixes
- Fixed issue where a table was unable to be switched to synthesized mode.
- Fixed issue where min and max on the random integer generator were not editable.
- Fixed issue where multiple changes might not be saved properly if executed within a few milliseconds of each other.
- Fixed issue where constraints failed to apply to a large Sql server database.
- Fixed display issue where the password input appeared to be filled in on the destination database connection screen.
- Swagger docs no longer report enum fields as integers.
- Fixed issue where edit, copy, and delete workspace buttons where still clickable even when disabled.
- Decreased clickable area on consistency and differential privacy switches.
- Fixed issue with NaN values in Postgres double fields.
May 29th, 2020
Features
- A new subsetting option allows you to process tables that are not included in your subset.
- Several Conditional Generator improvements:
- Conditional Generator can now filter rows by a regular expression.
- Conditional Generator now supports the following additional generators: Categorical, Custom Categorical, Alphanumeric Key, Numeric String Key, Integer Key.
- 'Is null' and 'is not null' operators added to conditional generator
- Custom Categorical Generator now supports numeric types.
- Random timestamp generator can now be added to text columns.
- Password managers are now prevented from interfering with the database connection form.
- Improved logging for constraint application on Sql server.
- Renamed 'private' to 'sensitive' when referring to a column with data that needs to be protected.
- Column headers are now red when a column is sensitive but not protected.
Performance Improvements
- Workspace updates no longer happen on keystroke and will now wait until you exit the field or popup.
- Changed the way the application loads workspaces to increase performance for large workspaces.
Bug Fixes
- Fix display issue with Custom Categorical Generator when there were no categories.
- Fixed error message when applying random timestamp generator.
- Fixed issue with column editor size changing when marking a column as sensitive.
- Fixed an issue with xml columns on Sql server
May 12th, 2020
Features
- Tables can now be filtered by their current mode in the database view.
- Added support for the 'contains' operator on the conditional generator.
- Company name generator now supports consistency.
- Added version check when editing a workspace to better support multitab/multibrowser use. Multiple users can now edit the same workspace without worrying about race conditions. Note: this is a first step, more multiuser features are in development.
Performance Improvements
- Improved speed of tables in mask mode by fixing issue introduced in v76.
Bug Fixes
- Fixed issue with applying a large number of constraints with Sql Server.
- Random boolean, IP address, and random integer generators now work correctly with the conditional generator
- Tasks that complete immediately now display correctly in the job details page.
- Fixed display issue with data preview button on privacy hub.
May 8th, 2020
Features
- Added a generator for shipping container codes.
- Custom Categorical Generator now works on json fields.
- Destination database names can now differ from the source for Sql Server.
- Job completion time has been added to the Job Details page.
- Additional diagnostic tools added to docker containers.
Performance
- Column search performance in bulk editor significantly improved.
- Removed Intercom
Bug Fixes
- Preserving a partitioned table on Sql Server no longer causes an error.
- Fixed issue with the Categorical Generator when the table is empty.
- Fixed issue with the last line break sometimes being removed from the category list on the Custom Category Generator.
- Fix preserve tables for non-Oracle databases.
- Styling fixes for Conditional Generator when used in Firefox.
- Fixed issue where insert into an XML column can fail on Sql Server when the payload has a large attribute, too many nodes, or too much nesting.
Apr 28, 2020
Features
- You can now apply generators conditionally based on the values within the column.
Conditional Generator
- Added support for array types in Postgres
- Added a custom categorical generator, which allows you to specify a list of categories for a given column
Custom Categorical Generator
- Starting a data generation via the API will now return the ID of the job
- Primary key generators can now be applied in the bulk edit view.
- Added additional information on the job details page and in the logging, especially around subsetting.
- Return existing refresh tokens if they exist to better enable multiple open sessions for the same account.
Performance
- Improved the speed of the integer primary key generator
- Improved subsetting performance when subset targets are referenced by large tables
Bug Fixes
- Tonic will now ignore tables created by unsupported database add-ons that previously would require the user to truncate.
- You may see these tables show up as a schema change alert. Most common ones are spatial_ref_sys and sysdiagrams.
- Skip some the relationship checks needed for synthesis when table mode is not set to synthesize.
- Fixed display issue where primary key would show as needing a generator for synthesis mode.
- Fixed issue where the table drop down on the subset configuration view sometimes rendered in the wrong location.
- Json path generators will no longer overwrite objects or arrays.
- Fixed issue where identity sequence was not being set correctly on the destination table.
- Fix issue when empty strings are present in a column with the numeric string or alphanumeric string primary key generators attached
Apr 14, 2020
Features
- Support hex strings for inserting into binary columns in Sql Server
Bug Fixes
- Replaced browser alert with a toast notification when a job fails to be scheduled
- Oracle now returns columns in the order of their ordinal position
- Adjusted sizing on subset dropdown
- Fixed issue where the progress tracker could fail to write events near the end of a generation.
Apr 6, 2020
Features
- Full Oracle support is out of beta
- Completion times now show on job details.
- Subset configuration area now scrollable.
- Added a check to ensure destination version is not older than source version for MS Sql Server.
Security
- Upgraded Java version on docker images to resolve security issues.
Bug Fixes
- Fixed issue where consistency did not work across multiple workers.
- Ensure that subset configuration is validated when the subset configuration page has not been loaded yet.
- Fixed issue where Privacy Hub could continue to refresh even after a scan was complete.
- Incorrect generators are no longer showing in the MS Sql Server image data type.
- Fixed magnifying glass placement on Privacy Hub on field hover.
- Fixed various issues with toast notifications. Reset scrollbar position on table selection change in bulk editor.
Mar 25, 2020
Features
- Improved logging when testing a connection
- Added MySql 8 compatibility for our collated queries.
- Complete overhaul of progress tracking and the job details page. Jobs Details now has progress bars, spinners, and various icons to present the current status of a job.
Performance
- Index restoration parallelized on MySql.
- Foreign keys added to a MySql table are now added with 'foreign_key_checks=0'.
- Tables marked as 'Preserve Destination' now complete significantly faster on SQL Server.
Security
- Update third party packages to fix various security vulnerabilities.
Bug Fixes
- Fixed issue in MySql Restore Foreign Key query
- Fixed issue when user changes a JsonMask or XmlMask generator to other generator types
- Tiny Int not treated as Boolean in MySQL
- Fixed MySql issue with preserved tables having the same name in different schemas
- PII Scanner now picks up on zip code data by column name.
- PII detection using lookup tables now removes leading and trailing whitespaces and converts all values to lowercase. This fixes our PII detection for cities and states.
- Test Connection button no longer holds on to table locks on MySql databases.
- Workspace Name field now receives focus when workspace editor is opened.
- Enter key can now be used to save Workspace Editor.
- Allow UUID primary keys as downstream type in subsetting mode.
- Fix issue with GUID primary keys when subsetting is enabled.
- Long columns names now render properly on Privacy Hub.
- Fixed issue with bulk adding consistent generators.
- Account creation no longer hangs when server has no internet connection.
- Fixed spacing issue with JSON Mask Generator interface.
Mar 10, 2020
Features
- Bulk operations on multiple columns in the Database View:
- Add and remove generators
- Change privacy status
Bulk Column Operations
- UI now distinguishes between primary (gold) and foreign keyed (black) columns
- Advanced search for columns in the Database view.
Advanced Search
- Generator select popover on database view is now at feature parity with same popover in Table View.
- Ability to manually mark columns as private
- Columns with private data that are not protected are flagged as red (Database View only) and have a warning in the generator selector
- tonic_worker now has a /health endpoint
- Phone Number Generator supports integer columns
- Differential Privacy warnings now show for Categorical Generator when being used on JSON and XML columns
- Ability for user to provide their own secret to be used for encrypting both JWTs and database connection info
- Ability to change password (under User Settings in the hamburger menu located on the right of the menu bar)
Performance
- Preserved tables in MySQL are no longer copied into the preserve schema via SELECT INTO. Instead they are RENAME'd
- No longer drop and restore primary keys and indexes in MySQL
Bug Fixes
- Better support for 3 byte unicode characters when writing data to masked tables in Postgres
- Fixed rare concurrency exception being thrown in progress tracker when jobs are running
- Fixed malformed query in Sql Server which is used when subsetting
- MySQL handling of character
- Additional logging of MySQL index handling
Feb 19, 2020
Features
- Bulk operations on multiple tables in the Database View
- Where clause in subset - in addition to a target %, you can now specify a custom where clause as the target for subsetting
- URL Generator is more private
- PII type is shown in Privacy Hub
Bugs
- Styling issues for Chrome and Firefox
- Event Ordering defaults corrected
- Columns with nothing put nulls gave bad data previews
Feb 11, 2020
Features
- Health Endpoints /health added for web server and PII detection server
- Non-null values are no longer shown during data preview
- Sort by column name in privacy hub
- Ability to delete a workspace
- Columns part of truncated tables are now labelled as such in the Database view
- PII Detection now uses the Character Scramble on suspected phone numbers
- Support for Noda time in PostgresSQL and throughout the product
- Other columns no longer shows in the database view
- Email Generator now has an excluded domain filter
- Subset with Foreign Key Upload File
Performance
- Database View UI rendering improvements
- UI improvements for MySQL databases with 1000s of tables
- Ability to process tables in parallel across all databases. This setting is controlled by the TONIC_TABLE_PARALLELISM environmental variable
- Faster reads from source database. This affects SQL Server, MySQL, and Postgres and should see improvements for reads from the source database across the board
- Optimization for SQL Server to improve writes
- Better distribution of load to workers
Bug Fixes
- Fixed bug in XML generation for Postgres
- Reduce rate of false positive when detecting zip codes
- Subsetting now properly supports self-referential foreign keys
- SQL Server uniqueidentifier columns now properly work with the UUID generator
- Privacy hub has better styling for long column names
- Fixed issues related to change of focus on Target % input on subset configuration page
Jan 2, 2020
Features
- Support consistency for the email generator on a unique column
- Rename and consolidation of docker images. Please contact [email protected] before upgrading for a new docker-compose file
- Massive refactor of subsetting, mostly behind the scenes. User facing improvements:
- Support for all databases (not just Postgres and MySQL)
- Heroku no longer requires a temporary database
- Performance boost
- Categorical generator now supports differential privacy (https://www.tonic.ai/post/differential-privacy-comes-to-tonic/)
Bug Fixes
- Fix for certain unicode characters in table/schema names
- Fix for character scramble on numerical columns
- Several fixes for air gapped deployments
- Significant performance improvements for the privacy scan
Dec 27, 2019
Features
- Support for SQL Server partition functions and partition schemes
- Data Preview in Database View
- Suggested Generators now show privacy scan suggested generator when applicable
Bug Fixes
- Better support for SSH Tunneling over private IPs
Nov 15, 2019
Features
- Generator improvements:
- Email Generator can be used on columns with a unique constraint
- Character scramble now supports consistency
- Filename and Email generators make use of character scramble as opposed to character substitution (more secure). They both also now support consistency.
- More improvements to PII Detection, specifically speed, resilience, and better name detection
- Ability to duplicate workspaces (see screenshot below)
- Show generators applied on column in bulk view when table is truncated
- Support for SSH Bastion when connecting to source and output database
- Added new ENV variable for controlling parallelism for MySQL generation
Duplicate Workspace
Bug Fixes
- Fixed bad description on FK Relationships upload help text
- Fixed layout issue in Replacement Panel
- Fixed issue with ICD10 PII detection
- Fixed routing issues on Heroku
Last modified 6mo ago