LogoLogo
Release notesAPI docsDocs homeStructural CloudTonic.ai
  • Tonic Structural User Guide
  • About Tonic Structural
    • Structural data generation workflow
    • Structural deployment types
    • Structural implementation roles
    • Structural license plans
  • Logging into Structural for the first time
  • Getting started with the Structural free trial
  • Managing your user account
  • Frequently Asked Questions
  • Tutorial videos
  • Creating and managing workspaces
    • Managing workspaces
      • Viewing your list of workspaces
      • Creating, editing, or deleting a workspace
      • Workspace configuration settings
        • Workspace identification and connection type
        • Data connection settings
        • Configuring secrets managers for database connections
        • Data generation settings
        • Enabling and configuring upsert
        • Writing output to Tonic Ephemeral
        • Writing output to a container repository
        • Advanced workspace overrides
      • About the workspace management view
      • About workspace inheritance
      • Assigning tags to a workspace
      • Exporting and importing the workspace configuration
    • Managing access to workspaces
      • Sharing workspace access
      • Transferring ownership of a workspace
    • Viewing workspace jobs and job details
  • Configuring data generation
    • Privacy Hub
    • Database View
      • Viewing and configuring tables
      • Viewing the column list
      • Displaying sample data for a column
      • Configuring an individual column
      • Configuring multiple columns
      • Identifying similar columns
      • Commenting on columns
    • Table View
    • Working with document-based data
      • Performing scans on collections
      • Using Collection View
    • Identifying sensitive data
      • Running the Structural sensitivity scan
      • Manually indicating whether a column is sensitive
      • Built-in sensitivity types that Structural detects
      • Creating and managing custom sensitivity rules
    • Table modes
    • Generator information
      • Generator summary
      • Generator reference
        • Address
        • Algebraic
        • Alphanumeric String Key
        • Array Character Scramble
        • Array JSON Mask
        • Array Regex Mask
        • ASCII Key
        • Business Name
        • Categorical
        • Character Scramble
        • Character Substitution
        • Company Name
        • Conditional
        • Constant
        • Continuous
        • Cross Table Sum
        • CSV Mask
        • Custom Categorical
        • Date Truncation
        • Email
        • Event Timestamps
        • File Name
        • Find and Replace
        • FNR
        • Geo
        • HIPAA Address
        • Hostname
        • HStore Mask
        • HTML Mask
        • Integer Key
        • International Address
        • IP Address
        • JSON Mask
        • MAC Address
        • Mongo ObjectId Key
        • Name
        • Noise Generator
        • Null
        • Numeric String Key
        • Passthrough
        • Phone
        • Random Boolean
        • Random Double
        • Random Hash
        • Random Integer
        • Random Timestamp
        • Random UUID
        • Regex Mask
        • Sequential Integer
        • Shipping Container
        • SIN
        • SSN
        • Struct Mask
        • Timestamp Shift Generator
        • Unique Email
        • URL
        • UUID Key
        • XML Mask
      • Generator characteristics
        • Enabling consistency
        • Linking generators
        • Differential privacy
        • Partitioning a column
        • Data-free generators
        • Supporting uniqueness constraints
        • Format-preserving encryption (FPE)
      • Generator types
        • Composite generators
        • Primary key generators
    • Generator assignment and configuration
      • Reviewing and applying recommended generators
      • Assigning and configuring generators
      • Document View for file connector JSON columns
      • Generator hints and tips
      • Managing generator presets
      • Configuring and using Structural data encryption
      • Custom value processors
    • Subsetting data
      • About subsetting
      • Using table filtering for data warehouses and Spark-based data connectors
      • Viewing the current subsetting configuration
      • Subsetting and foreign keys
      • Configuring subsetting
      • Viewing and managing configuration inheritance
      • Viewing the subset creation steps
      • Viewing previous subsetting data generation runs
      • Generating cohesive subset data from related databases
      • Other subsetting hints and tips
    • Viewing and adding foreign keys
    • Viewing and resolving schema changes
    • Tracking changes to workspaces, generator presets, and sensitivity rules
    • Using the Privacy Report to verify data protection
  • Running data generation
    • Running data generation jobs
      • Types of data generation
      • Data generation process
      • Running data generation manually
      • Scheduling data generation
      • Issues that prevent data generation
    • Managing data generation performance
    • Viewing and downloading container artifacts
    • Post-job scripts
    • Webhooks
  • Installing and Administering Structural
    • Structural architecture
    • Using Structural securely
    • Deploying a self-hosted Structural instance
      • Deployment checklist
      • System requirements
      • Deploying with Docker Compose
      • Deploying on Kubernetes with Helm
      • Enabling the option to write output data to a container repository
        • Setting up a Kubernetes cluster to use to write output data to a container repository
        • Required access to write destination data to a container repository
      • Entering and updating your license key
      • Setting up host integration
      • Working with the application database
      • Setting up a secret
      • Setting a custom certificate
    • Using Structural Cloud
      • Structural Cloud notes
      • Setting up and managing a Structural Cloud pay-as-you-go subscription
      • Structural Cloud onboarding
    • Managing user access to Structural
      • Structural organizations
      • Determining whether users can create accounts
      • Creating a new account in an existing organization
      • Single sign-on (SSO)
        • Structural user authentication with SSO
        • Enabling and configuring SSO on Structural Cloud
        • Synchronizing SSO groups with Structural
        • Viewing the list of SSO groups in Tonic Structural
        • AWS IAM Identity Center
        • Duo
        • GitHub
        • Google
        • Keycloak
        • Microsoft Entra ID (previously Azure Active Directory)
        • Okta
        • OpenID Connect (OIDC)
        • SAML
      • Managing Structural users
      • Managing permissions
        • About permission sets
        • Built-in permission sets
        • Available permissions
        • Viewing the lists of global and workspace permission sets
        • Configuring custom permission sets
        • Selecting default permission sets
        • Configuring access to global permission sets
        • Setting initial access to all global permissions
        • Granting Account Admin access for a Structural Cloud organization
    • Structural monitoring and logging
      • Monitoring Structural services
      • Performing health checks
      • Downloading the usage report
      • Tracking user access and permissions
      • Redacted and diagnostic (unredacted) logs
      • Data that Tonic.ai collects
      • Verifying and enabling telemetry sharing
    • Configuring environment settings
    • Updating Structural
  • Connecting to your data
    • About data connectors
    • Overview for database administrators
    • Data connector summary
    • Amazon DynamoDB
      • System requirements and limitations for DynamoDB
      • Structural differences and limitations with DynamoDB
      • Before you create a DynamoDB workspace
      • Configuring DynamoDB workspace data connections
    • Amazon EMR
      • Structural process overview for Amazon EMR
      • System requirements for Amazon EMR
      • Structural differences and limitations with Amazon EMR
      • Before you create an Amazon EMR workspace
        • Creating IAM roles for Structural and Amazon EMR
        • Creating Athena workgroups
        • Configuration for cross-account setups
      • Configuring Amazon EMR workspace data connections
    • Amazon Redshift
      • Structural process overview for Amazon Redshift
      • Structural differences and limitations with Amazon Redshift
      • Before you create an Amazon Redshift workspace
        • Required AWS instance profile permissions for Amazon Redshift
        • Setting up the AWS Lambda role for Amazon Redshift
        • AWS KMS permissions for Amazon SQS message encryption
        • Amazon Redshift-specific Structural environment settings
        • Source and destination database permissions for Amazon Redshift
      • Configuring Amazon Redshift workspace data connections
    • Databricks
      • Structural process overview for Databricks
      • System requirements for Databricks
      • Structural differences and limitations with Databricks
      • Before you create a Databricks workspace
        • Granting access to storage
        • Setting up your Databricks cluster
        • Configuring the destination database schema creation
      • Configuring Databricks workspace data connections
    • Db2 for LUW
      • System requirements for Db2 for LUW
      • Structural differences and limitations with Db2 for LUW
      • Before you create a Db2 for LUW workspace
      • Configuring Db2 for LUW workspace data connections
    • File connector
      • Overview of the file connector process
      • Supported file and content types
      • Structural differences and limitations with the file connector
      • Before you create a file connector workspace
      • Configuring the file connector storage type and output options
      • Managing file groups in a file connector workspace
      • Downloading generated file connector files
    • Google BigQuery
      • Structural differences and limitations with Google BigQuery
      • Before you create a Google BigQuery workspace
      • Configuring Google BigQuery workspace data connections
      • Resolving schema changes for de-identified views
    • MongoDB
      • System requirements for MongoDB
      • Structural differences and limitations with MongoDB
      • Configuring MongoDB workspace data connections
      • Other MongoDB hints and tips
    • MySQL
      • System requirements for MySQL
      • Before you create a MySQL workspace
      • Configuring MySQL workspace data connections
    • Oracle
      • Known limitations for Oracle schema objects
      • System requirements for Oracle
      • Structural differences and limitations with Oracle
      • Before you create an Oracle workspace
      • Configuring Oracle workspace data connections
    • PostgreSQL
      • System requirements for PostgreSQL
      • Before you create a PostgreSQL workspace
      • Configuring PostgreSQL workspace data connections
    • Salesforce
      • System requirements for Salesforce
      • Structural differences and limitations with Salesforce
      • Before you create a Salesforce workspace
      • Configuring Salesforce workspace data connections
    • Snowflake on AWS
      • Structural process overviews for Snowflake on AWS
      • Structural differences and limitations with Snowflake on AWS
      • Before you create a Snowflake on AWS workspace
        • Required AWS instance profile permissions for Snowflake on AWS
        • Other configuration for Lambda processing
        • Source and destination database permissions for Snowflake on AWS
        • Configuring whether Structural creates the Snowflake on AWS destination database schema
      • Configuring Snowflake on AWS workspace data connections
    • Snowflake on Azure
      • Structural process overview for Snowflake on Azure
      • Structural differences and limitations with Snowflake on Azure
      • Before you create a Snowflake on Azure workspace
      • Configuring Snowflake on Azure workspace data connections
    • Spark SDK
      • Structural process overview for the Spark SDK
      • Structural differences and limitations with the Spark SDK
      • Configuring Spark SDK workspace data connections
      • Using Spark to run de-identification of the data
    • SQL Server
      • System requirements for SQL Server
      • Before you create a SQL Server workspace
      • Configuring SQL Server workspace data connections
    • Yugabyte
      • System requirements for Yugabyte
      • Structural differences and limitations with Yugabyte
      • Before you create a Yugabyte workspace
      • Configuring Yugabyte workspace data connections
      • Troubleshooting Yugabyte data generation issues
  • Using the Structural API
    • About the Structural API
    • Getting an API token
    • Getting the workspace ID
    • Using the Structural API to perform tasks
      • Configure environment settings
      • Manage generator presets
        • Retrieving the list of generator presets
        • Structure of a generator preset
        • Creating a custom generator preset
        • Updating an existing generator preset
        • Deleting a generator preset
      • Manage custom sensitivity rules
      • Create a workspace
      • Connect to source and destination data
      • Manage file groups in a file connector workspace
      • Assign table modes and filters to source database tables
      • Set column sensitivity
      • Assign generators to columns
        • Getting the generator IDs and available metadata
        • Updating generator configurations
        • Structure of a generator assignment
        • Generator API reference
          • Address (AddressGenerator)
          • Algebraic (AlgebraicGenerator)
          • Alphanumeric String Key (AlphaNumericPkGenerator)
          • Array Character Scramble (ArrayTextMaskGenerator)
          • Array JSON Mask (ArrayJsonMaskGenerator)
          • Array Regex Mask (ArrayRegexMaskGenerator)
          • ASCII Key (AsciiPkGenerator)
          • Business Name (BusinessNameGenerator)
          • Categorical (CategoricalGenerator)
          • Character Scramble (TextMaskGenerator)
          • Character Substitution (StringMaskGenerator)
          • Company Name (CompanyNameGenerator)
          • Conditional (ConditionalGenerator)
          • Constant (ConstantGenerator)
          • Continuous (GaussianGenerator)
          • Cross Table Sum (CrossTableAggregateGenerator)
          • CSV Mask (CsvMaskGenerator)
          • Custom Categorical (CustomCategoricalGenerator)
          • Date Truncation (DateTruncationGenerator)
          • Email (EmailGenerator)
          • Event Timestamps (EventGenerator)
          • File Name (FileNameGenerator)
          • Find and Replace (FindAndReplaceGenerator)
          • FNR (FnrGenerator)
          • Geo (GeoGenerator)
          • HIPAA Address (HipaaAddressGenerator)
          • Hostname (HostnameGenerator)
          • HStore Mask (HStoreMaskGenerator)
          • HTML Mask (HtmlMaskGenerator)
          • Integer Key (IntegerPkGenerator)
          • International Address (InternationalAddressGenerator)
          • IP Address (IPAddressGenerator)
          • JSON Mask (JsonMaskGenerator)
          • MAC Address (MACAddressGenerator)
          • Mongo ObjectId Key (ObjectIdPkGenerator)
          • Name (NameGenerator)
          • Noise Generator (NoiseGenerator)
          • Null (NullGenerator)
          • Numeric String Key (NumericStringPkGenerator)
          • Passthrough (PassthroughGenerator)
          • Phone (USPhoneNumberGenerator)
          • Random Boolean (RandomBooleanGenerator)
          • Random Double (RandomDoubleGenerator)
          • Random Hash (RandomStringGenerator)
          • Random Integer (RandomIntegerGenerator)
          • Random Timestamp (RandomTimestampGenerator)
          • Random UUID (UUIDGenerator)
          • Regex Mask (RegexMaskGenerator)
          • Sequential Integer (UniqueIntegerGenerator)
          • Shipping Container (ShippingContainerGenerator)
          • SIN (SINGenerator)
          • SSN (SsnGenerator)
          • Struct Mask (StructMaskGenerator)
          • Timestamp Shift (TimestampShiftGenerator)
          • Unique Email (UniqueEmailGenerator)
          • URL (UrlGenerator)
          • UUID Key (UuidPkGenerator)
          • XML Mask (XmlMaskGenerator)
      • Configure subsetting
      • Check for and resolve schema changes
      • Run data generation jobs
      • Schedule data generation jobs
    • Example script: Starting a data generation job
    • Example script: Polling for a job status and creating a Docker package
Powered by GitBook
On this page
  • Selecting the collection to view
  • Assigning a collection mode to the collection
  • Available collection modes
  • Assigning the collection mode
  • Selecting the type of view
  • Hybrid document view
  • Single document view
  • Information on the field list
  • Toggling between source and preview data
  • Filtering collection fields
  • Filtering single document view by field name or value
  • Filtering hybrid view by field name
  • Filtering hybrid view by field properties
  • Filters panel filters
  • At-risk fields
  • Sensitivity
  • Protection status
  • Recommended generators
  • Field data type
  • Unresolved schema changes
  • Sensitivity type
  • Sensitivity confidence
  • Primary or foreign keys
  • Commenting on fields
  • Adding a new comment
  • Replying to an existing comment
  • Indicating whether a field is sensitive
  • Assigning a generator to a field and type
  • Disabling examples for sparse collections

Was this helpful?

Export as PDF
  1. Configuring data generation
  2. Working with document-based data

Using Collection View

Last updated 2 months ago

Was this helpful?

For MongoDB and Amazon DynamoDB, Collection View replaces Database View and Table View. From Collection View, you can view the fields in a selected collection. You can then assign a collection mode to the collection, and assign generators to fields.

Selecting the collection to view

From the Collection dropdown list, select the collection to view.

Assigning a collection mode to the collection

Collection mode is the term used for table mode. The collection mode determines at the collection level how Structural uses the collection data to generate the destination database.

Available collection modes

By default, the collection mode is De-Identify. In this mode, Structural uses the assigned generators to transform the source database into the destination database.

For MongoDB and DynamoDB, the only other options are Truncate and Preserve Destination.

  • Truncate means that only the collection structure is included in the destination database. The collection has no data in the destination database.

  • Preserve Destination means that Tonic does not change the data that is currently in the destination database.

Assigning the collection mode

Required workspace permission: Assign table modes

To assign the collection mode:

  1. Click the Collection Mode dropdown list.

  2. On the panel, click the current collection mode.

  3. From the drop-down list, select the mode to use.

Selecting the type of view

You can view a collection either as a hybrid document or as single documents. From the View dropdown list, select the view to use.

Hybrid document view

The default view is Hybrid Document. For the hybrid document view, the key list reflects all of the permutations of every field from every document. For example, a field might sometimes be a datetime value and sometimes a string. Hybrid document view lists both types.

Single document view

Single Document view displays a single document at a time. You can then page through up to 100 documents. For each document, you see the structure for that particular document.

Information on the field list

For each field, Collection View always displays:

  • The field name and type.

  • For fields that you configured as primary or foreign keys, a key icon.

  • The assigned generator.

  • An example value. For the hybrid view, you can use the magnifying glass icon to display additional example values.

For the hybrid document view, there is also a Field Freq column. Field Freq shows the percentage of documents that contain that permutation of field and type.

For example, you might see that a field is Null 33% of the time and contains a numeric value 67% of the time. Or a field value is an Int32 value 3% of the time and an Int64 value 6% of the time. The percentages apply to the first 100 documents.

Toggling between source and preview data

Required workspace permission:

  • Source data: Preview source data

  • Destination data: Preview destination data

The Preview toggle at the top right of Collection View allows you to choose whether to display original source data or the transformed data. You can switch back and forth to see exactly how Tonic Structural transforms the data based on the collection and field configuration.

By default, the Preview toggle is in the on position, and the displayed data reflects the selected collection mode and the assigned generators. For collections that use Truncate mode, the preview data is empty. Truncated collections do not have data in the destination database.

To display the original source data, toggle Preview to the off position.

Filtering collection fields

In the single document view, you can filter the fields by either the field name or the field value.

In the hybrid document view, you can filter the fields based on either the field name or field properties.

Filtering single document view by field name or value

You can filter single document view to only display fields that have specific text in either the field name or the field value.

To filter by value, toggle Search by Value to the on position.

After you select the filter type, in the search field, type text that is in the field name or value. As you type, Structural filters the list to only include fields that contain the filter text.

Filtering hybrid view by field name

To filter hybrid view by field name, in the search field, begin to type text that is in the field name. As you type, Structural filters the list to only include fields with names that include the filter text.

Filtering hybrid view by field properties

From the hybrid document view, you can filter the fields based on field properties.

To display the Filters panel, click Filters.

Searching for a filter

To search for a filter or a filter value, in the search field, start to type the value. The search looks for text in the individual settings.

Adding a filter

To add a filter, depending on the filter type, either check the checkbox or select a filter option. As you add filters, Structural applies them to the field list.

Above the list, Structural displays tags for the selected filters.

Clearing the selected filters

To clear all of the currently selected filters, click Clear All.

Filters panel filters

The Filters panel in hybrid view includes the following fields.

At-risk fields

An at-risk field:

  • Is marked as sensitive

  • Is assigned the Passthrough generator.

To only display at-risk fields, on the Filters panel, check At-Risk Field.

When you check At-Risk Field, Structural adds the following filters under Privacy Settings:

  • Sets the sensitivity filter to Sensitive.

  • Sets the protection status filter to Not protected.

Sensitivity

You can filter the fields based on the field sensitivity.

On the Filters panel, under Privacy Settings, the sensitivity filter is by default set to All, which indicates to display both sensitive and non-sensitive fields.

  • To only display sensitive fields, click Sensitive.

  • To only display non-sensitive fields, click Not sensitive.

Note that when you check At-risk Field, Structural automatically selects Sensitive.

Protection status

You can filter the fields based on whether they have any generator other than Passthrough assigned.

On the Filters panel, under Privacy Settings, the field protection filter is by default set to All, which indicates to display both protected and not protected fields.

  • To only display fields that have an assigned generator, click Protected.

  • To only display fields that do not have an assigned generator, click Not protected.

Note that when you check At-Risk Field, Structural automatically selects Not protected.

Recommended generators

When Structural detects that a field is sensitive, it can also determine a recommended generator.

For example, when it detects a name value, it also recommends the Name generator.

You can filter the fields to display the fields that have recommended generators.

On the Filters panel, under Recommended Generators, check the checkbox next to the recommended generator for which to display the fields that have that recommendation.

Field data type

You can filter the fields by the field data type. For example, you might only display columns that contain either numeric or integer values.

To only display fields that have specific data types, on the Filters panel, under Database Data Types, check the checkbox for each data type to include.

The list of data types only includes data types that are present in the currently displayed fields and that are compatible with other applied filters.

To search for a specific data type, in the Filters search field, begin to type the data type.

Unresolved schema changes

When the source database schema changes, you might need to update the configuration to reflect those changes. If you do not resolve the schema changes, then the data generation might fail. The data generation fails if there are unresolved conflicting changes, or if you configure Structural to always fail data generation when there are any unresolved changes.

For more information about schema changes, go to Viewing and resolving schema changes.

To only display fields that have unresolved schema changes, on the Filters panel, check Unresolved Schema Changes.

Sensitivity type

For detected sensitive fields, the sensitivity type indicates the type of data that was detected. Examples of sensitivity types include First Name, Address, and Email.

To only display fields that contain specific sensitivity types, on the Filters panel, under Sensitivity Type, check the checkbox for each sensitivity type to include.

The list of sensitivity types only includes sensitivity types that are present in the currently displayed fields.

To search for a specific sensitivity type, in the Filters search field, type the sensitivity type.

Sensitivity confidence

When the Structural sensitivity scan identifies a value as belonging to a sensitivity type, it also determines how confident it is in that determination.

You can filter the columns based on the confidence level.

To only display columns that have a specific confidence level, on the Filters panel, under Sensitivity confidence, check the checkbox next to each confidence level to include.

Primary or foreign keys

You can filter the column list to indicate whether to include:

  • Columns that are not primary or foreign keys.

  • Columns that are foreign keys.

  • Columns that are primary keys.

On the Filters panel, under Field Type:

  • To display fields that are neither a primary key nor a foreign key, check Non-keyed.

  • To display fields that are primary keys, check Primary key.

  • To display fields that are foreign keys, check Foreign key.

Commenting on fields

Required license: Professional or Enterprise

You can add comments to fields. For example, you might use a comment to explain why you selected a particular generator or marked a field as sensitive or not sensitive.

Adding a new comment

If a field does not have any comments, then to add a comment:

  1. Click the comment icon.

  2. In the comment field, type the comment text.

  3. Click Comment.

Replying to an existing comment

When a field has existing comments, the comment icon is green. To add comments:

  1. Click the comment icon. The comments panel shows the previous comments. Each comment includes the comment user and timestamp.

  2. In the comment field, type the comment text.

  3. Click Reply.

Indicating whether a field is sensitive

Required workspace permission: Configure column sensitivity

On the field configuration panel, the sensitivity toggle at the top right indicates whether the field is marked as sensitive.

To mark a field as sensitive, toggle the setting to the Sensitive position.

To mark a field as not sensitive, toggle the setting to the Not Sensitive position.

Assigning a generator to a field and type

Required workspace permission: Configure column generators

You can assign a generator to each combination of field and type. For example, depending on the document, the data type for a field might be either string or integer. You can indicate to use the Character Scramble generator when the field type is a string and the Random Integer generator when the field type is integer.

In hybrid document view, the Null type reflects when the column value is Null. You do not assign a generator to it.

To assign a generator:

  1. Click the generator value for the field.

  2. On the configuration panel, from the Generator Type dropdown list, select the generator.

  3. Configure the generator options. For details about the available configuration options for each generator, go to the Generator reference.

Disabling examples for sparse collections

By default, Structural retrieves 100 documents. It then uses the data in these documents to populate example values in the hybrid document.

For sparsely populated collections, where less common fields are not present in those 100 documents, Structural retrieves extra documents until it has example values for all fields. For very sparsely populated collections, this might cause the collection view to load slowly, because it must retrieve many documents.

To disable examples for sparse collections, set the environment setting TONIC_MONGO_DISABLE_EXTRA_EXAMPLES to true. You can add this setting manually to the Environment Settings list on Structural Settings.

Note that this setting applies to both MongoDB and Amazon DynamoDB.

When this setting is true, fields that do not have a retrieved value use a dummy default value that is based on the data type.

Collection View for a MongoDB workspace
Hybrid Document view of Collection View
Single Document view of Collection View
Filter field and Search by Value toggle for single document view
Filter field and Filters button for hybrid view
Filters panel for hybrid view on Collection View