Schema management settings

On Workspace Settings view for a workspace, the schema management settings are generally at the end of the Source Settings section.

Schema change management settings for a workspace

Responding to schema changes

Schema changes include:

• Schema changes that could expose data, which if not addressed can result in data leakage. These changes include new tables and columns, and changes to data types.

• Notifications, which Structural can handle automatically during each data generation. These include removed tables and columns.

On the Workspace Settings view, under Block Data Generation on Schema Changes, select how Structural responds when there are unaddressed changes to the database schema.

The options are:

• Do Not Block - With this option, schema changes never block data generation. Structural ignores sensitive schema changes, and automatically handles notifications during data generation.

• Block On Changes That Could Expose Data - Indicates to only block data generation if there are schema changes that might expose data, such as new columns. Structural automatically handles notifications during data generation. For this option, Structural does not block data generation for schema changes on truncated tables.

• Block On All Changes - For this option, if there are any unaddressed schema changes at all, either sensitive changes or notifications, then data generation fails.

For more information, go to Viewing and resolving schema changes.

Indicating whether to cache the source schema

Schema caching is not available for document-based databases (MongoDB, DynamoDB).

About the schema cache

By default, every time you load a workspace, Structural queries the source database to retrieve the schema.

You can instead configure the workspace to cache the schema. Structural then updates the cache at a regular interval, and whenever a change to the workspace triggers a schema cache update.

You can also trigger a cache update manually.

By default, the schema cache is only used by calls from within Structural. To enable an external API request to use the cached schema, add the query parameter useSchemaCache=true to the request.

In the application, each update to the schema cache is represented by a schema retrieval job. Schema retrieval jobs are short-lived, and run on the Structural web server. You can view the schema retrieval jobs from the workspace Jobs view.

Note that the schema cache does not include the schema for JSON columns that use Document View. Those schemas are detected by a different scan.

Enabling and configuring schema caching

To enable and configure the caching:

1. On the Workspace Settings view, toggle Cache source schema for faster loading to the on position.

2. Under Schema Freshness, configure the maximum length of time between schema retrievals.

   1. In the field, provide the value.

   2. From the dropdown list, select the unit of time. You can configure the length of time in minutes, hours, or days.

   If the cached schema is older than that length of time, then the next time the application loads, it queries the source database for the current schema. The default value is 6 hours. Note that for some data connectors, schema retrievals run automatically in the background. This setting does not affect the frequency of those schema retrievals. For example, a schema retrieval runs automatically in the background every 2 hours. If you set the schema freshness to 6 hours, the background retrieval still runs every 2 hours. However, if you set the schema freshness to 1 hour, then schema retrieval occurs no more than 1 hour after the previous schema retrieval.

3. You can optionally enable diagnostic logging for the schema retrieval. Diagnostic logging adds additional diagnostic errors to help with troubleshooting. Note that this additional information might contain sensitive information such as schema identifiers. To enable diagnostic logging:

   1. Click Show advanced options.

   2. Toggle Enable diagnostic logging to the on position.