Schema management settings

Responding to schema changes

Schema changes include:

• Sensitive schema changes, which if not addressed can result in data leakage. These changes include new tables and columns, and changes to data types.

• Notifications, which Structural can handle automatically during each data generation. These include removed tables and columns.

Under Block Data Generation on Schema Changes, select how Structural responds when there are unaddressed changes to the database schema.

The options are:

• Do Not Block - With this option, schema changes never block data generation. Structural ignores sensitive schema changes, and automatically handles notifications during data generation.

• Block On Sensitive Changes - Indicates to only block data generation if there are sensitive schema changes. Structural automatically handles notifications during data generation.

• Block On All Changes - For this option, if there are any unaddressed schema changes at all, either sensitive changes or notifications, data generation fails.

For more information, go to Viewing and resolving schema changes.

Indicating whether to cache the source schema

Schema caching is not available for document-based databases (MongoDB, DynamoDB).

About the schema cache

By default, every time you load a workspace, Structural queries the source database to retrieve the schema.

You can instead configure the workspace to cache the schema. Structural then updates the cache at a regular interval, and whenever a change to the workspace triggers a schema cache update.

You can also trigger a cache update manually.

By default, the schema cache is only used by calls from within Structural. To enable an external API request to use the cached schema, add the query parameter useSchemaCache=true to the request.

In the application, each update to the schema cache is represented by a schema retrieval job. Schema retrieval jobs are short-lived, and run on the Structural web server. You can view the schema retrieval jobs from the workspace Jobs view.

Note that the schema cache does not include the schema for JSON columns that use Document View. Those schemas are detected by a different scan.

Enabling and configuring schema caching

To enable and configure the caching:

1. Toggle Cache source schema for faster loading to the on position.

2. Under Schema Freshness, configure the length of time that the cached schema is considered up to date.

   1. In the field, provide the value.

   2. From the dropdown list, select the unit of time. You can configure the length of time in minutes, hours, or days.

   If the cached schema is older than that length of time, then the next time the application loads, it queries the source database for the current schema. The default value is 6 hours.

3. You can optionally enable diagnostic logging for the schema retrieval. Diagnostic logging adds additional diagnostic errors to help with troubleshooting. Note that this additional information might contain sensitive information such as schema identifiers. To enable diagnostic logging:

   1. Click Show advanced options.

   2. Toggle Enable diagnostic logging to the on position.