Structural differences and limitations with Google BigQuery

No hidden datasets as source or destination

Google BigQuery workspaces cannot have a hidden dataset as its source or destination dataset.

Post-job scripts run in transactions

Post-job scripts run inside of transactions. They are limited to statements that are supported within transactions.

Table mode limitations

Google BigQuery workspaces cannot use the following table modes:

• Incremental

Generator limitations

Google BigQuery workspaces cannot use the following generators:

• Cross Table Sum

Cannot process STRUCT or INTERVAL types

Tonic Structural cannot process STRUCT or INTERVAL types in Google BigQuery.

Customer-managed encryption keys not copied to destination

Structural does not copy customer-managed encryption keys (CMEK) from the source tables to the tables that it creates in the destination dataset.

To apply a CMEK to the destination dataset, we recommend that you apply a dataset or project-wide default CMEK in the destination dataset.

External, snapshot, and cloned table handling

In the destination database, Structural creates external, snapshot, and cloned tables as normal tables.

Cannot assign generators when partition filters required

You cannot assign generators to partitioned tables that require a partition filter.

The environment setting TONIC_GRPC_ENABLED indicates whether to use GRPC-based endpoints to access Google BigQuery.

If TONIC_GRPC_ENABLED is true, then you can leave all of the columns set to Passthrough.

If TONIC_GRPC_ENABLED is false, then you must truncate the tables.

No subsetting, but support for table filtering

Google BigQuery workspaces do not support subsetting.

However, for tables that use the De-Identify table mode, you can provide a WHERE clause to filter the table. For details, go to Using table filtering for data warehouses and Spark-based data connectors.

No upsert

Google BigQuery workspaces do not support upsert.

No output to a container repository

For Google BigQuery workspaces, you cannot write the destination data to a container repository.