Structural differences and limitations with Google BigQuery

No hidden datasets as source or destination

Google BigQuery workspaces cannot have a hidden dataset as its source or destination dataset.

Post-job scripts run in transactions

Post-job scripts run inside of transactions. They are limited to statements that are supported within transactions.

Table mode limitations

Google BigQuery workspaces cannot use the following table modes:

• Incremental

Generator limitations

Google BigQuery workspaces cannot use the following generators:

• Cross Table Sum

Cannot process STRUCT or INTERVAL types

Tonic Structural cannot process STRUCT or INTERVAL types in Google BigQuery.

External, snapshot, and cloned table handling

In the destination database, Structural creates external, snapshot, and cloned tables as normal tables.

Cannot assign generators when partition filters required

You cannot assign generators to partitioned tables that require a partition filter.

The environment setting TONIC_GRPC_ENABLED indicates whether to use GRPC-based endpoints to access Google BigQuery.

If TONIC_GRPC_ENABLED is true, then you can leave all of the columns set to Passthrough.

If TONIC_GRPC_ENABLED is false, then you must truncate the tables.

No subsetting, but support for table filtering

Google BigQuery workspaces do not support subsetting.

However, for tables that use the De-Identify table mode, you can provide a WHERE clause to filter the table. For details, go to Using table filtering for data warehouses and Spark-based data connectors.

No upsert

Google BigQuery workspaces do not support upsert.

No output to container artifacts

For Google BigQuery workspaces, you cannot write the destination data to container artifacts.

No output to an Ephemeral snapshot

For Google BigQuery workspaces, you cannot write the destination data to an Ephemeral snapshot.