About data connectors

Each Tonic Structural workspace has a source database, which contains the original data, and a destination database, where Structural writes the data after it runs a data generation job and applies the table mode, generator, and subsetting configuration.

Structural provides a set of available data connectors. Most Structural connectors are associated with specific application databases, data warehouses, or Spark products. The file connector allows you to transform data from files. When you create a workspace, you choose the connector to use based on how your source data is stored.

Each data connector can have specific configuration requirements. A data connector also might not support all of the Structural features.

For an overview list of the available data connectors, go to Data connector summary.

Types of data connectors

Data connectors fall into one of the following general categories. Structural connects and writes to the data slightly differently for each type.

Application database

For an application database, Structural connects to both the source and destination databases as a SQL client.

It reads data, transforms the data, and then writes the data directly to the destination database. The transformation occurs on the Structural server.

Data warehouse

For data warehouses other than Google BigQuery, Structural uses temporary file storage when it reads and processes the data. It then writes the data from the temporary storage to the destination database.

Spark

For Spark data connectors, Structural uses a metastore or data catalog to get the file format. It then uses Spark to process the source files and write to the destination files.

Software as a Service (SaaS)

Software as a Service (Saas) delivers an application remotely from the Internet.

Structural allows you to transform the data that is associated with using the application.

File connector

The file connector allows you to use files as the source data. You add files to a file group, which contains files that have an identical format and structure. Each file group is treated as a table for the purposes of generator configuration.

For files that you add from your local file system, Structural encrypts the files and then stores them in the large file store of the Structural application database.

Structural does not store files that you add from cloud storage.

When the generation is complete, Structural writes transformed versions of the source files to an output location. The output files have the same name and structure as the input files. Previous versions of the generated files are replaced.

Availability requirements for data connectors

Whether a data connector is available to use is based on the following:

• License plan

• Deployment type (Structural Cloud or self-hosted)

License plan requirements for data connectors

With a Basic license, then you can use one data connector, which must be either MySQL or PostgreSQL.

With a Professional license, you can use two data connectors, which can be any data connector other than Oracle or DB2 for LUW.

With an Enterprise license, there are no limits on the number of data connectors, and you can use Oracle and DB2 for LUW.

Structural Cloud support for data connectors

Structural Cloud does not support the following data connectors, regardless of the license plan:

• Amazon EMR
• Amazon Redshift
• Db2 for LUW
• Spark SDK

Availability overview

The following table summarizes the data connector availability based on the license plan and deployment type:

License Plan	Structural Cloud	Self-Hosted
Basic (can use one type)	MySQL PostgreSQL	MySQL PostgreSQL
Professional (can use two types)	Amazon DynamoDB Databricks File connector Google BigQuery MongoDB MySQL PostgreSQL Salesforce Snowflake on AWS Snowflake on Azure SQL Server Yugabyte	Amazon DynamoDB Amazon EMR Amazon Redshift Databricks File connector Google BigQuery MongoDB MySQL PostgreSQL Salesforce Snowflake on AWS Snowflake on Azure Spark SDK SQL Server Yugabyte
Enterprise (unlimited types)	Amazon DynamoDB Databricks File connector Google BigQuery MongoDB MySQL Oracle PostgreSQL Salesforce Snowflake on AWS Snowflake on Azure SQL Server Yugabyte	Amazon DynamoDB Amazon EMR Amazon Redshift Databricks Db2 for LUW File connector Google BigQuery MongoDB MySQL Oracle PostgreSQL Salesforce Snowflake on AWS Snowflake on Azure Spark SDK SQL Server Yugabyte
Structural free trial	Amazon DynamoDB Databricks File connector Google BigQuery MongoDB MySQL PostgreSQL Snowflake on AWS Snowflake on Azure SQL Server	N/A

General rules for data connectors

Same data connector for source and destination

The source and destination databases use the same connector. For example, you cannot take data from a MySQL source database and then have Structural generate an Oracle destination database.

Same version for source and destination

The source and destination databases also should use the same database version. At the very least, the version used by the destination database cannot be older than the version used by the source database.

For example, if the source database uses PostgreSQL 12.8, then the destination database cannot use a lower version such as PostgreSQL 12.4.

Allowlisting Structural static IP addresses for Structural Cloud

If you use Structural Cloud, and your database only allows connections from allowlisted IP addresses, then you need to allowlist Structural static IP addresses.

This is not required for self-hosted instances of Structural.

For the United States-based instance (app.tonic.ai), the static IP addresses are:

• 54.92.217.68

• 3.217.201.126

• 52.22.13.250

For the Europe-based instance (app-de.tonic.ai), the static IP addresses are:

• 18.159.127.160

• 3.69.249.144