Overview of the file connector process

The file connector uses text files for the source and destination data.

Using files from cloud storage

The recommended option is to have the files in cloud storage. The files can be in either:

• Amazon S3

• MinIO

• Google Cloud Storage (GCS)

Within a file connector workspace, you create file groups. A file group is a set of files that have the same format and structure. The file group points to the files in the cloud storage location.

Tonic Structural treats each file group as a table. Within the file groups, you configure the generators to apply to the columns.

When you run data generation, Structural writes the output files to the configured destination file storage location for the workspace.

In the destination location, each source file is represented by a corresponding destination file that has the same name.

Local files

You can also use files from a local file system.

For a file connector workspace that uses local files, when you add files to a file group, Structural encrypts the selected files. It then stores the files in the large file store of the Structural application database.

When you remove a file from a file group, remove an entire file group, or delete the workspace, Structural also removes those stored files.

When you run data generation, Structural also writes the output files to the large file store of the Structural application database. You can then download the most recently generated files.