Ctrlk
Tonic.ai
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Manage file groups in a file connector workspace

Included in the Basic API.

The file connector requires a Professional or Enterprise license.

In a file connector workspace (dataType is Files), you use file groups to identify the data and configure data generation. Each file group contains a set of files that are of the same type (.csv, .json, .xml) and that have an identical format.

For a file group that contains .csv files, you also provide information about the file structure and parsing. This includes the escape character, quote character, null character, and delimiter.

When you set the .csv parsing options, you must properly escape any escapable characters. For example, in the API request body, to use \N as the null character, you must use an extra \ to escape the value. So instead of "nullChar": "\N", the request body would include "nullChar": "\\N".

Getting the list of file groups

To retrieve the list of file groups in a file connector workspace, use:

GET /api/FileGroup

You provide the identifier of the workspace.

The results contain an array of FileGroupResponseModel objects. Each object represents a file group.

fileType identifies the type of file in the group.

For a file group that contains .csv files, the object includes the delimiter configuration (escapeChar, quoteChar, hasHeader, delimiter, nullChar). If the files have a header row, then the object includes the header list.

For a file group that contains files from cloud storage, the files object contains bucketKeyPair objects.

{
  "id": "string",
  "name": "string",
  "files": [
    {
      "bucketKeyPair": {
        "bucketName": "string",
        "key": "string"
      },
      "expectedFileType": "csv"
    }
  ],
  "createdDate": "2023-08-17T16:24:46.8443973Z",
  "workspaceId": "string",
  "escapeChar": "string",
  "quoteChar": "string",
  "hasHeader": true,
  "delimiter": "string",
  "nullChar": "string",
  "fileType": "string",
  "csvHeaderColumns": [
    "string"
  ]
}

For a file group that contains files from a local file system, the files object contains localFile objects, and also indicates whether there are available generated files to download:

Managing cloud storage file groups

Creating a cloud storage file group

For a file connector workspace that uses files from cloud storage, to create a file group, use:

POST /api/FileGroup

You identify the workspace and provide the file group name. You also provide the list of files and, for a file group that contains .csv files, the delimiter configuration.

Updating a cloud storage file group

To update a file group that contains files from Amazon S3 or Google Cloud Storage, to change the files, use:

PUT /api/FileGroup

In the request, you provide a FileGroupDefinitionModel object that contains a revised list of files. You cannot change the file group name, file type, or the .csv delimiter configuration.

Managing local file file groups

Creating a local files file group

To create a file group that contains files from a local file system, use:

POST /api/FileGroup/create_local_filegroup

In the request, you provide a FileGroupDefinitionModel object that identifies the workspace and provides the file group name. If the file group contains .csv files, you also provide the delimiter configuration.

Adding a file to a local files file group

To add a file to a file group from a local file system, use:

POST /api/FileGroup/upload_local_file

You identify the file group to add the file to, and include the file.

Deleting a file group

To remove a file group from a file connector workspace, use:

DELETE /api/FileGroup

In the request, you identify the workspace and the file group.

Downloading generated files for a local files file group

For a file group that contains files from a local file system, both the source files and the generated output files are stored in the Tonic Structural application database.

To identify file groups that available output files to download, use:

GET /api/FileGroup/generated_file_availability

In the request, you provide the workspace identifier.

The response contains a list of identifiers for file groups that have generated files available to download.

To download the available files for a file group, use:

GET /api/FileGroup/download/{workspaceId}/{fileGroupId}

Last updated

Was this helpful?