Structural Cloud API docs
Other product guides
Tonic.ai

Before you create a file connector workspace

For a file connector workspace that reads files from and writes files to Amazon S3 or Google Cloud Storage, make sure to set up the appropriate permissions so that Tonic Structural can locate the source files and write the destination files.

You can also set up permissions to protect buckets that contain files that you do not want used in a workspace.

Amazon S3 permissions

For a self-hosted instance, to get access to Amazon S3, Structural uses either:

  • The credentials set in the following environment settings:

    • TONIC_AWS_ACCESS_KEY_ID - An AWS access key that is associated with an IAM user or role.

    • TONIC_AWS_SECRET_ACCESS_KEY - The secret key that is associated with the access key

    • TONIC_AWS_REGION - The AWS Region to send the authentication request to

    For more information, go to Configuring environment settings.

  • The credentials for the IAM role on the host machine.

On Structural Cloud, you must provide the AWS credentials in the workspace configuration.

The IAM user that is associated with the credentials must have the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        ## Lists all buckets that are tied to given AWS account.
        ## Allows Structural to view the list of buckets during file group creation.
        ## If not granted, then users must type bucket names manually.
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": "*"
        },
        ## Allows Structural to list all objects within a specified bucket.
        ## Needed to list files during file group creation.
        ## Use Resource to restrict the displayed buckets to a specific prefix 
        ## or bucket name.
        ## Use Condition to restrict the listed files to a specific prefix.
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ],
            "Condition": {
                 "StringLike": {
                     "s3:prefix": "*"
                 }
             }
        },
        ## File level permissions for reading and writing.
        ## Can use Resource to restrict access based on bucket name or prefix.
        ## For example: arn:aws:s3:::bucket-prefix-*/object/prefix/*
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::*/*"
        }
    ]
}

If the source and destination S3 buckets are in different accounts, or are in an account that is different from the account or instance profile that Structural uses, then the configuration must include cross-account permissions. For assistance with this, contact support@tonic.ai.

Google Cloud Storage permissions

To get access to Google Cloud Storage, Structural uses the Google Cloud Platform credentials that you provide in the workspace configuration.

The service account that you specify must have the following permissions.

  • storage.buckets.list - This allows Structural to see the list of buckets when it creates a file group. If the service account does not have this permission, then on the file group creation panel, users must type the name of the bucket where a file is located.

  • For buckets that contain source files, the following permissions allow Structural to get the list of files within buckets, and to retrieve the actual files and file content.

    • storage.objects.get

    • storage.objects.list

    If the permissions are assigned globally, then Structural can list and retrieve files from any bucket. If the permissions are assigned to individual buckets, the file group creation view displays a list of all buckets. However, if you select a bucket for which the service account does not have the permissions, Structural returns an error.

  • For buckets that contain destination files, the following permissions allow Structural to see and get access to the bucket content and to create the generated files. This includes deleting and overwriting existing files that are regenerated.

    • storage.buckets.get

    • storage.objects.get

    • storage.objects.list

    • storage.objects.create

    • storage.objects.delete

    If the permissions are assigned globally, then Structural can write files to any bucket. If the permissions are assigned to individual buckets, then Structural can only write files to those buckets.

Last updated