Creating a dataset

Required global permission: Create datasets

When you create a dataset, you specify:

• The type of output to produce
• The source location for the files.
• If the files are in cloud storage, the connection credentials.

Setting the name, source type, and output type

To create a dataset:

1. On the Datasets page, click Create a Dataset.

Dataset creation panel

1. In the Dataset Name field, provide a name for the dataset.
2. Under Output Format, select the type of output to generate.
3. Under File Source, select the source type.
4. If the source type is a cloud storage option, then provide the required credentials.

   • Amazon S3
   • Azure
   • Sharepoint
5. To import settings from another dataset:

   1. Expand the Import settings from another dataset section.
   2. From the Dataset dropdown list, select the dataset to import settings from. To import settings from a dataset, you must have the View dataset settings permission for that dataset.
   3. Under Settings to import, select the type of settings to import. Project settings include the tags, .docx settings, and PDF settings. Entity settings include the entity type handling configuration and enabled custom entity types.

Import settings section on the dataset creation panel

1. Click Save.
2. For cloud storage datasets:

   1. Textual prompts you to configure the initial file selection. For more information, go to Selecting cloud storage files.
   2. After you select the files, it prompts you to select an output location. For more information, go to Changing cloud storage credentials and output location.

Providing credentials for Amazon S3

If the source type is Amazon S3, provide the credentials to use to connect to Amazon S3.

After you provide the credentials, click Save. Textual prompts you to select the dataset files.

Selecting the type of credentials to use

Under AWS Credentials Location, click the type of credentials to use. The options are:

• User credentials - Indicates to use the provided AWS user credentials.
• Assume role - Only available on self-hosted instances. Indicates to use the specified assumed role.
• Environment - Only available on self-hosted instances. Indicates to use either:

  • The credentials for the IAM role on the host machine.
  • The credentials set in the following environment settings:

    • AWS_DEFAULT_REGION - AWS Region
    • AWS_ACCESS_KEY_ID - AWS access key
    • AWS_SECRET_ACCESS_KEY - AWS secret key

Providing AWS user credentials

When you select User as the credentials location:

Credentials fields for an Amazon S3 dataset

1. In the Access Key field, provide an AWS access key that is associated with an IAM user or role. For an example of a role that has the required permissions for an Amazon S3 dataset, go to Required IAM role permissions for Amazon S3.
2. In the Secret Access Key field, provide the secret key that is associated with the access key.
3. From the Region dropdown list, select the AWS Region to send the authentication request to.
4. In the Session Token field, provide the session token to use for the authentication request.
5. To test the credentials, click Test AWS Connection.

Providing an assumed role

When you select Assume Role as the credentials location, whether you complete the configuration here or in the dataset Project Settings depends on how you determine the uniqueness of the external ID.

Fields to configure an assumed role for an Amazon S3 dataset

Making the external ID unique for the organization

1. Under External ID Uniqueness, to make the external ID unique for your organization, click Organization.
2. In the Role ARN field, provide the Amazon Resource Name (ARN) for the role.
3. In the Session Name field, provide the role session name. If you do not provide a session name, then Textual automatically generates a default unique value.
4. In the Duration (in seconds) field, provide the maximum length in seconds of the session. The default is 3600, indicating that the session can be active for up to 1 hour. The provided value must be less than the maximum session duration that is allowed for the role.
5. From the AWS Region dropdown list, select the AWS Region where the S3 bucket that the dataset reads from is located.
6. To test the connection, click Test AWS Connection.

Making the external ID unique for the dataset

Under External ID Uniqueness, to make the external ID unique for the dataset, click Dataset.

Because Textual cannot generate the external ID before the dataset is created, Textual hides the remaining configuration fields for the assumed role (role ARN, session name, duration, and AWS Region).

When you save the dataset, Textual automatically routes you to Project Settings to provide those values.

Requirements for the role trust policy

Your role’s trust policy must be configured to condition on the external ID.

To view an example trust policy, click View example policy. Here is an example trust policy:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Principal": {
      "AWS": "<originating-account-id>"
    },
    "Action": "sts:AssumeRole",
    "Condition": {
      "StringEquals": {
        "sts:ExternalId": "<external-id>"
      }
    }
  }
}

Changing the S3 bucket encryption

Connections to Amazon S3 use the default encryption for the S3 bucket. To change the encryption used, click Show Advanced Options.

To use Amazon S3 encryption, from the Server-side Encryption Type dropdown, select Amazon S3.

To use AWS KMS encryption:

1. From the Server-Side Encryption Type dropdown list, select AWS KMS.
2. In the Server-side Encryption AWS KMS ID field, provide the KMS key ID. Note that if the KMS key doesn't exist in the same account that issues the command, you must provide the full key ARN instead of the key ID.

Note that after you save the new dataset, you cannot change the encryption type.

Providing Azure credentials

If the source type is Azure, provide the connection information:

Credentials fields for an Azure dataset

1. In the Account Name field, provide the name of your Azure account.
2. In the Account Key field, provide the access key for your Azure account.
3. To test the connection, click Test Azure Connection.
4. Click Save. Textual prompts you to select the dataset files.

Providing SharePoint credentials

If the source type is SharePoint, provide the credentials for the Entra ID application.

Credentials fields for a SharePoint dataset

The credentials must have the following application permissions (not delegated permissions):

• Files.Read.All - To see the SharePoint files
• Files.ReadWrite.All -To write redacted files and metadata back to SharePoint
• Sites.ReadWrite.All - To view and modify the SharePoint sites

To provide the credentials:

1. In the Tenant ID field, provide the SharePoint tenant identifier for the SharePoint site.
2. In the Client ID field, provide the client identifier for the SharePoint site.
3. In the Client Secret field, provide the secret to use to connect to the SharePoint site.
4. To test the connection, click Test SharePoint Connection.
5. Click Save. Textual prompts you to select the dataset files.