For the complete documentation index, see llms.txt. This page is also available as Markdown.

Configuring Apache Iceberg workspace data connections

During workspace configuration, under Connection Type, click Apache Iceberg.

Providing the source connection details

Under Source Settings, you provide:

  • An assumed role to use to connect to Amazon S3 and AWS Glue.

  • Optionally, a list of schemas to either include or exclude.

Providing the assumed role

Structural automatically retrieves the available schemas and source data based on the assumed role.

  1. In the Role ARN field, provide the Amazon Resource Name (ARN) for the role.

  2. In the Session Name field, provide the role session name. If you do not provide a session name, then Structural automatically generates a default unique value. The generated value begins with TonicStructural.

  3. In the Duration (in seconds) field, provide the maximum length in seconds of the session. The default is 3600, indicating that the session can be active for up to 1 hour. The provided value must be less than the maximum session duration that is allowed for the role.

  4. From the AWS Region dropdown list, select the AWS Region to send the authentication request to.

For each assumed role, Structural generates the external ID that is used in the assume role request. Your role’s trust policy must be configured to condition on your unique external ID.

Here is an example trust policy:

{
  "Version": "2012-10-17",
  "Statement": {
    "Effect": "Allow",
    "Principal": {
      "AWS": "<originating-account-id>"
    },
    "Action": "sts:AssumeRole",
    "Condition": {
      "StringEquals": {
        "sts:ExternalId": "7aab6da2-62ed-fae9-efb5-03b3b4daed8a"
      }
    }
  }
}

Limiting the included schemas

To limit the schemas to include:

  1. Toggle Limit Schemas to the on position.

  2. From the filter option dropdown list:

    1. To only include the listed schemas, select Include these schemas.

    2. To exclude the listed schemas, select Exclude these schemas.

  3. In the field, provide a semicolon-separated list of the schemas to either include or exclude.

Providing the destination connection details

Under Destination settings, you provide:

  • An assumed role to connect to Amazon S3 and AWS Glue.

  • Within Amazon S3, the warehouse and namespace where Structural writes the de-identified data.

Providing the assumed role

To provide the assumed role to use to connect to Amazon S3 and AWS Glue and write the destination data:

  1. In the Role ARN field, provide the Amazon Resource Name (ARN) for the role.

  2. In the Session Name field, provide the role session name. If you do not provide a session name, then Structural automatically generates a default unique value. The generated value begins with TonicStructural.

  3. In the Duration (in seconds) field, provide the maximum length in seconds of the session. The default is 3600, indicating that the session can be active for up to 1 hour. The provided value must be less than the maximum session duration that is allowed for the role.

  4. From the AWS Region dropdown list, select the AWS Region to send the authentication request to.

For each assumed role, Structural generates the external ID that is used in the assume role request. Your role’s trust policy must be configured to condition on your unique external ID.

Here is an example trust policy:

Providing the warehouse and namespace

To identify the specific location on Amazon S3 where Structural writes the destination data:

  1. In the Warehouse field, provide the path to the S3 bucket where Structural writes the destination data.

  2. In the Namespace field, provide the namespace where Structural writes the destination data.

Last updated

Was this helpful?