Enabling point-in-time recovery on source tables
For all source tables, you must enable point-in-time recovery.
If point-in-time recovery is not enabled for every source table, then data generation fails.
Structural automatically enables point-in-time recovery when it creates the destination tables.
Setting up S3 buckets for the source and destination databases
For each source and destination database, you must set up a corresponding S3 bucket. Structural uses the S3 buckets for temporary storage.
You must use a separate S3 bucket for each database. You cannot use the same S3 bucket for both the source and destination database.
The source database user must have access to the source S3 bucket.
The destination database user must have access to the destination S3 bucket.
Structural does delete files from the S3 buckets after each job. However, to ensure that files do not accumulate, set up lifecycle rules to empty the S3 buckets.
Required permissions for source database user roles
The source database user must have permissions for DynamoDB and Amazon S3. We recommend that you create a custom policy for each service.
Note that the required Amazon S3 permissions include access to the S3 bucket that you created for the source database.
Required DynamoDB source user permissions
Here is an example of a DynamoDB policy that grants the required permissions for the source database user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:DescribeTable",
"dynamodb:ListTables",
"dynamodb:Scan",
"dynamodb:Query",
"dynamodb:ExportTableToPointInTime",
"dynamodb:DescribeExport",
"dynamodb:ListExports",
"dynamodb:DescribeContinuousBackups"
],
"Resource": "*"
}
]
}
Required Amazon S3 source user permissions
For Amazon S3, the source database user requires access to the S3 bucket that you created for the source database.
Here is an example of an Amazon S3 policy that grants the required permissions for the source database user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObjectAcl",
"s3:GetObject",
"s3:AbortMultipartUpload",
"s3:DeleteObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::<S3-source-bucket-name>/*"
]
},
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<S3-source-bucket-name>"
]
}
]
}
Required permissions for destination database user roles
Required DynamoDB destination user permissions
Here is an example of a DynamoDB policy that grants the required permissions for the destination database user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:CreateTable",
"dynamodb:BatchGetItem",
"dynamodb:DescribeImport",
"dynamodb:BatchWriteItem",
"dynamodb:PutItem",
"dynamodb:ListTables",
"dynamodb:ListImports",
"dynamodb:DeleteTable",
"dynamodb:ImportTable",
"dynamodb:UpdateContinuousBackups",
"dynamodb:DescribeTable",
"dynamodb:DescribeContinuousBackups"
],
"Resource": "*"
}
]
}
Required Amazon S3 destination user permissions
For Amazon S3, the destination database user requires access to the S3 bucket that you created for the destination database.
Here is an example of an Amazon S3 policy that grants the required permissions for the destination database user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:ListBucket",
"s3:DeleteObject",
"s3:PutObjectAcl",
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": [
"arn:aws:s3:::<destination-S3-bucket>/*",
"arn:aws:s3:::<destination-S3-bucket>"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "s3:ListAllMyBuckets",
"Resource": "*"
}
]
}
Required Amazon CloudWatch destination user permissions
For Amazon S3, the destination database user requires access to Amazon CloudWatch.
Here is an example of an Amazon CloudWatch policy that grants the required permissions for the destination database user:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents",
"logs:PutRetentionPolicy"
],
"Resource": "arn:aws:logs:<destination-region>:<destination-account>:log-group:/aws-dynamodb/*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"logs:DescribeLogStreams",
"logs:CreateLogGroup"
],
"Resource": "arn:aws:logs:<destination-region>:<destination-account>:log-group:/aws-dynamodb/*"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": "logs:DescribeLogGroups",
"Resource": "arn:aws:logs:<destination-region>:<destination-account>:log-group:*"
}
]
}