Managing data generation performance
During Tonic Structural data generation, performance bottlenecks typically come from one of the following sources:
Network IO. Specifically, the bandwidth capacity of the network that connects Structural to the database instances.
Disk IO. The disk IO of the databases.
Tonic server and workspace configuration. Structural performs several complex data computations and transformations. Depending on your workspace selections, these tasks can take a long time to perform.
In most cases, slow data generation times are caused by disk IO and network IO.
Network IO
When possible, ensure that Structural has a fast network pipe between Structural and each source and destination database.
It is always advisable to install Structural on or near the hardware that runs your database instances.
Disk IO
This is normally limited by the database hardware.
If you run in a public cloud, you can configure options to access faster disks.
Reducing data loads
To reduce the required disk and network IO, you can copy less data from the source to the destination.
In some cases, you don't need the data from every table, or from specific columns within a table. Or you might be happy with the data that is already in the destination, and so you don't need to copy it again from the source.
Here are some tips to reduce the data load:
For example, audit or transaction tables might not be needed for typical QA testing.
Avoid copying over large columns such as varchar(max), blob, XML, and JSON columns.
If you do not need the data in a column, then to reduce the required IO, either:
If the column is nullable, apply a NULL generator.
Apply a Constant generator
For subsequent generation runs from the same source database:
Configuring parallel processing
When you believe that the Structural server is the bottleneck, then to improve performance, you can tune the following settings that control parallel processing.
Settings that are not data connector-specific
The following settings are not limited to specific data connectors:
Data connector-specific settings
The following settings apply to specific data connectors:
Last updated