Job types and statuses

Job types

Tonic Structural runs the following types of jobs on a workspace:

• Sensitivity scans - Analyze the source database to identify sensitive data.

• Collection scans - Analyze the source data for a MongoDB workspace to determine the available fields in each collection, the field types, and how prevalent the fields are.

• Schema retrieval jobs - Refresh the cached version of the source database schema.

• Data generation, data pipeline generation, and containerized generation jobs - Generate the destination data from the source data.

• Upsert data generation jobs - Generate the intermediate database from the source database.

• Upsert jobs - Use data from the intermediate database to add new rows to and update changed rows in the destination database. If the migration process is enabled, then it is a step in the upsert job.

• SDK table statistics jobs - Only run when you use the SDK to generate data in a Spark workspace, and the assigned generators require the statistics.

Job statuses

A job can have one of the following statuses:

• Queued - The job is queued to run, but has not yet started. A job is queued for one of the following reasons:

  • Another job is currently running on the same workspace. For example, you cannot run a sensitivity scan and a data generation, or multiple data generations, at the same time on the same workspace. This is true regardless of the number of workers on the instance. On Structural Cloud, there is also a limit on the number of concurrent running jobs for each organization. When that maximum is reached, a new job remains queued until a current running job completes.

  • There isn't an available worker on the instance to run the job. A Structural instance with one worker can only run one job at a time. If a job from one workspace is currently running, a job from another workspace cannot start until the first job is finished.

  To view information about why a job is queued, click the status value.

• Running - The job is in progress.

• Canceled - The job is canceled.

• Completed - The job completed successfully.

• Failed - The job failed to complete.

Each of these statuses has a corresponding "with warnings" status. For example, Running with warnings, Completed with warnings. A "with warnings" status indicates that the job had at least one warning at the time of the request.