Links

Viewing workspace jobs and job details

Tonic runs the following types of jobs on a workspace:
  • Sensitivity scans, which analyze the source database to identify sensitive data.
  • Collection scans, which analyze the source data for a MongoDB workspace to determine the available fields in each collection, the field types, and how prevalent the fields are.
  • Data generation, data pipeline generation, and containerized generation jobs, which generate the destination data from the source data.
  • Upsert data generation jobs, which generate the intermediate database from the source database.
  • Upsert jobs, which use data from the intermediate database to add new rows to and update changed rows in the destination database. If the migration process is enabled, then it is a step in the upsert job.
  • SDK table statistics jobs. These jobs only run when you use the SDK to generate data in a Spark workspace, and the assigned generators require the statistics.
  • Model training jobs. These jobs only run on data science mode workspaces. A model training job shows the results of a model being trained. A trained model can be used to generate synthetic data.
You can view a list of jobs that ran on the workspace, and view details for individual jobs.

Viewing the list of jobs

The Job History page displays the list of jobs that ran on the workspace. The list includes the 100 most recent jobs.
Job History view
To display the Job History view:
  • On the workspace management view, in the workspace navigation bar, click Jobs.
  • On Workspaces view, from the dropdown menu in the Name column, select Jobs.

Information in the job list

For each job, the job list includes the following information:
  • Job ID - The identifier of the job. To copy the job ID, click the icon at the left of the row.
  • Type - The type of job.
  • Status - The current status of the job, and how long ago the job reached that status. When you hover over the status, a tooltip displays the actual timestamp for the status change, and a summary of how long the job ran. For queued jobs, to display a panel with information about why the job is queued, click the status value.
  • Submitted - The date and time when the job was submitted.
  • Completed - The date and time when the job finished running.

Job statuses

A job can have one of the following statuses:
  • Queued - The job is queued to run, but has not yet started. A job is queued for one of the following reasons:
    • Another job is currently running on the same workspace. For example, you cannot run a sensitivity scan and a data generation, or multiple data generations, at the same time on the same workspace. This is true regardless of the number of workers on the instance.
    • There isn't an available worker on the instance to run the job. A Tonic instance with one worker can only run one job at a time. If a job from one workspace is currently running, a job from another workspace cannot start until the first job is finished.
    To view information about why a job is queued, click the status value.
  • Running - The job is in progress.
  • Canceled - The job is canceled.
  • Completed - The job completed successfully.
  • Failed - The job failed to complete.
Each of these statuses has a corresponding "with warnings" status. For example, Running with warnings, Completed with warnings. A "with warnings" status indicates that the job had at least one warning at the time of the request.

Filtering the job list

You can filter the list by either the type or the status.
To filter the list by the job type:
Job type filter options for the Job History view
  1. 1.
    Click the filter icon in the Type column heading. By default, all types are included, and none of the checkboxes are checked.
  2. 2.
    To only include specific types of jobs, check the checkbox next to each type to include. Checking all of the checkboxes has the same effect as unchecking all of the checkboxes.
To filter the list by the job status:
Job status filter options for the Job History view
  1. 1.
    Click the filter icon in the Status column heading. The status panel displays all of the statuses that are currently in the list. For example, if there are no Queued jobs, then the Queued status is not in the list. By default, all of the statuses are included, and none of the checkboxes are checked.
  2. 2.
    To only include jobs that have specific statuses, check the checkbox next to each status to include. Checking all of the checkboxes has the same effect as unchecking all of the checkboxes.

Sorting the job list

You can sort the jobs by either the submission or completion timestamp.
To sort by submission date, click the Submitted column heading. To reverse the sort order, click the heading again.
To sort by completion date, click the Completed column heading. To reverse the sort order, click the heading again.

Viewing details for a selected job

For jobs other than Queued jobs, you can display details about the workspace and the job progress.
From the Job History view, to display the details for a job, click the job row.
Job details page for a data generation job

Workspace information

The left side of the job details view contains the workspace information.
For a sensitivity scan, the workspace information is limited to the owner, database type, and worker version.
For a data generation job, the workspace information also includes:
  • Whether subsetting, post-job scripts, or webhooks are used.
  • The number of schemas, tables, and columns in the source database.
  • The number of schemas, tables and columns in the destination database.

Job Log

The Job Log tab shows the start date, start time, and duration of the job, followed by the list of job process steps.

Privacy Report

For data generation jobs, the Privacy Report tab displays the number of at-risk, protected, and not sensitive columns in the source database.
Privacy Report tab on the job details page
At-risk columns contain sensitive data, but still have Passthrough as the assigned generator.
Protected columns have an assigned generator other than Passthrough.
Not sensitive columns have Passthrough as the assigned generator, but do not contain sensitive data.

Copying the job ID

The job identifier is a unique identifier for the job. To copy the job ID, either:
  • From the Job History view, click the copy (
    ) icon in the leftmost column.
  • From the job details view, click the copy (
    ) icon next to the job ID.

Canceling a job

You can cancel Queued or Running jobs.
For jobs with those statuses, the rightmost column in the job list contains a cancel icon.
Job list with a Running job that can be canceled
To cancel the job, click the icon.

Downloading job information

For workspaces that are configured to write destination data to container artifacts, the Job History view also provides access to those artifacts. For more information, go to Viewing and downloading container artifacts.

Downloading the job logs for a job

Required workspace permission: Download job logs
To download diagnostic logs, you must have the Enable diagnostic logging global permission.
For all jobs, the job logs provide detailed information about the job processing. Tonic support might request the job logs to help diagnose issues.
For upsert jobs where the migration process is enabled, and you configured the GET Schema Change Logs endpoint, the upsert job logs include the migration process logs.

Where to download the job logs

You can download the job logs from the Job History view or the job details view. The download includes up to 1MB of log entries.
On the Job History view, to download the logs for a job, click the download icon in the rightmost column.
On the job details view, to download the logs for a job, click Download, then select Job Logs.

Downloading diagnostic logs

By default, Tonic redacts sensitive values from the job logs. To help support troubleshooting, you can configure data connectors or an individual data generation job to create unredacted versions of the log files, referred to as diagnostic logs. For more information, go to Redacted and diagnostic (unredacted) logs.
To access diagnostic log files, you must have the Enable diagnostic logging global permission.
If you do not have the Enable diagnostic logging global permission, then you cannot download the logs for that job. The download option is disabled.

Downloading the Privacy Report for a data generation job

Required workspace permission: View and download Privacy Report
From the job details view, you can download a Privacy Report file that provides an overview of the current protection status of the database columns based on the workspace configuration at the time that the job ran.
You can download either:
  • The Privacy Report .csv file, which provides details about the table columns, the column content, and the current protection configuration.
  • The Privacy Report PDF file, which provides charts that summarize the privacy ranking scores for the table columns. It also includes the table from the .csv file.
To display the download options, click Download. In the download menu:
Download menu for a data generation job
  • To download the Privacy Report .csv file, click Privacy Report CSV.
  • To download the Privacy Report PDF file, click Privacy Report PDF.
For more information about the Privacy Report files and their content, go to Using the Privacy Report to verify data protection.

Downloading the CloudWatch logs for a data generation job

For workspaces that are connected to Amazon Redshift or Snowflake on AWS databases, the data generation job requires multiple calls to a Lambda function. For these data generation jobs, the CloudWatch logs monitor the progress of and display errors for these Lambda function calls.
To download the CloudWatch logs for a data generation job, on the job details view, click Download, then select CloudWatch Logs.
The CloudWatch Logs option only displays for Amazon Redshift and Snowflake on AWS data generation jobs.

Downloading Oracle SQL Loader log files

Required workspace permission: Download SqlLdr Files
For an Oracle data generation, if both of the following are true:
  • The data generation job ran SQL Loader (sqlldr).
  • sqlldr either failed or succeeded with errors.
Then to download the sqlldr log files, click Download, then select sqlldr Logs.
Last modified 16d ago