Scheduling data generation
Last updated
Last updated
Required workspace permission: Run data generation
Not available for Structural free trials, or for expired pay-as-you-go accounts.
You can also configure data generation to run on a regular schedule. For example, if new data is added regularly to your source database, then you might want to automatically run data generation jobs to transform the new data.
To schedule the job, you configure one or more configurations. Each configuration includes a cron expression to specify the schedule.
You manage the job schedule from Jobs view.
To create a schedule for a job, click Create Schedule.
To edit an existing job schedule, click Edit Schedule.
Set up the schedule configurations, then click Save.
To add a configuration to the schedule:
Click Add Configuration.
By default, the configuration is active, and Active is in the on position. To have Structural ignore the configuration, toggle Active to the off position.
In the Cron Expression field, provide the schedule expression. The default value is
0 0 * * *, which means to run the job every day at midnight.
From the time zone dropdown list, select the time zone to use for the schedule.
A cron expression is made up of five values separated by a space.
<minute> <hour> <day of month> <month> <day of week>
Where:
<minute>
The minute at which to run the job. Is a value between 0 and 59.
Use * to run every minute.
Use , to separate a list of values. For example, to run at 10 and 40 past the hour, use 10,40.
Use - to provide a range of values. For example, to run every minute between 20 and 25 past the hour, use 20-25.
<hour>
The hour at which to run the job. Is a value between 0 and 23.
Use * to run every hour.
Use , to separate a list of values. For example, to run at 11:00 AM and 2:00 PM, use 11,14.
Use - to provide a range of values. For example, to run every hour between 5:00 PM and 9:00 PM, use 17-21.
<day of month>
The day of the month on which to run the job. Is a value between 1 and 31.
Use * to run every day.
Use , to separate a list of values. For example, to run on the 5th and the 15th of the month, use 5,15.
Use - to provide a range of values. For example, to run on the 8th through the 12th of the month, use 8-12.
<month>
The month in which to run the job. Can use either numbers (1 through 12) or 3-letter abbreviations (JAN through DEC).
Use * to run every month.
Use , to separate a list of values. For example, to run in March and September, use 3,9 or MAR,SEP.
Use - to provide a range of values. For example, to run every month between June and August, use 6-8 or JUN-AUG.
<day of week>
The day of the week on which to run the job. Can use either numbers (0 through 6) or 3-letter abbreviations (SUN through SAT).
Use * to run every day.
Use , to separate a list of values. For example, to run every Monday, Wednesday, and Friday, use 1,3,5 or MON,WED,FRI.
Use - to provide a range of values. For example, to run every week from Tuesday through Thursday, use 2-4 or TUE-THU.
Here are some example expressions:
15 * * * *
Every hour at 15 minutes past the hour.
15 10,14,18 * * *
Every day at 10:15 AM, 2:15 PM, and 6:15 PM.
0 0 15 * *
Midnight on the 15th of every month.
0 0 10-15 6 *
Midnight on June 10 through June 15th.
30 12 * * TUE
Every Tuesday at 12:30 PM.
To remove a configuration from the schedule, click its Delete option.
If you remove all of the configurations and then save, the job is no longer scheduled.
When a job runs on a schedule, you cannot configure the options that are available from the Confirm Generation panel when you run data generation manually.
For a workspace that has subsetting configured, whether the scheduled job generates a subset is based on the Use Subsetting toggle on Subsetting view.
For a workspace that has upsert enabled, whether a scheduled job completes both the initial data generation and the upsert process is determined by the workspace setting Automatically Start Upsert After Successful Data Generation.
If this is in the on position, then the scheduled job does both data generation and upsert.
If this is in the off position, then the scheduled job only does the data generation to the intermediate database.
For a scheduled job, whether the job uses diagnostic logging is based on the environment setting configuration for the data connector.
A scheduled job does not generate performance traces.
