Webhooks

Tonic Structural allows you to set up webhooks that are triggered when either:

• A data generation or upsert job completes successfully, fails, or is canceled. Webhooks are not supported for other types of jobs.

• Structural detects a schema change in the source data.

Structural supports the following types of webhooks:

• A custom webhook that fires an HTTP POST request. For example, you might use a webhook to post a message to a Slack channel.

• A GitHub workflow webhook that runs a GitHub workflow.

Webhooks allow Structural to integrate more seamlessly into your workflow. They can be used to trigger actions in other systems.

Child workspaces never inherit the webhooks configuration from their parent workspace. Child workspaces always have their own webhooks.

Displaying the list of webhooks

You manage webhooks from the Post-Job Actions view. To display the Post-Job Actions view, either:

• On the workspace management view, in the workspace navigation bar, click Post-Job Actions.

• On Workspaces view, from the dropdown menu in the Name column, select Post-Job Actions.

On the Post-Job Actions view, the Webhooks list contains the list of webhooks.

For each webhook, the list contains:

• A toggle to enable or disable the webhook.

• Whether the webhook is custom or GitHub.

• The name of the webhook.

• How the webhook is triggered.

• Information about the webhook configuration.

• The user who created the webhook.

• The date and time when the webhook was most recently updated.

• Options to edit and delete the webhook.

Creating and managing webhooks

Creating a webhook

To create a webhook:

1. In the Webhooks panel, click Create Webhook.

1. On the webhook configuration dialog, in the Webhook Name field, provide a name for the webhook.

2. From the Webhook Trigger Type dropdown list, select the type of trigger for the webhook.

   • To trigger the webhook based on a data generation job, select Data Generation.

   • To trigger the webhook based on a detected schema change, select Schema Change

   After you save the webhook, you cannot change the trigger type.

3. For a data generation webhook, under Trigger Events, select the data generation job events that trigger the webhook. The webhook can be triggered when a job succeeds, a job fails, or a job is canceled. To trigger a webhook in response to an event, check the event checkbook. For example, to trigger the webhook when a job is canceled, check the Job Cancelled checkbox. After you save the webhook, you can change the selected events.

4. From the Webhook Type dropdown list, select the type of webhook to create:

   1. To create a webhook that posts an HTTP POST request, select Custom.

   2. To create a webhook that runs a GitHub workflow, select GitHub Workflow.

   After you save the webhook, you cannot change the webhook type.

5. Configure the webhook.

   • Custom webhook configuration

   • GitHub workflow webhook configuration

6. To save the webhook, click Save. The webhook is added to the Webhooks list.

Editing a webhook

To edit a webhook:

1. In the Webhooks list, click the edit icon for the webhook.

2. On the webhook configuration dialog, update the webhook configuration. You cannot change the webhook trigger type or or the webhook type.

   • Custom webhook configuration

   • GitHub workflow webhook configuration

3. Click Save.

Enabling and disabling webhooks

You use the toggle at the left of each webhook to determine whether the webhook is enabled.

When the toggle is in the on position, the webhook is enabled and responds to the configured trigger.

When the toggle is in the off position, the webhook is not enabled and does not respond to the configured trigger.

Deleting a webhook

To delete a webhook:

1. In the Webhooks list, click the delete icon for the webhook.

2. On the confirmation dialog, click Delete.

Custom webhook configuration

A custom webhook, which sends an HTTP POST request, includes the following requirements and configuration.

Confirming the notifications endpoint location

Webhooks require access to the Structural notifications server. The notifications server URL and port are set as the value of the environment setting TONIC_NOTIFICATIONS_URL.

On a Docker deployment, the default value is https://tonic_notifications:7001. For a Kubernetes deployment deployed using Structural's provided Helm chart, the default value is https://tonic-notifications:7001.

If the notifications server on your instance does not match the default value, then you must update the value of TONIC_NOTIFICATIONS_URL.

Required information for a custom webhook

Before you create a webhook, make sure that you have the required information.

Webhook URL

Each webhook requires a webhook URL. This is the URL that receives the webhook message.

The URL cannot resolve to a private IPv4 range.

The application that you send the webhook to should provide information about how to obtain the URL. For example, for information on how to generate the webhook URL for a Slack notification, go to Sending messages using Incoming Webhooks in the Slack documentation.

Header values

Check whether the webhook requires any header values.

For example, an application might require:

• A content-type header. For example, Content-type: application/json

• The version of an API to use. This might be needed to send an API call to perform an action based on the job status. For example, Accept: application/vnd.pagerduty+json;version=2

• Authorization for a third-party service. For example, Authorization: Bearer <token value>

Message properties

By default, the webhook message contains the workspace identifier and name, the job identifier, and the job status.

You must also determine whether your application requires any other properties.

For example, for a Slack notification webhook, you must provide a text property that contains the text of the Slack notification.

Configuring the webhook URL and headers

For a data generation webhook, on the Settings & Headers tab:

1. In the Webhook URL field, provide the URL to send the webhook request to.

2. By default, a webhook requires SSL certificate validation. To bypass the validation, and trust the server certificate, check Trust the Server Certificate (bypass SSL certificate validation). You can use this option if the server has a trustworthy self-signed certificate.

3. The header list always contains a Content-Type header. The default value is application/json. You cannot delete the Content-Type header, but you can change the value.

4. To add custom header values for the webhook request:

   1. To add a header row, click Add Header.

   2. In the Header Name field for each header, provide the header name.

   3. In the Header Value field for each header, provide the header value.

   4. To remove a header row, click its delete icon.

Customizing the webhook request body

On the Message Body tab, you can customize the body of the request. The message body is sent as a JSON payload that consists of a set of keys and values.

For each property, the Property Name field contains the key, and the Property Value field contains the value.

Default properties

By default, the message body contains the following properties. The values are variables that are replaced by the actual values for the triggering event. You can use these variables in the values of your custom properties.

For a data generation webhook, the default properties are:

• jobId - The identifier of the job. To include the job ID in a custom property value, use the {jobId} variable.

• jobStatus - The status of the job. To include the job status in a custom property value, use the {jobStatus} variable.

• jobType - The type of job (data generation or upsert). To include the job type in a custom property, use the {jobType} variable.

• workspaceId - The identifier of the workspace. To include the workspace ID in a custom property value, use the {workspaceId} variable.

• workspaceName - The name of the workspace. To include the workspace name in a custom property value, use the {workspaceName} variable.

For a schema change webhook, the default properties are:

• workspaceId - The identifier of the workspace. To include the workspace ID in a custom property value, use the {workspaceId} variable.

• workspaceName - The name of the workspace. To include the workspace name in a custom property value, use the {workspaceName} variable.

• {schemaChangesList} - The list of detected schema changes. To include the schema change list in a custom property value, use the {schemaChangesList} variable.

• {formattedMessage} - A human-readable message that describes the schema changes. To include the message in a custom property value, use the {formattedMessage} variable.

Adding properties

You can also add other properties to the message body that are needed for the particular webhook. For example, for a Slack notification webhook, you provide a text property that contains the text of the notification.

To add a property:

1. Click Add Property.

2. In the Property Name field, provide the key name.

3. In the Property Value field, provide the value. You can include the default variables in the value. The following example of a text value for a Slack notification includes the job type, job identifier, workspace name, and job status: {jobType} job {jobId} for workspace {workspaceName} completed with a status of {jobStatus).

Removing properties

To remove a property, click its delete icon.

Previewing the request JSON

The Preview tab contains a preview of the JSON body of the request. In the preview, the variables are replaced by sample values.

To copy the JSON to the clipboard, click Copy to clipboard. You can then, for example, use the copied JSON to test the webhook request in another tool such as Postman.

Testing the webhook

From the webhook configuration dialog, you can send a test request. The test request includes the configured headers and message body. The message body uses sample values for the variables.

To send a test request, click Test Webhook.

GitHub workflow webhook configuration

A GitHub workflow webhook, which runs a GitHub workflow, has the following configuration.

Connecting to the GitHub repository

Before you can select the workflow to run, you must connect to the GitHub repository where the workflow is configured.

On the Settings & Triggers tab:

1. In the GitHub Owner field, provide the owner of the repository.

2. In the GitHub Repository field, provide the name of the repository.

3. In the Personal Access Token field, provide a GitHub access token to use to connect to the repository.

After you provide the connection information, Structural attempts to use it to connect to GitHub.

Selecting the GitHub workflow to trigger

If Structural is able to connect to GitHub, it retrieves the available workflows in the repository.

From the Workflow Selection dropdown list, select the workflow to run when the webhook is triggered.

Providing the workflow input values

When you select a workflow, Structural populates the Workflow Inputs tab with the inputs for the workflow.

For each input, provide the value to use. Depending on the input, you might enter text or a number, select from a dropdown list, or set a toggle.

In text fields, you can use the same default properties that are available for custom webhooks.

Previewing the GitHub input message

The Preview tab displays a preview of the message that is sent to GitHub.

To copy the message to the clipboard, click Copy to clipboard.

Testing the Github workflow webhook

To test the workflow, click Test Workflow Dispatch.

This triggers the GitHub action.