Self-hosted LLM configuration

Self-hosted customers can choose whether to enable AI features on their instance.

A self-hosted instance can either:

Use an LLM that Tonic.ai hosts. This is the default option.
Configure a connection to a supported LLM provider.
Not use an LLM at all. This disables all Structural AI features.

You configure the connection either:

From the AI Settings tab on Structural Settings.
Using environment settings. The settings can be added to the Environment Settings list on Structural Settings.

If you enable AI features, then Structural provides additional options to configure data use by the LLM.

Supported providers

Tonic Structural supports the following LLM providers:

Anthropic
Anthropic on Amazon Bedrock
OpenAI. Note that we are continuing to test OpenAI for the Structural Agent.

Using the hosted LLM

Structural provides a hosted LLM that you can use to support the Structural AI features. The hosted LLM uses Anthropic on Amazon Bedrock.

For new self-hosted instances, this is the default option.

The LLM is hosted in multiple locations. Structural routes the request through a SaaS proxy to the appropriate location for the customer.

Using AI settings view

On AI settings view, to use the hosted LLM:

From the LLM Provider dropdown list, select Tonic.ai-hosted LLM.
From the from the Region dropdown list, select the hosted LLM location to use.
To test the connection to the hosted LLM, click Test Tonic.ai-hosted LLM Connection.

Using environment settings

To use the environment settings to select the hosted LLM:

Set TONIC_LLM_PROVIDER to TonicHosted.
Set TONIC_HOSTED_REGION to either US or EU.

Allowlisting the SaaS proxy

If you use the hosted LLM, make sure to allowlist the SaaS proxy, either:

http://us-east-1.saasproxy.tonic.ai/ or 98.84.248.119 (US location)
http://eu-central-1.saasproxy.tonic.ai/ or 3.120.214.225 (EU location)

Disabling AI features on Structural

To not connect to an LLM, which disables Structural AI features, from the LLM Provider dropdown list, select Do not use an LLM.

To use an environment setting to disable Structural AI features, set TONIC_LLM_PROVIDER to LlmOptOut.

Using Anthropic

Required models

If you use a hosted Anthropic deployment, Structural requires the following models:

claude-opus-4-6
claude-sonnet-4-5
claude-haiku-4-5

Using AI settings to configure the Anthropic connection

From AI Settings, to configure Anthropic as your LLM:

From the LLM Provider dropdown list, select Anthropic.
In the Anthropic Endpoint field, enter the Anthropic endpoint URL for generative AI services. Should include the endpoint base URL for the Anthropic API, without the version. The default is https://api.anthropic.com.
In the Anthropic API Key field, enter the API key for Anthropic. In production environments, make sure that this is kept secure and encrypted.
Optionally, in the Model Name field, specify the model to use. If you do not specify a model, then the default is:
- For lightweight, quick tasks, claude-haiku-4-5.
- For standard tasks, claude-sonnet-4-5.
- For complex, high-quality tasks, claude-opus-4-6.
To test the connection, click Test Anthropic Connection.

Using environment settings to configure the Anthropic connection

To use environment settings to connect to Anthropic:

TONIC_LLM_PROVIDER - Set to Anthropic.
TONIC_ANTHROPIC_ENDPOINT - The Anthropic endpoint URL. Include the endpoint base URL for the Anthropic API, without the version.
TONIC_ANTHROPIC_API_KEY - The Anthropic API key.
TONIC_LLM_MODEL - The model to use.

Using Anthropic on Amazon Bedrock

When you use Anthropic on Amazon Bedrock, you can either:

Use the environment configuration for AWS authentication.
Provide an Amazon Bedrock API key.

Required IAM permissions for Amazon Bedrock

To call Amazon Bedrock, most model and regions require the use of cross-Region inference profiles. The AWS documentation includes information about model support by AWS Region.

The IAM principal that authenticates the request must be granted the following permissions.

Actions

bedrock:InvokeModel
bedrock:InvokeModelWithResponseStream

Resources

The principal must be able to invoke the cross-Region inference profiles that apply to your Region and model.

By default, Structural uses the following models:

us.anthropic.claude-haiku-4-5-20251001-v1:0
us.anthropic.claude-sonnet-4-5-20250929-v1:0
us.anthropic.claude-opus-4-6-v1

AWS Regions

Cross-Region inference also requires invoke permission on the underlying foundation model in each AWS Region that the profile spans.

For example, for the US, the principal needs the invoke permission in:

us-east-1
us-east-2
us-west-2

For the recommended IAM policy structure, go to the AWS documentation.

Selecting Anthropic on Amazon Bedrock as your LLM provider

From AI Settings, to use Anthropic on Amazon Bedrock, from the LLM Provider dropdown list, select Anthropic+Bedrock.

Using the environment configuration for authentication

To use the environment configuration for authentication, click Environment Configuration.

Structural uses the credentials set in the following environment settings:

TONIC_AWS_ACCESS_KEY_ID - An AWS access key that is associated with an IAM user or role.
TONIC_AWS_SECRET_ACCESS_KEY - The secret key that is associated with the access key.
TONIC_AWS_REGION - The AWS Region to send the authentication request to.

If these environment settings are not configured, then Structural uses the AWS SDK default resolving chain.

Using an Amazon Bedrock API key for authentication

From AI Settings, to use an Amazon Bedrock API key for authentication:

Click API Key.
In the Amazon Bedrock API Key field, provide the API key.

Selecting the model to use

Optionally, in the Model Name field, specify the model to use. In most cases, the value is a cross-Region inference profile. For combinations of Region and model that do not require cross-Region inference, the you can provide a model name.

If you do not specify a model, then the default is:

For lightweight, quick tasks, us.anthropic.claude-haiku-4-5-20251001-v1:0.
For standard tasks, us.anthropic.claude-sonnet-4-5-20250929-v1:0.
For complex, high-quality tasks, us.anthropic.claude-opus-4-6-v1.

Using environment settings to configure the Anthropic on Amazon Bedrock connection

To use environment settings to connect to Anthropic on Amazon Bedrock:

TONIC_LLM_PROVIDER - Set to AnthropicBedrock.
TONIC_BEDROCK_API_KEY - To use an API key for authentication, set this to the Amazon Bedrock API key. To use the environment configuration for authentication, do not set this setting.
TONIC_LLM_MODEL - The model to use.

Using OpenAI

Supported deployments

Structural supports any OpenAI deployment that:

Supports API key authentication.
Supports the OpenAI Responses API.
Supports OpenAI proprietary models, such as GPT-4/5.x.

Required model deployments for hosted deployments

If you use a hosted OpenAI deployment such as Microsoft Foundry, Structural requires the following model deployments. The deployment name must match the model name.

gpt-4.1-mini
gpt-4.1
gpt-5.2

Using AI Settings to configure the OpenAI connection

From AI Settings, to use OpenAI as your LLM provider:

From the LLM Provider dropdown list, select OpenAI.
In the OpenAI Endpoint field, enter the OpenAI endpoint URL to use for AI-enhanced features. Should include the endpoint base URL for the OpenAI API, without the version. The default is https://api.openai.com. Here is an example URL for Microsoft Foundry: https://my-foundry-instance.ai.azure.com/openai
In the OpenAI API Key field, enter the API key for OpenAI.
Optionally, in the Model Name field, specify the model to use. If you do not specify a model, then the default is:
- For lightweight, quick tasks, gpt-4.1-mini.
- For standard tasks, gpt-4.1.
- For complex, high-quality tasks, gpt-5.2.
To test the connection, click Test OpenAI Connection.

Using environment settings to configure the OpenAI connection

To use environment settings to connect to OpenAI:

TONIC_LLM_PROVIDER - Set to OpenAI.
TONIC_OPENAI_ENDPOINT - The OpenAI endpoint URL. Include the endpoint base URL for the OpenAI API, without the version.
TONIC_OPENAI_API_KEY - The API key for OpenAI.
TONIC_LLM_MODEL - The model to use.

Configuring AI-based Structural functionality

Disabling LLM-based sensitivity detection

By default, Structural sensitivity detection includes LLM-based sensitivity detection, which sends database information to an LLM.

To disable the LLM-based sensitivity detection, set the environment setting TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS to false.

You can set this from the Environment Settings tab on Structural Settings.

LLM-based sensitivity detection also processes LLM-based sensitivity rules. If you disable LLM-based sensitivity detection, then even if you enable LLM-based sensitivity rules, the sensitivity scan does not process any LLM-based sensitivity rules.

Enabling LLM-based sensitivity rules

On a self-hosted instance, to allow LLM-based sensitivity rules to be created and processed:

LLM-based sensitivity rules must be enabled - The environment setting TONIC_LLM_ENABLE_CUSTOM_SENSITIVITY_RULES must be set to true. This allows you to create LLM-based sensitivity rules.
LLM-based sensitivity detection must be enabled - The environment setting TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS must be set to true. This allows LLM-based sensitivity rules to be processed. If this is false, then those rules are ignored.

Excluding sample data from LLM-based sensitivity detection

By default, when LLM-based sensitivity detection is enabled, it sends both the database schema and sample source data values to the LLM.

To exclude the contextual source data, and only send the schema, set the environment setting TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS_SAMPLE_DATA to false.

You can set this from the Environment Settings tab on Structural Settings.

Excluding sample data from all LLM-based features

To configure Structural to never send sample data for any LLM-based feature, either:

On AI Settings view, under LLM Data Handling, uncheck Allow LLM access to sample data.
Set the environment setting TONIC_LLM_ENABLE_SAMPLE_DATA to false. You can configure this setting from the Environment Settings tab on Structural Settings.

Last updated 27 days ago

Was this helpful?