Ctrlk
Tonic.ai

Self-hosted LLM configuration

Self-hosted customers can choose whether to enable AI features on their instance.

A self-hosted instance can either:

  • Use an LLM that Tonic.ai hosts. This is the default option.

  • Configure a connection to a supported LLM provider.

  • Not use an LLM at all. This disables all Structural AI features.

You configure the connection either:

  • From the AI Settings tab on Structural Settings.

  • Using environment settings. The settings can be added to the Environment Settings list on Structural Settings.

If you enable AI features, then Structural provides additional options to configure data use by the LLM.

Supported providers

Tonic Structural supports the following LLM providers:

  • Anthropic

  • Anthropic on Amazon Bedrock

  • OpenAI. Note that we are continuing to test OpenAI for the Structural Agent.

Using the hosted LLM

Structural provides a hosted LLM that you can use to support the Structural AI features. The hosted LLM uses Anthropic on Amazon Bedrock.

For new self-hosted instances, this is the default option.

The LLM is hosted in multiple locations. Structural routes the request through a SaaS proxy to the appropriate location for the customer.

On the AI settings view, to use the hosted LLM:

  1. From the LLM Provider dropdown list, select Tonic.ai-hosted LLM.

  2. To test the connection to the hosted LLM, click Test Tonic.ai-hosted LLM Connection.

To use environment settings to select and configure the hosted LLM:

  • Set TONIC_LLM_PROVIDER to TonicHosted.

  • Set TONIC_HOSTED_REGION to either US or EU.

If you use the hosted LLM, make sure to allowlist the SaaS proxy, either:

  • http://us-east-1.saasproxy.tonic.ai/ or 98.84.248.119 (US location)

  • http://eu-central-1.saasproxy.tonic.ai/ or 3.120.214.225 (EU location)

Disabling AI features on Structural

To not connect to an LLM, which disables Structural AI features, from the LLM Provider dropdown list, select Do not use an LLM.

AI Settings page with configuration to not use an LLM

To use an environment setting to disable Structural AI features, set TONIC_LLM_PROVIDER to LlmOptOut.

Using Anthropic

Required models

If you use a hosted Anthropic deployment, Structural requires the following models:

  • claude-opus-4-6

  • claude-sonnet-4-5

  • claude-haiku-4-5

Using AI settings to configure the Anthropic connection

From AI Settings, to configure Anthropic as your LLM:

AI Settings page configured to use Anthropic as the LLM provider

  1. From the LLM Provider dropdown list, select Anthropic.

  2. In the Anthropic Endpoint field, enter the Anthropic endpoint URL for generative AI services. Should include the endpoint base URL for the Anthropic API, without the version. The default is https://api.anthropic.com.

  3. In the Anthropic API Key field, enter the API key for Anthropic. In production environments, make sure that this is kept secure and encrypted.

  4. Optionally, in the Model Name field, specify the model to use. If you do not specify a model, then the default is:

  • For lightweight, quick tasks, claude-haiku-4-5.

  • For standard tasks, claude-sonnet-4-5.

  • For complex, high-quality tasks, claude-opus-4-6.

  1. To test the connection, click Test Anthropic Connection.

Using environment settings to configure the Anthropic connection

To use environment settings to connect to Anthropic:

  • TONIC_LLM_PROVIDER - Set to Anthropic.

  • TONIC_ANTHROPIC_ENDPOINT - The Anthropic endpoint URL. Include the endpoint base URL for the Anthropic API, without the version.

  • TONIC_ANTHROPIC_API_KEY - The Anthropic API key.

  • TONIC_LLM_MODEL - The model to use.

Using Anthropic on Amazon Bedrock

When you use Anthropic on Amazon Bedrock, you can either:

  • Use the environment configuration for AWS authentication.

  • Provide an Amazon Bedrock API key.

AI Settings page configured to use Anthropic on Amazon Bedrock using environment credentials

Required IAM permissions for Amazon Bedrock

To call Amazon Bedrock, Structural uses cross-region inference profiles. The IAM principal that authenticates the request must be granted the following permissions.

Actions

  • bedrock:InvokeModel

  • bedrock:InvokeModelWithResponseStream

Resources

The principal must be able to invoke the following cross-region inference profiles:

  • us.anthropic.claude-haiku-4-5-20251001-v1:0

  • us.anthropic.claude-sonnet-4-5-20250929-v1:0

  • us.anthropic.claude-opus-4-6-v1

AWS Regions

Cross-region inference also requires invoke permission on the underlying foundation model in each AWS Region that the profile spans. The required profiles currently span the following AWS Regions:

  • us-east-1

  • us-east-2

  • us-west-2

For the recommended IAM policy structure, go to the AWS documentation.

Selecting Anthropic on Amazon Bedrock as your LLM provider

From AI Settings, to use Anthropic on Amazon Bedrock, from the LLM Provider dropdown list, select Anthropic+Bedrock.

Using the environment configuration for authentication

To use the environment configuration for authentication, click Environment Configuration.

Structural uses the credentials set in the following environment settings:

  • TONIC_AWS_ACCESS_KEY_ID - An AWS access key that is associated with an IAM user or role.

  • TONIC_AWS_SECRET_ACCESS_KEY - The secret key that is associated with the access key.

  • TONIC_AWS_REGION - The AWS Region to send the authentication request to.

If these environment settings are not configured, then Structural uses the AWS SDK default resolving chain.

Using an Amazon Bedrock API key for authentication

From AI Settings, to use an Amazon Bedrock API key for authentication:

AI Settings page configured to use Anthropic on Amazon Bedrock using an API key

  1. Click API Key.

  2. In the Amazon Bedrock API Key field, provide the API key.

Selecting the model to use

Optionally, in the Model Name field, specify the model to use.

If you do not specify a model, then the default is:

  • For lightweight, quick tasks, us.anthropic.claude-haiku-4-5-20251001-v1:0.

  • For standard tasks, us.anthropic.claude-sonnet-4-5-20250929-v1:0.

  • For complex, high-quality tasks, us.anthropic.claude-opus-4-6-v1.

Using environment settings to configure the Anthropic on Amazon Bedrock connection

To use environment settings to connect to Anthropic on Amazon Bedrock:

  • TONIC_LLM_PROVIDER - Set to AnthropicBedrock.

  • TONIC_BEDROCK_API_KEY - To use an API key for authentication, set this to the Amazon Bedrock API key. To use the environment configuration for authentication, do not set this setting.

  • TONIC_LLM_MODEL - The model to use.

Using OpenAI

Supported deployments

Structural supports any OpenAI deployment that:

  • Supports API key authentication.

  • Supports the OpenAI Responses API.

  • Supports OpenAI proprietary models, such as GPT-4/5.x.

Required model deployments for hosted deployments

If you use a hosted OpenAI deployment such as Microsoft Foundry, Structural requires the following model deployments. The deployment name must match the model name.

  • gpt-4.1-mini

  • gpt-4.1

  • gpt-5.2

Using AI Settings to configure the OpenAI connection

From AI Settings, to use OpenAI as your LLM provider:

AI Settings page configured to use OpenAI as the LLM provider

  1. From the LLM Provider dropdown list, select OpenAI.

  2. In the OpenAI Endpoint field, enter the OpenAI endpoint URL to use for AI-enhanced features. Should include the endpoint base URL for the OpenAI API, without the version. The default is https://api.openai.com. Here is an example URL for Microsoft Foundry: https://my-foundry-instance.ai.azure.com/openai

  3. In the OpenAI API Key field, enter the API key for OpenAI.

  4. Optionally, in the Model Name field, specify the model to use. If you do not specify a model, then the default is:

    • For lightweight, quick tasks, gpt-4.1-mini.

    • For standard tasks, gpt-4.1.

    • For complex, high-quality tasks, gpt-5.2.

  5. To test the connection, click Test OpenAI Connection.

Using environment settings to configure the OpenAI connection

To use environment settings to connect to OpenAI:

  • TONIC_LLM_PROVIDER - Set to OpenAI.

  • TONIC_OPENAI_ENDPOINT - The OpenAI endpoint URL. Include the endpoint base URL for the OpenAI API, without the version.

  • TONIC_OPENAI_API_KEY - The API key for OpenAI.

  • TONIC_LLM_MODEL - The model to use.

Configuring AI-based Structural functionality

Disabling LLM-based sensitivity detection

By default, Structural sensitivity detection includes LLM-based sensitivity detection, which sends database information to an LLM.

To disable the LLM-based sensitivity detection, set the environment setting TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS to false.

You can set this from the Environment Settings tab on Structural Settings.

Excluding sample data from LLM-based sensitivity detection

By default, when LLM-based sensitivity detection is enabled, it sends both the database schema and sample source data values to the LLM.

To exclude the contextual source data, and only send the schema, set the environment setting TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS_SAMPLE_DATA to false.

You can set this from the Environment Settings tab on Structural Settings.

Excluding sample data from all LLM-based features

To configure Structural to never send sample data for any LLM-based feature, either:

  • On AI Settings view, under LLM Data Handling, uncheck Allow LLM access to sample data.

  • Set the environment setting TONIC_LLM_ENABLE_SAMPLE_DATA to false. You can configure this setting from the Environment Settings tab on Structural Settings.

Last updated

Was this helpful?