> For the complete documentation index, see [llms.txt](https://docs.tonic.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.tonic.ai/app/admin/structural-ai-use/self-hosted-llm-configuration.md). # Self-hosted LLM configuration Self-hosted customers can choose whether to enable AI features on their instance. A self-hosted instance can either: * Use an LLM that Tonic.ai hosts. This is the default option. * Configure a connection to a supported LLM provider. * Not use an LLM at all. This disables all Structural AI features. You configure the connection either: * From the **AI Settings** tab on **Structural Settings**. * Using [environment settings](/app/admin/environment-variables-setting.md). The settings can be added to the **Environment Settings** list on **Structural Settings**. If you enable AI features, then Structural provides additional options to [configure data use by the LLM](#configuring-ai-based-structural-functionality). ## Supported providers Tonic Structural supports the following LLM providers: * Anthropic * Anthropic on Amazon Bedrock * OpenAI. Note that we are continuing to test OpenAI for the Structural Agent. ## Using the hosted LLM Structural provides a hosted LLM that you can use to support the Structural AI features. The hosted LLM uses Anthropic on Amazon Bedrock. For new self-hosted instances, this is the default option. The LLM is hosted in multiple locations. Structural routes the request through a SaaS proxy to the appropriate location for the customer. On the **AI settings** view, to use the hosted LLM: 1. From the **LLM Provider** dropdown list, select **Tonic.ai-hosted LLM**. 2. To test the connection to the hosted LLM, click **Test Tonic.ai-hosted LLM Connection**. To use [environment settings](/app/admin/environment-variables-setting.md) to select and configure the hosted LLM: * Set `TONIC_LLM_PROVIDER` to `TonicHosted`. * Set `TONIC_HOSTED_REGION` to either `US` or `EU`. If you use the hosted LLM, make sure to allowlist the SaaS proxy, either: * **** or 98.84.248.119 (US location) * **** or 3.120.214.225 (EU location) ## Disabling AI features on Structural To not connect to an LLM, which disables Structural AI features, from the **LLM Provider** dropdown list, select **Do not use an LLM**.

AI Settings page with configuration to not use an LLM

To use an environment setting to disable Structural AI features, set `TONIC_LLM_PROVIDER` to `LlmOptOut`. ## Using Anthropic ### Required models If you use a hosted Anthropic deployment, Structural requires the following models: * `claude-opus-4-6` * `claude-sonnet-4-5` * `claude-haiku-4-5` ### Using AI settings to configure the Anthropic connection From **AI Settings**, to configure Anthropic as your LLM:

AI Settings page configured to use Anthropic as the LLM provider

1. From the **LLM Provider** dropdown list, select Anthropic. 2. In the **Anthropic Endpoint** field, enter the Anthropic endpoint URL for generative AI services.\ \ Should include the endpoint base URL for the Anthropic API, without the version.\ \ The default is `https://api.anthropic.com`. 3. In the **Anthropic API Key** field, enter the API key for Anthropic.\ \ In production environments, make sure that this is kept secure and encrypted. 4. Optionally, in the **Model Name** field, specify the model to use. \ \ If you do not specify a model, then the default is: * For lightweight, quick tasks, `claude-haiku-4-5.` * For standard tasks, `claude-sonnet-4-5`. * For complex, high-quality tasks, `claude-opus-4-6`. 1. To test the connection, click **Test Anthropic Connection**. ### Using environment settings to configure the Anthropic connection To use [environment settings](/app/admin/environment-variables-setting.md) to connect to Anthropic: * `TONIC_LLM_PROVIDER` - Set to `Anthropic`. * `TONIC_ANTHROPIC_ENDPOINT` - The Anthropic endpoint URL. Include the endpoint base URL for the Anthropic API, without the version. * `TONIC_ANTHROPIC_API_KEY` - The Anthropic API key. * `TONIC_LLM_MODEL` - The model to use. ## Using Anthropic on Amazon Bedrock When you use Anthropic on Amazon Bedrock, you can either: * Use the environment configuration for AWS authentication. * Provide an Amazon Bedrock API key.

AI Settings page configured to use Anthropic on Amazon Bedrock using environment credentials

### Required IAM permissions for Amazon Bedrock To call Amazon Bedrock, Structural uses [cross-region inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/geographic-cross-region-inference.html). The IAM principal that authenticates the request must be granted the following permissions. #### **Actions** * `bedrock:InvokeModel` * `bedrock:InvokeModelWithResponseStream` #### Resources The principal must be able to invoke the following cross-region inference profiles: * `us.anthropic.claude-haiku-4-5-20251001-v1:0` * `us.anthropic.claude-sonnet-4-5-20250929-v1:0` * `us.anthropic.claude-opus-4-6-v1` #### AWS Regions Cross-region inference also requires invoke permission on the underlying foundation model in each AWS Region that the profile spans. The required profiles currently span the following AWS Regions: * `us-east-1` * `us-east-2` * `us-west-2` For the recommended IAM policy structure, go to the [AWS documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/geographic-cross-region-inference.html). ### Selecting Anthropic on Amazon Bedrock as your LLM provider From **AI Settings**, to use Anthropic on Amazon Bedrock, from the **LLM Provider** dropdown list, select **Anthropic+Bedrock**. ### Using the environment configuration for authentication To use the environment configuration for authentication, click **Environment Configuration**. Structural uses the credentials set in the following [environment settings](/app/admin/environment-variables-setting.md): * `TONIC_AWS_ACCESS_KEY_ID` - An AWS access key that is associated with an IAM user or role. * `TONIC_AWS_SECRET_ACCESS_KEY` - The secret key that is associated with the access key. * `TONIC_AWS_REGION` - The AWS Region to send the authentication request to. If these environment settings are not configured, then Structural uses the [AWS SDK default resolving chain](https://docs.aws.amazon.com/sdk-for-net/v4/developer-guide/creds-assign.html). ### Using an Amazon Bedrock API key for authentication From **AI Settings**, to use an Amazon Bedrock API key for authentication:

AI Settings page configured to use Anthropic on Amazon Bedrock using an API key

1. Click **API Key**. 2. In the **Amazon Bedrock API Key** field, provide the API key. ### Selecting the model to use Optionally, in the **Model Name** field, specify the model to use. If you do not specify a model, then the default is: * For lightweight, quick tasks, `us.anthropic.claude-haiku-4-5-20251001-v1:0`. * For standard tasks, `us.anthropic.claude-sonnet-4-5-20250929-v1:0`. * For complex, high-quality tasks, `us.anthropic.claude-opus-4-6-v1`. ### Using environment settings to configure the Anthropic on Amazon Bedrock connection To use [environment settings](/app/admin/environment-variables-setting.md) to connect to Anthropic on Amazon Bedrock: * `TONIC_LLM_PROVIDER` - Set to `AnthropicBedrock`. * `TONIC_BEDROCK_API_KEY` - To use an API key for authentication, set this to the Amazon Bedrock API key. To use the environment configuration for authentication, do not set this setting. * `TONIC_LLM_MODEL` - The model to use. ## Using OpenAI ### Supported deployments Structural supports any OpenAI deployment that: * Supports API key authentication. * Supports the OpenAI Responses API. * Supports OpenAI proprietary models, such as GPT-4/5.x. ### Required model deployments for hosted deployments If you use a hosted OpenAI deployment such as Microsoft Foundry, Structural requires the following model deployments. The deployment name must match the model name. * `gpt-4.1-mini` * `gpt-4.1` * `gpt-5.2` ### Using AI Settings to configure the OpenAI connection From **AI Settings**, to use OpenAI as your LLM provider:

AI Settings page configured to use OpenAI as the LLM provider

1. From the **LLM Provider** dropdown list, select **OpenAI**. 2. In the **OpenAI Endpoint** field, enter the OpenAI endpoint URL to use for AI-enhanced features.\ \ Should include the endpoint base URL for the OpenAI API, without the version.\ \ The default is `https://api.openai.com`.\ \ Here is an example URL for Microsoft Foundry: `https://my-foundry-instance.ai.azure.com/openai` 3. In the **OpenAI API Key** field, enter the API key for OpenAI. 4. Optionally, in the **Model Name** field, specify the model to use. \ \ If you do not specify a model, then the default is: * For lightweight, quick tasks, `gpt-4.1-mini`. * For standard tasks, `gpt-4.1`. * For complex, high-quality tasks, `gpt-5.2`. 5. To test the connection, click **Test OpenAI Connection**. ### Using environment settings to configure the OpenAI connection To use [environment settings](/app/admin/environment-variables-setting.md) to connect to OpenAI: * `TONIC_LLM_PROVIDER` - Set to `OpenAI`. * `TONIC_OPENAI_ENDPOINT` - The OpenAI endpoint URL. Include the endpoint base URL for the OpenAI API, without the version. * `TONIC_OPENAI_API_KEY` - The API key for OpenAI. * `TONIC_LLM_MODEL` - The model to use. ## Configuring AI-based Structural functionality ### Disabling LLM-based sensitivity detection By default, Structural sensitivity detection includes [LLM-based sensitivity detection](/app/generation/identify-sensitive-data/running-the-structural-sensitivity-scan.md#llm-based-sensitivity-detection-medium-confidence), which sends database information to an LLM. To disable the LLM-based sensitivity detection, set the [environment setting](/app/admin/environment-variables-setting.md) `TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS` to `false`. You can set this from the **Environment Settings** tab on **Structural Settings**. LLM-based sensitivity detection also processes LLM-based sensitivity rules. If you disable LLM-based sensitivity detection, then even if you enable LLM-based sensitivity rules, the sensitivity scan does not process any LLM-based sensitivity rules. ### Enabling LLM-based sensitivity rules On a self-hosted instance, to allow LLM-based sensitivity rules to be created and processed: * **LLM-based sensitivity rules must be enabled -** The environment setting `TONIC_LLM_ENABLE_CUSTOM_SENSITIVITY_RULES` must be set to `true`. This allows you to create LLM-based sensitivity rules. * **LLM-based sensitivity detection must be enabled** **-** The environment setting `TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS` must be set to `true`. This allows LLM-based sensitivity rules to be processed. If this is `false`, then those rules are ignored. ### Excluding sample data from LLM-based sensitivity detection By default, when LLM-based sensitivity detection is enabled, it sends both the database schema and sample source data values to the LLM. To exclude the contextual source data, and only send the schema, set the [environment setting](/app/admin/environment-variables-setting.md) `TONIC_LLM_ENABLE_ENHANCED_RECOMMENDATIONS_SAMPLE_DATA` to false. You can set this from the **Environment Settings** tab on **Structural Settings**. ### Excluding sample data from all LLM-based features To configure Structural to never send sample data for any LLM-based feature, either: * On **AI Settings** view, under **LLM Data Handling**, uncheck **Allow LLM access to sample data**. * Set the [environment setting](/app/admin/environment-variables-setting.md) `TONIC_LLM_ENABLE_SAMPLE_DATA` to `false`. You can configure this setting from the **Environment Settings** tab on **Structural Settings**. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.tonic.ai/app/admin/structural-ai-use/self-hosted-llm-configuration.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.