Enabling model-based custom entity types

Model-based custom entity types require a connection to an LLM.

You can use the configured default provider and model, or select a provider and model to use specifically for model-based entity types.

You must also provide the URL for the Textual API.

Setting the URL for the Textual API

To support model-based custom entity types, the ML container must be able to connect to the Textual API.

You set the following environment variable to the public base URL of the Textual API:

SOLAR_API_URL

For example, https://textual.example.com.

The URL must be accessible from the ML container.

Selecting a custom provider and model

If you do not specify a provider and model for model-based custom entity types, Textual uses the default provider and model.

To identify a specific provider and model to use for model-based custom entity types, set the following environment variables:

• MODEL_BASED_ENTITIES_LLM_PROVIDER - The LLM provider to use for model-based custom entity types. For information about setting up connections to LLM providers, go to Configuring connections to LLM providers.
• MODEL_BASED_ENTITIES_LLM_MODEL_NAME - The model to use for model-based custom entity types.

Connection timeout and retry settings

You can also configure information the following optional settings:

• MODEL_BASED_ENTITIES_LLM_TIMEOUT_IN_SECONDS - The request timeout in seconds. The default value is 300.
• MODEL_BASED_ENTITIES_LLM_RETRY_COUNT - The number of times to retry when a request fails. The default value is 5.
• MODEL_BASED_ENTITIES_LLM_SLOW_REQUEST_THRESHOLD_SECONDS - If the request exceeds this number of seconds, then log a warning. The default value is 120.