Inference

Settings → Inference is where your organization picks the model behind its AI judge filters. Nothing else on the gateway uses it yet, so today the page exists to answer one question: which endpoint reads your tool responses when a judge filter runs.

Resolution is per organization and fails closed: your active candidate first, then any provider your operator has bound for your organization, and nothing after that. With neither, a judge filter can't run, and its rule's on_error setting decides what happens: the default blocks the call.

Connections and candidatesCopy Anchor Link

The page separates the endpoint from the choice of model:

• An inference connection is a model provider's API endpoint plus the API key to reach it. It speaks one of two APIs: the OpenAI API or the Anthropic API. Register your own, or use one SecureAuth provides. It's unrelated to a user's connections to third-party services.
• A candidate pairs a connection with a model name, for one purpose. Several candidates can exist for the DLP judge; exactly one is active.

Splitting them means you register an API key once and try several models against it without re-entering the key.

Registering your own connectionCopy Anchor Link

1. Open Settings → Inference and click Add connection
2. Name it, then pick the Provider type and enter the Base URL:
   • OpenAI API (or compatible) covers OpenAI, Gemini's OpenAI-compatible endpoint, vLLM, and OpenRouter. Enter the URL with its version path, the part before /chat/completions, such as https://api.openai.com/v1.
   • Anthropic API (or compatible) covers Anthropic and Anthropic-compatible endpoints. Enter the origin without /v1, such as https://api.anthropic.com. The gateway adds /v1/messages itself.
3. Paste the API key. An Anthropic connection requires one. The gateway encrypts the key per organization. The API never returns it: the list shows only key stored.
4. Optionally restrict Allowed models to the ones you want bound; leave * to allow any
5. Create connection

You can't change a connection's provider type after you create it. If you change the base URL's origin (scheme, host, or port), enter the API key again in the same save. The gateway never sends a stored key to an origin you didn't enter it for.

Test connection on a saved connection sends the smallest possible completion. It exercises what the judge uses: the endpoint, the stored key, and a model. It passes only if that completion succeeds. A model list doesn't prove your account can use a model. Some providers, such as OpenRouter, advertise their models without an API key.

The completion runs against the first model you pin in Allowed models. With only * there, it runs against the first model the provider advertises. If that model is one your account can't use, the test fails and names it. Pin a model you have access to and run it again.

The base URL is checked against the gateway's egress policy when you save, so an endpoint the gateway may not reach is rejected then rather than failing when the judge first runs.

SecureAuth-provided connectionsCopy Anchor Link

Where SecureAuth provides a connection, it appears in the list badged SecureAuth and marked Provided by SecureAuth. It's read-only: you can add a candidate on it, but you can't edit, delete, or test it. You don't manage its API key. Its endpoint isn't listed either: you pick from the models it allows, not where they run. Your organization's entitlement decides whether SecureAuth provides one.

Activating a candidateCopy Anchor Link

On the DLP judge tab, Add candidate, name it, pick a connection, and type the model. The first candidate for a purpose is created active; later ones arrive inactive, and Set active promotes one. Activation is exclusive: it deactivates whichever candidate was active for that purpose.

The model must be in the connection's allowed list. A candidate that names a model the connection doesn't allow is rejected at save time.

A change here takes effect on an agent's next connection, not mid-session: the judge client is resolved when an agent connects, and activating a candidate or editing its connection drops what the gateway cached for your organization. Rotating a connection's API key works the same way, so a rotation doesn't leave the judge using the old one. That's the same timing filter changes have.

Testing before you commitCopy Anchor Link

Candidates aren't tested from this page. The judge tester lives with the feature that runs it: on Data Protection, open a rule's judge filter and use Test instruction. A Run against selector picks which candidate answers, so you can try one against a sample before making it live.