Inference

Settings → Inference is where your organization picks the model behind its LLM judge filters. Nothing else on the gateway uses it yet, so today the page exists to answer one question: which endpoint reads your tool responses when a judge filter runs.

Resolution is per organization and fails closed: your active candidate first, then any provider your operator has bound for your organization, and nothing after that. With neither, a judge filter can't run, and its rule's on_error setting decides what happens: the default blocks the call.

Connections and candidates

The page separates the endpoint from the choice of model:

• An inference connection is an OpenAI-compatible endpoint plus the credential to reach it. Register your own, or use one SecureAuth provides. It's unrelated to a user's connections to third-party services.
• A candidate pairs a connection with a model name, for one purpose. Several candidates can exist for the DLP judge; exactly one is active.

Splitting them means you register a credential once and try several models against it without re-entering the key.

Registering your own connection

1. Open Settings → Inference and click Add connection
2. Name it, then enter the Base URL of an OpenAI-compatible endpoint (the part before /chat/completions)
3. Paste the API key. It's encrypted per organization and never returned by the API: the list shows only key stored.
4. Optionally restrict Allowed models to the ones you want bound; leave * to allow any
5. Create connection

Test connection on a saved row sends the smallest possible completion, so it exercises exactly what the judge will: the endpoint, the stored key, and a model. It passes only if that completion succeeds — asking the provider for its model list wouldn't prove anything, since providers such as OpenRouter publish theirs without a credential.

The completion runs against your pinned model when Allowed models names one, and otherwise against the first model the provider advertises. If that model is one your account can't use, the test fails and names it — pin a model you have access to and run it again.

The base URL is checked against the gateway's egress policy when you save, so an endpoint the gateway may not reach is rejected then rather than failing when the judge first runs.

SecureAuth-provided connections

Where SecureAuth offers a managed provider, it appears in the list badged SecureAuth and marked Provided by SecureAuth. It's read-only: you can bind a candidate to it, but you can't edit, delete, or test it, and its credential is never yours to manage. Its endpoint isn't listed either: you pick from the models it allows, not where they run. Whether it's offered at all depends on your organization's entitlement.

Activating a candidate

On the DLP judge tab, Add candidate, name it, pick a connection, and type the model. The first candidate for a purpose is created active; later ones arrive inactive, and Set active promotes one. Activation is exclusive: it deactivates whichever candidate was active for that purpose.

The model must be in the connection's allowed list. A candidate that names a model the connection doesn't allow is rejected at save time.

A change here takes effect on an agent's next connection, not mid-session: the judge client is resolved when an agent connects, and activating a candidate or editing its connection drops what the gateway cached for your organization. Rotating a connection's API key works the same way, so a rotation doesn't leave the judge using the old one. That's the same timing filter changes have.

Testing before you commit

Candidates aren't tested from this page. The judge tester lives with the feature that runs it: on Data Protection, open a rule's judge filter and use Test instruction. A Run against selector picks which candidate answers, so you can try one against a sample before making it live.