Skip to main content

​
What is the LiteLLM integration?

LiteLLM is an open-source proxy that puts one endpoint in front of every model provider. It ships a built-in connector, generic_guardrail_api, which calls an external service to judge each request. Pointed at Unbound, that connector checks every call twice — the prompt before it reaches the provider, the reply before it reaches the client — against the same Security policies and Tool policies that cover the rest of your organization. Prompts and responses appear under Logs.

​
Prerequisites

  • LiteLLM 1.91.0 or later.
  • Admin access to Unbound.

​
Get your API key

  1. Open Configure in Unbound.
  2. Under API access, copy the API key.
Set it in your proxy’s environment:

​
Setup

Add a top-level guardrails block to your LiteLLM config.yaml:
Restart the proxy.
guardrails belongs at the top level of config.yaml. Nested under litellm_settings it is read as the older v1 format and ignored, and the guardrail loads as nothing.

​
Checks fail open

unreachable_fallback: fail_open and fail_on_error: false let a call proceed when Unbound cannot judge it — a timeout, a network error, or a rejected key. A key the gateway rejects therefore produces no errors and no blocks. Confirm the integration with the steps below rather than reading an absence of errors as success.
Below LiteLLM 1.107.0, streaming_buffer_until_moderated has no effect. Tool-call fragments in a streamed reply reach the client before the reply check finishes, so a block arrives after the client already has them. Prompt checks and non-streamed replies are unaffected.

​
Verification

  1. Send a request through your proxy. It appears under Logs with its prompt and response.
  2. Send a prompt carrying something your Security policy detects, such as an API key. The detection is recorded against the policy that matched.
  3. From a coding agent pointed at the proxy, ask it to run a command your tool policies block, such as printenv. The call is refused and the agent reports an error. Tool policies match the tool calls in a model’s reply, so this step needs an agent rather than a plain chat request.
If nothing reaches Logs, confirm UNBOUND_API_KEY is set in the proxy’s environment and that api_base ends in /v1/app/analyze.

​
Attribution

Each call is attributed to the owner of the virtual key it was made with, read from that key’s user_email. Where callers share the master key there is no per-person identity to read, and every call is attributed to the owner of the Unbound API key. Organization-wide policies still apply. Per-user policies need a virtual key per person with user_email set.

Security Policies

Detect and redact secrets and personal data in prompts

Tool Policies

Audit, warn or block risky commands and MCP calls