> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getunbound.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Apply Unbound policies to the traffic passing through your LiteLLM proxy

## What is the LiteLLM integration?

[LiteLLM](https://docs.litellm.ai) is an open-source proxy that puts one endpoint in front of every model provider. It ships a built-in connector, `generic_guardrail_api`, which calls an external service to judge each request.

Pointed at Unbound, that connector checks every call twice — the prompt before it reaches the provider, the reply before it reaches the client — against the same [Security policies](/policies/security-policies) and [Tool policies](/policies/tool-policies) that cover the rest of your organization. Prompts and responses appear under [Logs](/dashboard/logs).

## Prerequisites

* **LiteLLM** 1.91.0 or later.
* **Admin access** to Unbound.

## Get your API key

1. Open [Configure](https://gateway.getunbound.ai/configure) in Unbound.
2. Under **API access**, copy the **API key**.

Set it in your proxy's environment:

```bash theme={null}
export UNBOUND_API_KEY=<your-api-key>
```

## Setup

Add a top-level `guardrails` block to your LiteLLM `config.yaml`:

```yaml theme={null}
guardrails:
  - guardrail_name: unbound
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      api_base: https://api.getunbound.ai/v1/app/analyze
      api_key: os.environ/UNBOUND_API_KEY
      default_on: true
      unreachable_fallback: fail_open
      fail_on_error: false
      streaming_buffer_until_moderated: true
```

Restart the proxy.

| Setting | What it does |
| - | - |
| `mode` | `pre_call` checks the prompt, `post_call` checks the reply. Keep both — a risky tool call appears in the model's reply, not the user's prompt |
| `api_base` | LiteLLM appends `/beta/litellm_basic_guardrail_api` to this value, so keep the `/v1/app/analyze` path. On a dedicated deployment, replace the host and keep the path |
| `default_on` | Applies the guardrail to every model in `model_list`. Without it, each call has to request the guardrail by name |
| `unreachable_fallback`, `fail_on_error` | Let calls through when Unbound cannot answer. See [Checks fail open](#checks-fail-open) |
| `streaming_buffer_until_moderated` | Holds a streamed reply until the check finishes. Requires LiteLLM 1.107.0 or later |

> [!NOTE]
>
> `guardrails` belongs at the top level of `config.yaml`. Nested under `litellm_settings` it is read as the older v1 format and ignored, and the guardrail loads as nothing.

## Checks fail open

`unreachable_fallback: fail_open` and `fail_on_error: false` let a call proceed when Unbound cannot judge it — a timeout, a network error, or a rejected key.

A key the gateway rejects therefore produces no errors and no blocks. Confirm the integration with the steps below rather than reading an absence of errors as success.

> [!WARNING]
>
> Below LiteLLM 1.107.0, `streaming_buffer_until_moderated` has no effect. Tool-call fragments in a streamed reply reach the client before the reply check finishes, so a block arrives after the client already has them. Prompt checks and non-streamed replies are unaffected.

## Verification

1. Send a request through your proxy. It appears under [Logs](/dashboard/logs) with its prompt and response.
2. Send a prompt carrying something your Security policy detects, such as an API key. The detection is recorded against the policy that matched.
3. From a coding agent pointed at the proxy, ask it to run a command your tool policies block, such as `printenv`. The call is refused and the agent reports an error. Tool policies match the tool calls in a model's reply, so this step needs an agent rather than a plain chat request.

If nothing reaches Logs, confirm `UNBOUND_API_KEY` is set in the proxy's environment and that `api_base` ends in `/v1/app/analyze`.

## Attribution

Each call is attributed to the owner of the [virtual key](https://docs.litellm.ai/docs/proxy/virtual_keys) it was made with, read from that key's `user_email`.

Where callers share the master key there is no per-person identity to read, and every call is attributed to the owner of the Unbound API key. Organization-wide policies still apply. Per-user policies need a virtual key per person with `user_email` set.

> **[Security Policies](/policies/security-policies)**
>
> Detect and redact secrets and personal data in prompts

> **[Tool Policies](/policies/tool-policies)**
>
> Audit, warn or block risky commands and MCP calls


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.