How to Redact PII from LiteLLM-Routed AI Calls

The problem: one router, many models, many leak paths

LiteLLM is the closest thing the LLM ecosystem has to a universal client. One SDK, one API shape, 100+ model providers. Teams use it because they don't want their code coupled to a specific provider, and they want to swap models based on cost, latency, or capability.

That convenience creates a problem: every model you route to is another exfiltration channel for whatever your users typed. A regex you wrote for OpenAI doesn't help when the same prompt lands at Anthropic. Per-provider redaction logic doesn't scale.

You need one redaction policy applied in the request path, before LiteLLM decides where to send it.

The setup

Grepture is a proxy. LiteLLM is also a proxy/router. They compose naturally — point LiteLLM's per-model api_base at Grepture, and Grepture redacts before forwarding upstream.

Option 1: LiteLLM SDK with Grepture as base URL

from litellm import completion

response = completion(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": user_input}],
    api_base="https://proxy.grepture.com/v1",
    extra_headers={
        "X-Grepture-Auth": f"Bearer {GREPTURE_API_KEY}",
        "X-Grepture-Auth-Forward": f"Bearer {OPENAI_API_KEY}",
    },
)

The X-Grepture-Auth-Forward header carries the upstream provider key. Grepture redacts the request body, forwards to OpenAI, restores reversible tokens in the response.

Option 2: LiteLLM Proxy server with Grepture upstream

If you run LiteLLM as a self-hosted router for an aggregator product, point each model's api_base in your config.yaml at Grepture:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_base: https://proxy.grepture.com/v1
      api_key: os.environ/OPENAI_API_KEY
      extra_headers:
        X-Grepture-Auth: os.environ/GREPTURE_API_KEY
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_base: https://proxy.grepture.com/v1
      api_key: os.environ/ANTHROPIC_API_KEY
      extra_headers:
        X-Grepture-Auth: os.environ/GREPTURE_API_KEY

Every provider in your LiteLLM router now flows through the same Grepture redaction rules. One policy, every model.

What gets caught, regardless of upstream

Because Grepture sits in the request path before LiteLLM decides on a provider, the same detection runs for every model:

PII (reversible): email, phone, SSN, credit card, IP address, physical address, person names, dates of birth
Secrets (permanent replacement): API keys, tokens, passwords, webhooks across 25+ credential families — OpenAI, Anthropic, AWS, GitHub, Stripe, Slack
Streaming preserved: SSE chunks are detokenized in real time, no buffering

The aggregator use case

If your product is a multi-model aggregator — users bring intent, you route to the best model — every provider in the chain is another exfiltration channel for whatever the user typed. One redaction policy enforced before the routing decision is the only thing that scales:

Migration from raw LiteLLM

If you're already running LiteLLM without redaction, the move is a base URL change per model. No application code changes, no per-prompt scrubbing, no provider-specific regex.

Sign up at app.grepture.com.
Update api_base on each model entry (or in your SDK calls) to the Grepture proxy.
Pick the PII and secret categories you want redacted from the dashboard.
Watch the Traffic Log to verify detections before enforcing.

Next steps

View pricing — free for up to 1,000 requests/month
Docs — SDK reference and dashboard guide