# Grepture

> Grepture is an AI gateway for securing and observing LLM API traffic. It works with OpenAI, Anthropic, Google, Azure, and any OpenAI-compatible provider — giving you PII redaction, cost tracking, observability, prompt management, evals, and request control from a single SDK.

The SDK supports two operating modes:

- **Proxy mode** (default) — requests route through the Grepture proxy, where detection rules run before traffic reaches the provider. Use this for PII redaction, request blocking, mask-and-restore, and prompt management.
- **Trace mode** — requests go directly to the LLM provider with zero proxy overhead. The SDK captures metadata (model, tokens, latency, cost) asynchronously and sends it to the dashboard. Use this for observability and cost tracking on latency-sensitive workloads.

Both modes share the same dashboard: traffic log, cost tracking, conversation tracing, evals, and reports.

## Core Concepts

- **Proxy mode**: A reverse proxy that intercepts AI API calls, applies detection rules, and forwards cleaned requests
- **Trace mode**: Direct-to-provider requests with async metadata capture — same dashboard visibility, zero latency overhead
- **Rules**: Configurable detection and action policies (detect PII, block prompt injections, redact fields, etc.)
- **Mask & Restore**: Tokenize sensitive values on the way out, restore them on the way back
- **Cost tracking**: Per-request token counts and cost attribution across all providers
- **Conversation tracing**: Group related requests by trace ID to see multi-step workflows end to end
- **Prompt management**: Version, manage, and deploy prompts server-side via API — no redeployments needed
- **Evals**: LLM-as-a-judge evaluation on your traffic logs with pre-built templates and custom judge prompts
- **Prompt A/B testing**: Split traffic between prompt versions, measure eval scores per version, and activate the winner — zero code changes needed
- **OpenTelemetry integration**: Export traffic as OTLP traces to Datadog, Grafana, Honeycomb, or any observability platform
- **Reports**: PDF export of cost, traffic, and GDPR compliance reports
- **CLI**: Local AI gateway via `npx @grepture/proxy` for development and self-hosted deployments
- **Zero-Data Mode**: When enabled, no request or response bodies are stored in traffic logs
- **SDK**: A TypeScript drop-in wrapper (`@grepture/sdk`) that works in both proxy and trace mode

## How It Works

### Proxy mode (default)

1. Your app sends an LLM API request through the Grepture proxy (via the SDK or direct HTTP)
2. The proxy scans the request against your configured rules
3. Matching rules trigger actions: redact PII, tokenize fields, block the request, or log-only
4. The cleaned request is forwarded to the LLM provider
5. The response is scanned with output rules, detokenized (restoring original values), and returned

### Trace mode

1. Your app sends the LLM API request directly to the provider — no proxy in the path
2. The SDK captures metadata (model, tokens, latency, status) after the response
3. Trace data is batched and sent asynchronously to Grepture (fire-and-forget, never blocks the caller)
4. The dashboard shows the same traffic log, cost data, and conversation traces

## Detection Capabilities

- **PII Detection**: Emails, phone numbers, SSNs, credit cards, IP addresses, names, addresses, dates of birth
- **AI-Powered PII**: NER-based detection of persons, locations, organizations (runs on Grepture infrastructure — no data leaves to external AI services)
- **PII Redaction for Embeddings**: A dedicated `POST /v1/embeddings` endpoint (OpenAI-compatible) redacts PII in the input string before forwarding to the provider, so the vector stored in Pinecone/pgvector/Weaviate is derived from clean text. Placeholder strategy preserves k-NN clustering. Header `x-grepture-on-pii: block` returns 422 instead of redacting. Free tier gets regex; Pro+ adds NER. Grepture does not store the input text or the response vectors.
- **Prompt Injection Detection**: Identifies adversarial prompts attempting to override system instructions
- **Toxicity Scanning**: Detects toxic, obscene, threatening, or hateful content
- **Data Loss Prevention**: Flags source code, credentials, internal documents, financial data
- **Compliance Flagging**: Healthcare, financial, legal, insurance domain awareness

## Actions

- `redact_pii` — Replace PII with placeholders, hashes, or masks
- `tokenize` — Replace field values with tokens stored in a vault; restore on response
- `redact_field` — Replace entire JSON fields with a custom string
- `find_replace` — Literal or regex find-and-replace on request/response body
- `block_request` — Reject the request with a custom status code and message
- `log_only` — Tag the request for monitoring without mutating it

## Integration

Install the SDK:

```
npm install @grepture/sdk
```

### Proxy mode (default) — with rules and redaction:

```typescript
import { Grepture } from "@grepture/sdk";
import OpenAI from "openai";

const grepture = new Grepture({
  apiKey: "gpt_your_key",
  proxyUrl: "https://proxy.grepture.com",
});

const client = new OpenAI(
  grepture.clientOptions({
    apiKey: "sk-your-openai-key",
    baseURL: "https://api.openai.com/v1",
  })
);
```

### Trace mode — observability without the proxy hop:

```typescript
const grepture = new Grepture({
  apiKey: "gpt_your_key",
  proxyUrl: "https://proxy.grepture.com",
  mode: "trace",
});

// Same clientOptions() API — requests go directly to the provider
const client = new OpenAI(
  grepture.clientOptions({
    apiKey: "sk-your-openai-key",
    baseURL: "https://api.openai.com/v1",
  })
);

// In serverless environments, flush pending traces before exit
await grepture.flush();
```

### Drop-in fetch wrapper (works in both modes):

```typescript
const response = await grepture.fetch(
  "https://api.openai.com/v1/chat/completions",
  {
    method: "POST",
    headers: { Authorization: "Bearer sk-..." },
    body: JSON.stringify({ model: "gpt-4o", messages: [...] }),
  }
);
```

Works with any OpenAI-compatible SDK (LangChain, Vercel AI SDK, etc.) via the `clientOptions()` method.

### IDE & tool integrations

- **Cursor**: Route Cursor AI through Grepture for observability and PII protection
- **Claude Code**: Route Claude Code through Grepture for cost tracking and data protection
- **Chat UIs**: Route TypingMind, LibreChat, LobeChat, Open WebUI, NextChat, ChatBox, BoltAI, and other BYOK chat clients through Grepture by setting a custom OpenAI/Anthropic base URL — no plugins or code changes

## Pricing

- **Free**: 1,000 requests/mo, 50+ detection patterns, 1 team member, 3 rules
- **Pro** (€49/mo): 100,000 requests/mo, 80+ patterns, AI PII detection, mask & restore, evals, OpenTelemetry integration, cost & traffic reports, 5 team members
- **Business** (€299/mo): 1,000,000 requests/mo, prompt injection detection, toxicity scanning, DLP, compliance flagging, GDPR compliance report, unlimited team members, zero-data mode, full activity log & audit trail
- **Custom**: Contact sales for custom limits, signed DPA, 99.99% SLA

## Open Source

The proxy core (`@grepture/proxy`) and SDK (`@grepture/sdk`) are open source under AGPL-3.0. Run locally without cloud dependencies:

```
npx @grepture/proxy
```

## Links

- [Homepage](https://grepture.com)
- [Dashboard](https://app.grepture.com)
- [Pricing](https://grepture.com/pricing)
- [How It Works](https://grepture.com/how-it-works)
- [Use Cases](https://grepture.com/use-cases)

## Product Pages

- [Observability](https://grepture.com/product/observability): See every AI request your app makes.
- [PII Redaction](https://grepture.com/product/pii-redaction): Detect and redact sensitive data automatically.
- [Cost Tracking](https://grepture.com/product/cost-tracking): Per-request token usage and spend analytics.
- [Prompt Management](https://grepture.com/product/prompt-management): Version and deploy prompts via API.
- [Routing](https://grepture.com/product/routing): Provider keys with automatic fallback across providers.
- [Evals](https://grepture.com/product/evals): LLM-as-a-Judge scoring on your traffic.
- [Claude Code](https://grepture.com/product/claude-code): Route Claude Code through Grepture.
- [Cursor](https://grepture.com/product/cursor): Route Cursor through Grepture.
- [Chat UIs](https://grepture.com/product/chat-ui): Route any BYOK ChatGPT client (TypingMind, LibreChat, LobeChat, Open WebUI, NextChat, ChatBox) through Grepture via a custom base URL.
- [Chrome Extension](https://grepture.com/product/shadow-ai): Protect browser-based AI usage.

## Docs

- [Quickstart](https://grepture.com/docs/quickstart): Install the SDK, get your API key, and protect your first request in under 5 minutes.
- [SDK Reference](https://grepture.com/docs/sdk): Grepture class, clientOptions(), grepture.fetch(), error classes, and TypeScript types.
- [Configuration](https://grepture.com/docs/configuration): Detection rules, custom patterns, zero-data mode, and action types.
- [Dashboard](https://grepture.com/docs/dashboard): Traffic monitoring, rule management, team settings, and billing.
- [Evals](https://grepture.com/docs/evals): LLM-as-a-judge evaluators, templates, sampling, and score analytics.
- [Embeddings](https://grepture.com/docs/embeddings): PII-redacting `POST /v1/embeddings` endpoint — request shape, headers, redaction strategies, block mode, SDK usage, errors.
- [Prompt Management](https://grepture.com/docs/prompts): Version, manage, and deploy prompts server-side via API.
- [Claude Code](https://grepture.com/docs/claude-code): Route Claude Code through Grepture for observability and protection.
- [Cursor](https://grepture.com/docs/cursor): Route Cursor through Grepture for observability and protection.

## Guides

- [How to Track and Control AI API Costs Across Providers](https://grepture.com/guides/track-ai-api-costs)
- [How to Monitor and Log All LLM API Calls in One Place](https://grepture.com/guides/monitor-log-llm-api-calls)
- [How to Make AI API Calls GDPR-Compliant](https://grepture.com/guides/gdpr-compliant-ai-api-calls)
- [How to Set Up AI Usage Alerts and Compliance Reports](https://grepture.com/guides/ai-compliance-reporting-audit-trail)
- [Zero-Retention AI Processing: How to Use LLMs Without Storing Data](https://grepture.com/guides/zero-retention-ai-processing)
- [How to Separate Prompts from Code with Server-Side Prompt Serving](https://grepture.com/guides/separate-prompts-from-code)
- [How to Version and Manage LLM Prompts Server-Side](https://grepture.com/guides/version-manage-llm-prompts)
- [How to Redact PII from OpenAI API Calls](https://grepture.com/guides/redact-pii-openai-api)
- [How to Redact PII from Anthropic Claude API Calls](https://grepture.com/guides/redact-pii-anthropic-claude-api)
- [How to Redact PII from Google Gemini API Calls](https://grepture.com/guides/redact-pii-google-gemini-api)
- [How to Redact PII from Azure OpenAI API Calls](https://grepture.com/guides/redact-pii-azure-openai-api)
- [How to Redact PII from AWS Bedrock API Calls](https://grepture.com/guides/redact-pii-aws-bedrock-api)
- [How to Redact PII from LangChain Pipelines](https://grepture.com/guides/redact-pii-langchain)
- [How to Redact PII from Vercel AI SDK Calls](https://grepture.com/guides/redact-pii-vercel-ai-sdk)
- [How to Redact PII from Any API Call](https://grepture.com/guides/redact-pii-any-api)

## Comparisons

- [Best PII Redaction APIs for LLMs (2026)](https://grepture.com/comparisons/best-pii-redaction-apis-for-llms)
- [Grepture vs. Microsoft Presidio](https://grepture.com/comparisons/grepture-vs-presidio)
- [Grepture vs. LLM Guard](https://grepture.com/comparisons/grepture-vs-llm-guard)

## Blog

- [Introducing Grepture — Content-Aware API Security Proxy](https://grepture.com/blog/introducing-grepture)
- [Introducing the Grepture CLI — A Local AI Gateway](https://grepture.com/blog/introducing-grepture-cli)
- [From PII Redaction to AI Gateway — Why We're Expanding Grepture](https://grepture.com/blog/from-pii-redaction-to-ai-gateway)
- [Your Vector Store Is a Permanent PII Leak](https://grepture.com/blog/pii-leak-vector-store): Why embedding raw user input into a vector store is a permanent, unscrubable data leak, and how `/v1/embeddings` with placeholder redaction keeps RAG retrieval working while preventing it.
- [Trace Mode — Full Observability Without the Proxy Hop](https://grepture.com/blog/trace-mode-zero-latency-observability)
- [LLM Evals on Real Traffic — Not Just Test Suites](https://grepture.com/blog/llm-evals-on-real-traffic)
- [Prompt Management: Version Control for Your LLM Prompts](https://grepture.com/blog/prompt-management-version-control-llm)
- [What is PromptOps? A Complete Guide for Engineering Teams](https://grepture.com/blog/what-is-promptops)
- [Route Cursor Through Grepture](https://grepture.com/blog/cursor-gateway)
- [Route Claude Code Through Grepture](https://grepture.com/blog/claude-code-gateway)
- [Your LLM Observability Tool Is Logging PII](https://grepture.com/blog/llm-observability-logging-pii)
- [PII Detection Best Practices for AI Pipelines](https://grepture.com/blog/pii-detection-best-practices)
- [EU AI Act Compliance Guide for AI Engineers](https://grepture.com/blog/eu-ai-act-compliance-guide-2026)
- [Prompt Injection Prevention for Production LLM Apps](https://grepture.com/blog/prompt-injection-prevention-guide)
- [Indirect Prompt Injection: The Attack That Hides in Your Data](https://grepture.com/blog/indirect-prompt-injection-attacks)
- [Mask and Restore: Reversible Redaction That Keeps LLMs Useful](https://grepture.com/blog/mask-and-restore-reversible-redaction-llm)
- [LLM Security Tools Compared: Gateways, DLP, Guardrails, and Proxies](https://grepture.com/blog/llm-security-tools-comparison)
- [How to Prevent Sensitive Data Leaks in LLM API Calls](https://grepture.com/blog/prevent-data-leaks-llm-api-calls)
- [How to Secure Your RAG Pipeline](https://grepture.com/blog/rag-pipeline-security)
- [Why Your AI Agents Are Leaking Data (And How to Stop Them)](https://grepture.com/blog/ai-agent-data-leaks)