Score every AI response automatically.

LLM-as-a-judge evaluation on the traffic you already log. Pick a template or write your own criteria, with no extra instrumentation.

app.grepture.com/evals
EvaluatorAvgStatus
Helpfulness
0.84on
Hallucination
0.91on
Tone and brand
0.62on
Safety
0.98off
Recent scores
  • 0.90Helpfulness2s ago
  • 1.00Hallucination2s ago
  • 0.70Helpfulness8s ago
  • 0.40Tone and brand12s ago
Judge prompt

Rate the response from 0 to 1 for helpfulness. Does it answer {{request}} directly? Is {{response}} accurate and concise?

What it does

Grepture Evals scores your AI outputs with a judge model. Because Grepture already logs every request and response, evaluations run on existing traffic. Define what good looks like and every sampled response gets a 0 to 1 score with reasoning.

How it works

Create an evaluator from a template (relevance, helpfulness, toxicity, conciseness, instruction-following, hallucination) or write your own judge prompt. Set filters and a sampling rate. A background worker scores your traffic and stores the reasoning. View results on each log, per evaluator or as trends.

Quality you can measure, on real traffic.

  • No separate pipeline

    Evaluations run on traffic Grepture already logs, with no extra instrumentation.

  • Six templates or your own

    Relevance, helpfulness, toxicity, conciseness, instruction-following, hallucination, or a custom judge prompt.

  • Scores with reasons

    Every sampled response gets a 0 to 1 score and the judge's reasoning.

  • No added latency

    A background worker does the scoring, so requests are never slowed down.

What's included

  • Six templates: relevance, helpfulness, toxicity, conciseness, instruction-following, hallucination
  • Custom evaluators with your own judge prompts
  • Filter by model, provider, prompt or status code
  • Sampling rate from 1 to 100%
  • Score on every traffic log entry
  • Trends, averages and per-evaluator breakdowns
  • Runs in the background, no added proxy latency
  • Compare prompt versions by score in A/B tests

Send your first clean request in five minutes.

Free for 1,000 requests a month. No credit card, no sales call.