How do I route a LangGraph graph on a verify result?

Add a verify node that calls a separate judge such as POST https://screwtape.ai/v1/judge, then a conditional edge: pass goes on, retry goes back to the agent with the reasons, and fail goes to a person.

Answers

Why shouldn't a LangGraph verify step use the same model that wrote the answer?

This page was created with the help of AI by the Screwtape.ai team.

Short answer: a verify step that asks the same model, with the same context, whether its own answer is right mostly measures the model's confidence, not correctness. It tends to repeat the same mistake and approve it. Make the verify step separate: deterministic checks for anything that can be computed (required fields, totals, matches against known values), and a judge that runs outside the agent for the rest. Route the graph on the verdict.

Why self-verification is weak

Shared blind spots. The same weights and the same prompt context produce the same misreading. If the model misread a total, it will usually re-read it the same way.
Agreement is not evidence. Two calls agreeing, or several agents agreeing, can be false consensus: they share a source of error. See Stop false consensus in multi-agent swarms.
Arithmetic and field checks don't need a model. "Line items plus tax equal the total" is a sum. Compute it.

The pattern

agent  ->  verify (separate)  ->  pass   -> act (e.g. pay)
                              ->  retry  -> agent (with reasons)
                              ->  fail   -> human

Verify node with Screwtape.ai

POST https://screwtape.ai/v1/judge takes task, agent_output, optional evidence, criteria and checks, and returns verdict (pass, fail or retry), a 0–1 score (null when no model could score the call), reasons, a signed receipt, and a next_step hint. Deterministic checks include required_fields, sum, equals and numeric. For invoices, POST /v1/duplicate adds a duplicate check against invoices your key recorded earlier.

import os, json, urllib.request

def verify(state):
    inv = state["invoice"]
    n = len(inv.get("items", []))
    body = {
        "task": "Extract the invoice and decide whether to pay it.",
        "agent_output": json.dumps(inv),
        "evidence": [state["invoice_text"]],
        "checks": [
            {"type": "required_fields", "paths": ["invoice_id", "vendor", "total"]},
            {"type": "sum", "parts": [f"items.{i}.amount" for i in range(n)] + ["tax"],
             "total": "total", "tolerance": 0.01},
        ],
    }
    req = urllib.request.Request(
        "https://screwtape.ai/v1/judge", data=json.dumps(body).encode(), method="POST",
        headers={"Authorization": f"Bearer {os.environ['SCREWTAPE_API_KEY']}",
                 "Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=60) as r:
        v = json.load(r)
    return {"verdict": v["verdict"], "reasons": v["reasons"], "receipt": v.get("receipt")}

def route(state):
    return {"pass": "pay", "retry": "agent", "fail": "human"}[state["verdict"]]

# graph.add_node("verify", verify)
# graph.add_conditional_edges("verify", route, {"pay": "pay", "agent": "agent", "human": "human"})

For a CrewAI guardrail version of the same check, see How to validate CrewAI and LangGraph worker output.

Using MCP instead

The MCP server at https://screwtape.ai/mcp (Streamable HTTP; listed in the official MCP Registry as ai.screwtape/screwtape) exposes judge, check_duplicate_invoice, consensus, get_api_key and publish_receipt, among others. An agent can call judge as its verify tool.

Cost

Free key: 100 calls per UTC day, no card (POST /v1/signup). Teams: Team $199/mo and Finance Control $399/mo add pass tokens, linked audit receipts and policy rules. See pricing.

Verdicts are produced by AI models and automated checks and can be wrong. Keep a person in charge of payments.