Screwtape.ai
Menu

Guide

How to add a validation node to a LangGraph agent (pass, retry, fail)

20 October 2026

How to add a validation node to a LangGraph agent (pass, retry, fail)

Short answer: Put a judge node after your worker node. Have it call a validator, store the verdict in state, and use add_conditional_edges to route: pass goes to accept, retry loops back to the worker (with a cap), and fail goes to reject. Screwtape.ai's POST /v1/judge returns exactly those three verdicts, plus a 0 to 1 score and reasons you can feed back to the worker on a retry.

Screwtape.ai is an agent validation API. It is not the arXiv paper that is also called Screwtape.

The graph

START -> worker -> judge --pass--> accept -> END
                     |---retry--> worker (attempts < 3)
                     '---fail/limit--> reject -> END

Working example

Tested on 2026-10-20 with langgraph 1.2.16:

"""LangGraph: a worker node, a Screwtape judge node, and routing on pass / retry / fail."""
import json
import os
import urllib.request
from typing import TypedDict

from langgraph.graph import END, START, StateGraph

API = "https://screwtape.ai"
MAX_ATTEMPTS = 3


def screwtape_post(path: str, payload: dict) -> dict:
    """POST JSON to Screwtape.ai. Reads SCREWTAPE_API_KEY from the environment."""
    request = urllib.request.Request(
        API + path,
        data=json.dumps(payload).encode(),
        headers={
            "Authorization": "Bearer " + os.environ["SCREWTAPE_API_KEY"],
            "Content-Type": "application/json",
        },
        method="POST",
    )
    with urllib.request.urlopen(request, timeout=60) as response:
        return json.load(response)


class State(TypedDict, total=False):
    task: str
    evidence: list[str]
    criteria: list[str]
    agent_output: str
    attempts: int
    feedback: list[str]
    verdict: dict
    status: str


def worker(state: State) -> dict:
    # Replace with your LLM call. Pass state.get("feedback") back into the prompt on a retry.
    return {
        "agent_output": "The crew size was 4. The mission launched from Cape Canaveral.",
        "attempts": state.get("attempts", 0) + 1,
    }


def judge(state: State) -> dict:
    payload = {"task": state["task"], "agent_output": state["agent_output"]}
    if state.get("evidence"):
        payload["evidence"] = state["evidence"]
    if state.get("criteria"):
        payload["criteria"] = state["criteria"]
    verdict = screwtape_post("/v1/judge", payload)
    return {"verdict": verdict, "feedback": verdict["reasons"]}


def route(state: State) -> str:
    v = state["verdict"]["verdict"]
    if v == "pass":
        return "accept"
    if v == "retry" and state.get("attempts", 0) < MAX_ATTEMPTS:
        return "worker"
    return "reject"


def accept(state: State) -> dict:
    return {"status": "accepted"}


def reject(state: State) -> dict:
    return {"status": "rejected"}


graph = StateGraph(State)
graph.add_node("worker", worker)
graph.add_node("judge", judge)
graph.add_node("accept", accept)
graph.add_node("reject", reject)
graph.add_edge(START, "worker")
graph.add_edge("worker", "judge")
graph.add_conditional_edges("judge", route, ["accept", "worker", "reject"])
graph.add_edge("accept", END)
graph.add_edge("reject", END)
app = graph.compile()

if __name__ == "__main__":
    result = app.invoke({
        "task": "State the crew size and launch site.",
        "evidence": [
            "The mission launched on 2024-03-01 from Cape Canaveral.",
            "The crew size was 4.",
        ],
        "criteria": ["crew size", "Cape Canaveral"],
    })
    print(result["status"], result["verdict"])

Notes:

Validating several workers at once

If you fan out to several workers, add a node that POSTs their outputs to /v1/consensus (2 to 16 answers) and route on false_consensus_risk before the judge node. See stop false consensus.

FAQ

How do I add validation to a LangGraph agent?

Add a node that calls a validator after the node that produces output, store the result in state, and add a conditional edge whose router returns the next node name based on that result.

How do I stop a LangGraph retry loop from running forever?

Keep an attempts counter in state and have the router return the reject node once the counter reaches your cap. LangGraph also has a recursion limit, but an explicit cap gives you a clean rejected status.

What should the judge node return?

A partial state update. In the example it returns the full Screwtape response under verdict and the reasons under feedback.

Is there an official Screwtape LangGraph package?

No. Screwtape has no SDK. The node calls the HTTP API directly. Screwtape's own one-node LangGraph example is at https://screwtape.ai/integrations/langgraph/.

Sources