Screwtape.ai
Menu

Guide

How to validate CrewAI task output before the next task uses it

20 October 2026

How to validate CrewAI task output before the next task uses it

Short answer: Attach a function-based guardrail to the CrewAI Task. CrewAI calls it with the TaskOutput. If it returns (False, message), CrewAI sends the message back to the agent and retries, up to guardrail_max_retries times (default 3, per the CrewAI docs). Inside the guardrail, send output.raw to an external judge such as Screwtape.ai's POST /v1/judge and branch on pass, fail or retry.

Screwtape.ai is an agent validation API. It is not the arXiv paper that is also called Screwtape.

Why an external judge instead of CrewAI's string guardrail?

CrewAI also accepts a string guardrail, which "use[s] the agent's LLM to validate outputs". That's convenient, but the same model then grades its own work. Panickssery et al. (2024) found that LLM evaluators recognize their own outputs and score them higher than human annotators do (arXiv:2404.13076). An external judge that you also give evidence to sidesteps that self-grading loop. Function guardrails and string guardrails can also be combined in guardrails=[...].

Working example

Tested for import and request shape on 2026-10-20 with crewai 1.15.28. The judge call was sent to the live API with a placeholder key and returned the expected 401. The request body matches JudgeRequest in openapi.json.

"""CrewAI task guardrail that asks Screwtape.ai for a verdict before the output moves on."""
import json
import os
import urllib.request

from crewai import Agent, Crew, Task, TaskOutput

API = "https://screwtape.ai"
EVIDENCE = [
    "The mission launched on 2024-03-01 from Cape Canaveral.",
    "The crew size was 4.",
]


def screwtape_post(path: str, payload: dict) -> dict:
    """POST JSON to Screwtape.ai. Reads SCREWTAPE_API_KEY from the environment."""
    request = urllib.request.Request(
        API + path,
        data=json.dumps(payload).encode(),
        headers={
            "Authorization": "Bearer " + os.environ["SCREWTAPE_API_KEY"],
            "Content-Type": "application/json",
        },
        method="POST",
    )
    with urllib.request.urlopen(request, timeout=60) as response:
        return json.load(response)


def screwtape_guardrail(output: TaskOutput) -> tuple[bool, str]:
    verdict = screwtape_post("/v1/judge", {
        "task": "State the crew size and launch site.",
        "agent_output": output.raw,
        "evidence": EVIDENCE,
        "criteria": ["crew size", "Cape Canaveral"],
    })
    if verdict["verdict"] == "pass":
        return True, output.raw
    # CrewAI sends this text back to the agent and retries, up to guardrail_max_retries.
    return False, "Screwtape verdict {} (score {}): {}".format(
        verdict["verdict"], verdict["score"], " ".join(verdict["reasons"])
    )


writer = Agent(
    role="Mission summarizer",
    goal="Answer only from the supplied facts.",
    backstory="You summarize mission briefs without adding details.",
)

task = Task(
    description="Facts: " + " ".join(EVIDENCE) + " State the crew size and launch site.",
    expected_output="Two short sentences.",
    agent=writer,
    guardrail=screwtape_guardrail,
    guardrail_max_retries=2,
)

if __name__ == "__main__":
    print(Crew(agents=[writer], tasks=[task]).kickoff().raw)

The guardrail treats both retry and fail as "send it back", because a CrewAI guardrail can only accept or retry. If you want a hard stop on fail, check task.output after kickoff() and call the judge again, or run the judge outside the crew. The retry vs fail page covers that pattern.

The response you branch on

POST /v1/judge returns:

Alternative: give the reviewer agent a tool

Screwtape's CrewAI integration page shows a BaseTool named screwtape_judge that a reviewer agent can call. Use a guardrail when the check must always run. Use a tool when you want the agent to decide when to check.

FAQ

How do I validate CrewAI task output?

Pass a function to the Task's guardrail parameter. The function receives a TaskOutput and returns (True, value) to accept or (False, message) to send feedback to the agent and retry, up to guardrail_max_retries times.

What does a CrewAI guardrail need to return?

A tuple. Return (True, validated_result) on success and (False, "error message") on failure, as described in the CrewAI Tasks documentation.

Can I use an LLM to check CrewAI output?

Yes. A string guardrail uses the agent's own LLM. An external judge such as Screwtape's POST /v1/judge keeps the grader separate from the model that wrote the output, and it can check claims against evidence you supply.

Does Screwtape need a CrewAI plugin or SDK?

No. It is a plain HTTP API, so the example uses Python's urllib. Set SCREWTAPE_API_KEY from a free key at https://screwtape.ai/signup/ (100 calls per UTC day).

Sources