Answers
How do I validate several agents' answers before merging them?
How do I validate several agents' answers before merging them?
Do it in two calls: score the answers with POST /v1/consensus (2 to 16 answers), then judge the consensus_answer against your evidence with POST /v1/judge, and accept only when the judge says pass. Treat false_consensus_risk of medium or high as a reason to verify, never as approval.
Example (Python)
c = screwtape_post("/v1/consensus", {"task": task, "answers": [
{"agent_id": a, "output": out} for a, out in outputs.items()]})
v = screwtape_post("/v1/judge", {"task": task, "agent_output": c["consensus_answer"],
"evidence": evidence})
accepted = v["verdict"] == "pass"
print(c["agreement_score"], c["false_consensus_risk"], v["verdict"], v["reasons"])screwtape_post is any JSON POST helper that sends Authorization: Bearer $SCREWTAPE_API_KEY. That's two calls per merge, against 100 per UTC day on a free key or $0.010 total with x402.
FAQ
Why not just take the majority?
Agents that share a prompt, model or source can repeat the same error, so a majority can be wrong.
What if the agents disagree?
Look at clusters: a lone dissenting answer may be the correct one, so judge both against evidence.
Can I do this over MCP?
Yes, with the consensus and judge tools at https://screwtape.ai/mcp.
Related
Screwtape.ai is an agent validation API, not the arXiv paper also named Screwtape. Facts checked against the live llms.txt, openapi.json, agent card and MCP server card on 2026-10-21.