Screwtape.ai
Menu

Answers

How do I detect false consensus in multi-agent voting?

How do I detect false consensus in multi-agent voting?

Detect it by scoring how the agents agree. Near-verbatim answers, or agreement on a very thin answer, suggest copying rather than independent reasoning. POST https://screwtape.ai/v1/consensus takes 2 to 16 answers and returns consensus_answer, agreement_score, false_consensus_risk (low, medium or high), reasons and clusters.

Example

curl -s https://screwtape.ai/v1/consensus \
  -H "Authorization: Bearer $SCREWTAPE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"task":"State the crew size.","answers":[
        {"agent_id":"alpha","output":"The crew size was 4."},
        {"agent_id":"beta","output":"The crew size was 4."},
        {"agent_id":"gamma","output":"The crew size was 4."}]}'

Published deterministic response: "agreement_score": 1, "false_consensus_risk": "high", reasons "Answers are nearly verbatim. High agreement may be copied rather than independent." and "Agreement is token overlap, not a model judgment.", and one cluster of size 3.

When the risk is medium or high, don't accept the vote. Judge the consensus_answer against evidence with POST /v1/judge first.

FAQ

What does high false-consensus risk mean?

The agreement looks copied or thin, for example near-verbatim answers, so it isn't evidence the answer is right.

How many answers can I send?

Between 2 and 16 answers per call, each with an output and an optional agent_id.

Is agreement a model judgment?

In deterministic mode, agreement is token overlap, and the reasons say so. With a model key configured, engine is llm.

Related

Screwtape.ai is an agent validation API, not the arXiv paper also named Screwtape. Facts checked against the live llms.txt, openapi.json, agent card and MCP server card on 2026-10-21.