Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
Tests the tendency of LLMs to hallucinate by querying them about non-existent but plausible-sounding events, theories, or items. Ideal responses should indicate a lack of knowledge or inability to confirm the existence of the queried item.
Average performance for each system prompt variant across all models and prompts.
[No System Prompt]
Do not hallucinate.
deepseek:deepseek-r1 (sys:0)openai:o4-mini (sys:0)openai:o4-mini (sys:1)xai:grok-4 (sys:0)xai:grok-4 (sys:1)Average key point coverage, broken down by system prompt variant. Select a tab to view its results.
| Prompts vs. Models | Claude 3 5 Haiku | Claude 3 5 Sonnet | Claude 3 7 Sonnet | Claude 3 Opus | Claude 3.5 Haiku | Claude Opus 4 | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro Preview 05 06 | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4.1 | GPT 4o Mini | GPT 4o | O4 Mini | Grok 3 Mini | Grok 3 | Grok 4 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 1st 80.3% | 12th 72.6% | 5th 77.9% | 4th 78.9% | 3rd 79.8% | 2nd 80.3% | 6th 77.4% | 7th 76.7% | 16th 67.3% | 19th 64.3% | 18th 65.3% | 14th 69.0% | 15th 68.6% | 21st 58.8% | 9th 75.1% | 17th 66.3% | 8th 76.6% | 22nd 57.5% | 13th 70.5% | 20th 62.6% | 11th 73.7% | 10th 73.9% | 23rd 56.3% | |
| 75.3% | 88% | 88% | 86% | 86% | 88% | 86% | 83% | 86% | 86% | 82% | 86% | 86% | 86% | 86% | 79% | 86% | 86% | 14% | 86% | 50% | 79% | 0% | 50% | |
| 88.9% | 83% | 80% | 80% | 100% | 78% | 100% | 95% | 93% | 100% | 98% | 100% | 100% | 80% | 100% | 88% | 100% | 98% | 80% | 83% | 98% | 98% | 93% | 20% | |
| 72.0% | 60% | 75% | 83% | 78% | 65% | 80% | 85% | 80% | 28% | 85% | 85% | 80% | 80% | 20% | 80% | 30% | 95% | 100% | 95% | 85% | 80% | 88% | 20% | |
| 88.3% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 98% | 50% | 100% | 40% | 100% | 100% | 40% | 100% | 100% | 100% | 100% | 83% | 50% | 80% | 100% | 90% | |
| 19.5% | 20% | 10% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 18% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | |
| 77.6% | 70% | 63% | 70% | 88% | 70% | 90% | 84% | 63% | 83% | 85% | 85% | 85% | 100% | 85% | 85% | 95% | 83% | 10% | 40% | 85% | 83% | 100% | 83% | |
| 92.7% | 90% | 75% | 95% | 95% | 85% | 98% | 100% | 100% | 98% | 80% | 100% | 95% | 93% | 80% | 95% | 93% | 95% | 95% | 100% | 100% | 95% | 100% | 75% | |
| 94.3% | 100% | 90% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 85% | 80% | 80% | 58% | 100% | 100% | 100% | 100% | 93% | 83% | 100% | 100% | 100% | |
| 57.3% | 98% | 95% | 100% | 30% | 98% | 47% | 42% | 100% | 77% | 37% | 35% | 45% | 50% | 62% | 35% | 28% | 47% | 35% | 30% | 40% | 100% | 47% | 40% | |
| 49.4% | 98% | 32% | 50% | 53% | 100% | 80% | 51% | 100% | 30% | 32% | 40% | 40% | 37% | 40% | 30% | 35% | 45% | 30% | 50% | 40% | 33% | 50% | 40% | |
| 95.6% | 90% | 98% | 100% | 100% | 90% | 100% | 98% | 95% | 100% | 98% | 100% | 75% | 100% | 80% | 100% | 100% | 100% | 95% | 100% | 100% | 80% | 100% | 100% | |
| 23.4% | 30% | 20% | 20% | 40% | 20% | 20% | 20% | 20% | 30% | 20% | 20% | 18% | 20% | 20% | 20% | 30% | 20% | 20% | 30% | 20% | 20% | 40% | 20% | |
| 87.0% | 98% | 85% | 100% | 93% | 98% | 100% | 100% | 95% | 100% | 80% | 100% | 100% | 100% | 83% | 100% | 0% | 100% | 0% | 90% | 78% | 100% | 100% | 100% | |
| 82.1% | 95% | 88% | 80% | 100% | 95% | 83% | 95% | 100% | 80% | 38% | 83% | 60% | 73% | 80% | 95% | 83% | 80% | 88% | 60% | 80% | 80% | 90% | 83% | |
| 59.4% | 85% | 90% | 85% | 100% | 90% | 100% | 89% | 0% | 28% | 10% | 0% | 53% | 10% | 28% | 100% | 95% | 80% | 75% | 98% | 10% | 58% | 80% | 3% |