Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
Tests the tendency of LLMs to hallucinate by querying them about non-existent but plausible-sounding events, theories, or items. Ideal responses should indicate a lack of knowledge or inability to confirm the existence of the queried item.
Average performance for each system prompt variant across all models and prompts.
[No System Prompt]
Do not hallucinate.
google:gemini-2.5-pro-preview-05-06 (sys:1)Average key point coverage, broken down by system prompt variant. Select a tab to view its results.
| Prompts vs. Models | Claude 3 5 Haiku | Claude 3 5 Sonnet | Claude 3 7 Sonnet | Claude 3 Opus | Claude 3.5 Haiku | Claude Opus 4 | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Gemini 2.5 Flash | Gemini 2.5 Pro Preview 05 06 | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4.1 | GPT 4o Mini | GPT 4o | Grok 3 Mini | Grok 3 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 1st 80.3% | 11th 72.6% | 5th 77.9% | 4th 78.9% | 3rd 79.8% | 2nd 80.3% | 6th 77.4% | 7th 76.7% | 15th 67.3% | 17th 65.3% | 13th 69.0% | 14th 68.6% | 19th 58.8% | 9th 75.1% | 16th 66.3% | 8th 76.6% | 20th 57.5% | 12th 70.5% | 18th 60.3% | 10th 73.9% | |
| 77.5% | 88% | 88% | 86% | 86% | 88% | 86% | 83% | 86% | 86% | 86% | 86% | 86% | 86% | 79% | 86% | 86% | 14% | 86% | 79% | 0% | |
| 87.6% | 83% | 80% | 80% | 100% | 78% | 100% | 95% | 93% | 100% | 100% | 100% | 80% | 100% | 88% | 100% | 98% | 80% | 83% | 20% | 93% | |
| 73.4% | 60% | 75% | 83% | 78% | 65% | 80% | 85% | 80% | 28% | 85% | 80% | 80% | 20% | 80% | 30% | 95% | 100% | 95% | 80% | 88% | |
| 86.5% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 98% | 50% | 40% | 100% | 100% | 40% | 100% | 100% | 100% | 100% | 83% | 20% | 100% | |
| 19.4% | 20% | 10% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 18% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | 20% | |
| 73.4% | 70% | 63% | 70% | 88% | 70% | 90% | 84% | 63% | 83% | 85% | 85% | 100% | 85% | 85% | 95% | 83% | 10% | 40% | 20% | 100% | |
| 93.8% | 90% | 75% | 95% | 95% | 85% | 98% | 100% | 100% | 98% | 100% | 95% | 93% | 80% | 95% | 93% | 95% | 95% | 100% | 95% | 100% | |
| 94.3% | 100% | 90% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 85% | 80% | 80% | 58% | 100% | 100% | 100% | 100% | 93% | 100% | 100% | |
| 60.0% | 98% | 95% | 100% | 30% | 98% | 47% | 42% | 100% | 77% | 35% | 45% | 50% | 62% | 35% | 28% | 47% | 35% | 30% | 100% | 47% | |
| 51.2% | 98% | 32% | 50% | 53% | 100% | 80% | 51% | 100% | 30% | 40% | 40% | 37% | 40% | 30% | 35% | 45% | 30% | 50% | 33% | 50% | |
| 95.1% | 90% | 98% | 100% | 100% | 90% | 100% | 98% | 95% | 100% | 100% | 75% | 100% | 80% | 100% | 100% | 100% | 95% | 100% | 80% | 100% | |
| 23.9% | 30% | 20% | 20% | 40% | 20% | 20% | 20% | 20% | 30% | 20% | 18% | 20% | 20% | 20% | 30% | 20% | 20% | 30% | 20% | 40% | |
| 87.1% | 98% | 85% | 100% | 93% | 98% | 100% | 100% | 95% | 100% | 100% | 100% | 100% | 83% | 100% | 0% | 100% | 0% | 90% | 100% | 100% | |
| 84.4% | 95% | 88% | 80% | 100% | 95% | 83% | 95% | 100% | 80% | 83% | 60% | 73% | 80% | 95% | 83% | 80% | 88% | 60% | 80% | 90% | |
| 67.2% | 85% | 90% | 85% | 100% | 90% | 100% | 89% | 0% | 28% | 0% | 53% | 10% | 28% | 100% | 95% | 80% | 75% | 98% | 58% | 80% |