Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
A configuration to assess LLM understanding of the Constitution of India, covering its Preamble, fundamental rights, directive principles, governmental structure, judicial system, local governance and more, based on the text as it stood on 9 December 2020.
Average key point coverage extent for each model across all prompts.
| Prompts vs. Models | Claude 3.5 Haiku | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Gemini 2.5 Flash | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4o | GPT 4o Mini | Grok 3 Mini | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 12th 80.4% | 5th 90.5% | 8th 86.3% | 3rd 91.4% | 1st 94.3% | 9th 84.8% | 6th 88.5% | 4th 90.9% | 10th 84.1% | 11th 82.0% | 7th 88.4% | 13th 77.8% | 2nd 94.3% | |
| 86.0% | 79% | 96% | 79% | 75% | 88% | 88% | 79% | 96% | 79% | 92% | 100% | 67% | 100% | |
| 78.9% | 75% | 75% | 75% | 75% | 100% | 75% | 75% | 100% | 75% | 63% | 75% | 63% | 100% | |
| 75.4% | 69% | 75% | 75% | 81% | 100% | 69% | 94% | 91% | 69% | 66% | 75% | 47% | 69% | |
| 90.0% | 73% | 100% | 88% | 96% | 100% | 83% | 94% | 94% | 92% | 85% | 98% | 71% | 96% | |
| 77.4% | 50% | 91% | 69% | 91% | 100% | 75% | 69% | 88% | 72% | 53% | 66% | 88% | 94% | |
| 99.7% | 100% | 100% | 100% | 100% | 96% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 87.5% | 67% | 75% | 100% | 92% | 100% | 83% | 83% | 100% | 79% | 92% | 83% | 83% | 100% | |
| 90.5% | 100% | 100% | 63% | 100% | 100% | 94% | 81% | 100% | 88% | 75% | 100% | 75% | 100% | |
| 97.2% | 92% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 71% | 100% | |
| 91.8% | 84% | 100% | 96% | 92% | 96% | 100% | 96% | 88% | 83% | 88% | 92% | 79% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 86.0% | 75% | 91% | 84% | 100% | 97% | 75% | 97% | 100% | 72% | 75% | 84% | 84% | 84% | |
| 91.2% | 91% | 100% | 81% | 100% | 75% | 100% | 91% | 100% | 100% | 81% | 100% | 66% | 100% | |
| 81.0% | 55% | 100% | 92% | 100% | 88% | 79% | 79% | 84% | 92% | 79% | 84% | 46% | 75% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 83.7% | 58% | 100% | 92% | 88% | 100% | 77% | 100% | 83% | 77% | 56% | 96% | 65% | 96% | |
| 66.5% | 100% | 100% | 50% | 46% | 100% | 46% | 42% | 13% | 0% | 88% | 100% | 79% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 75.3% | 59% | 81% | 94% | 66% | 84% | 94% | 88% | 78% | 41% | 69% | 78% | 69% | 78% | |
| 93.0% | 88% | 91% | 75% | 94% | 100% | 94% | 88% | 100% | 100% | 91% | 94% | 94% | 100% | |
| 97.8% | 92% | 100% | 100% | 100% | 92% | 100% | 96% | 100% | 100% | 100% | 96% | 96% | 100% | |
| 69.8% | 41% | 78% | 63% | 81% | 84% | 59% | 63% | 88% | 75% | 50% | 63% | 63% | 100% | |
| 83.8% | 80% | 90% | 80% | 98% | 100% | 73% | 93% | 100% | 78% | 60% | 95% | 53% | 90% | |
| 76.5% | 48% | 60% | 73% | 83% | 85% | 67% | 100% | 85% | 73% | 71% | 85% | 65% | 100% | |
| 94.7% | 80% | 100% | 98% | 100% | 100% | 100% | 100% | 90% | 100% | 90% | 100% | 73% | 100% | |
| 81.9% | 90% | 65% | 95% | 98% | 98% | 53% | 93% | 95% | 93% | 90% | 55% | 45% | 95% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 74.0% | 60% | 79% | 85% | 96% | 73% | 69% | 77% | 69% | 81% | 58% | 71% | 71% | 73% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 92.6% | 81% | 84% | 100% | 100% | 100% | 66% | 100% | 100% | 100% | 88% | 88% | 97% | 100% | |
| 78.2% | 75% | 75% | 81% | 81% | 75% | 75% | 81% | 75% | 81% | 81% | 81% | 81% | 75% | |
| 70.5% | 67% | 67% | 67% | 100% | 79% | 67% | 67% | 67% | 67% | 67% | 67% | 67% | 67% | |
| 76.4% | 67% | 100% | 54% | 67% | 100% | 100% | 71% | 100% | 67% | 50% | 67% | 50% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 93.1% | 93% | 90% | 98% | 98% | 93% | 90% | 93% | 95% | 90% | 90% | 90% | 90% | 100% | |
| 86.3% | 86% | 86% | 86% | 84% | 86% | 86% | 86% | 86% | 86% | 86% | 86% | 82% | 96% |