Loading blueprint versions...
Please wait while we gather all the unique runs for this blueprint.
Please wait while we gather all the unique runs for this blueprint.
Please wait while we prepare the detailed comparison.
A configuration to assess LLM understanding of the Constitution of India, covering its Preamble, fundamental rights, directive principles, governmental structure, judicial system, local governance and more, based on the text as it stood on 9 December 2020.
Average key point coverage extent for each model across all prompts.
| Prompts vs. Models | Claude 3.5 Haiku | Claude Sonnet 4 | Command A | Deepseek Chat V3 | Deepseek R1 | Gemini 2.5 Flash | Gemini 2.5 Pro Preview 05 06 | Mistral Large 2411 | Mistral Medium 3 | GPT 4.1 | GPT 4.1 Mini | GPT 4.1 Nano | GPT 4o | GPT 4o Mini | O4 Mini | Grok 3 | Grok 3 Mini | Grok 4 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Score | 15th 78.5% | 6th 89.8% | 12th 83.7% | 8th 87.8% | 5th 90.3% | 3rd 92.8% | 18th 73.3% | 13th 82.8% | 11th 85.7% | 7th 88.6% | 14th 81.1% | 16th 76.9% | 10th 86.8% | 17th 73.8% | 9th 87.6% | 2nd 93.6% | 4th 90.3% | 1st 98.2% | |
| 82.0% | 67% | 75% | 58% | 88% | 96% | 88% | 88% | 88% | 88% | 88% | 67% | 71% | 100% | 42% | 100% | 88% | 88% | 96% | |
| 74.3% | 67% | 67% | 67% | 67% | 67% | 100% | 75% | 67% | 75% | 100% | 67% | 42% | 67% | 42% | 100% | 100% | 67% | 100% | |
| 76.0% | 66% | 97% | 72% | 69% | 72% | 100% | 66% | 75% | 59% | 94% | 69% | 63% | 97% | 59% | 69% | 75% | 66% | 100% | |
| 84.9% | 58% | 100% | 100% | 98% | 100% | 100% | 44% | 85% | 85% | 88% | 63% | 81% | 100% | 60% | 75% | 92% | 100% | 100% | |
| 74.3% | 63% | 91% | 56% | 72% | 63% | 100% | 31% | 63% | 88% | 56% | 56% | 88% | 63% | 81% | 100% | 94% | 75% | 97% | |
| 99.6% | 100% | 100% | 100% | 100% | 100% | 92% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 89.6% | 67% | 71% | 100% | 100% | 96% | 100% | 100% | 100% | 71% | 100% | 79% | 79% | 100% | 67% | 100% | 83% | 100% | 100% | |
| 92.1% | 100% | 100% | 100% | 94% | 100% | 100% | 100% | 63% | 88% | 100% | 100% | 63% | 100% | 50% | 100% | 100% | 100% | 100% | |
| 99.8% | 96% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 91.7% | 100% | 100% | 83% | 83% | 100% | 96% | 75% | 92% | 92% | 83% | 83% | 92% | 83% | 88% | 100% | 100% | 100% | 100% | |
| 97.6% | 100% | 100% | 100% | 100% | 100% | 100% | 56% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 84.1% | 66% | 94% | 75% | 100% | 100% | 88% | 88% | 59% | 91% | 100% | 78% | 66% | 75% | 56% | 100% | 100% | 78% | 100% | |
| 89.2% | 88% | 100% | 75% | 88% | 100% | 75% | 100% | 100% | 72% | 100% | 81% | 88% | 75% | 63% | 100% | 100% | 100% | 100% | |
| 78.5% | 33% | 100% | 75% | 75% | 100% | 75% | 100% | 79% | 67% | 67% | 79% | 88% | 79% | 59% | 79% | 83% | 92% | 83% | |
| 99.3% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 88% | 100% | 100% | 100% | 100% | |
| 81.2% | 54% | 100% | 83% | 100% | 100% | 100% | 2% | 79% | 100% | 100% | 79% | 33% | 92% | 56% | 85% | 100% | 98% | 100% | |
| 57.5% | 100% | 100% | 21% | 38% | 96% | 96% | 63% | 21% | 21% | 25% | 0% | 58% | 100% | 67% | 8% | 100% | 21% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 65.9% | 47% | 72% | 84% | 53% | 63% | 63% | 75% | 75% | 81% | 81% | 25% | 47% | 50% | 50% | 50% | 88% | 91% | 91% | |
| 90.5% | 94% | 97% | 100% | 84% | 100% | 100% | 31% | 94% | 100% | 100% | 94% | 94% | 97% | 91% | 69% | 84% | 100% | 100% | |
| 97.6% | 100% | 100% | 100% | 92% | 96% | 96% | 100% | 88% | 96% | 96% | 100% | 100% | 96% | 96% | 100% | 100% | 100% | 100% | |
| 66.5% | 38% | 72% | 56% | 75% | 56% | 100% | 34% | 47% | 56% | 50% | 75% | 41% | 75% | 56% | 91% | 75% | 100% | 100% | |
| 75.7% | 80% | 75% | 70% | 78% | 75% | 100% | 20% | 70% | 85% | 100% | 73% | 53% | 70% | 70% | 80% | 100% | 63% | 100% | |
| 70.2% | 44% | 58% | 77% | 67% | 83% | 83% | 33% | 38% | 88% | 75% | 83% | 54% | 83% | 60% | 71% | 83% | 92% | 92% | |
| 87.7% | 75% | 100% | 98% | 100% | 98% | 100% | 20% | 100% | 100% | 85% | 85% | 55% | 100% | 85% | 85% | 100% | 93% | 100% | |
| 81.0% | 68% | 60% | 95% | 95% | 95% | 98% | 58% | 95% | 95% | 95% | 95% | 75% | 58% | 35% | 48% | 95% | 98% | 100% | |
| 97.9% | 100% | 100% | 88% | 100% | 100% | 100% | 100% | 83% | 96% | 100% | 100% | 100% | 100% | 96% | 100% | 100% | 100% | 100% | |
| 75.1% | 75% | 69% | 85% | 79% | 83% | 67% | 54% | 67% | 79% | 73% | 85% | 69% | 67% | 67% | 90% | 71% | 79% | 92% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 95.7% | 84% | 100% | 100% | 100% | 100% | 100% | 100% | 78% | 100% | 100% | 100% | 91% | 81% | 88% | 100% | 100% | 100% | 100% | |
| 77.2% | 75% | 75% | 75% | 75% | 88% | 75% | 63% | 75% | 75% | 75% | 75% | 75% | 88% | 75% | 88% | 75% | 75% | 88% | |
| 78.0% | 67% | 67% | 67% | 100% | 67% | 67% | 100% | 100% | 67% | 67% | 67% | 67% | 67% | 67% | 67% | 100% | 100% | 100% | |
| 78.8% | 75% | 100% | 54% | 100% | 67% | 100% | 67% | 100% | 71% | 100% | 67% | 38% | 67% | 38% | 100% | 100% | 75% | 100% | |
| 100.0% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | 100% | |
| 91.2% | 70% | 98% | 98% | 93% | 93% | 90% | 68% | 83% | 98% | 95% | 88% | 90% | 95% | 98% | 100% | 90% | 95% | 100% | |
| 88.3% | 86% | 86% | 86% | 86% | 86% | 86% | 100% | 100% | 86% | 86% | 86% | 86% | 86% | 82% | 86% | 86% | 95% | 95% |