MMLU-Pro
Broad academic knowledge and reasoning.
Prompting and reasoning budgets can differ between submissions.Choose the capabilities that matter to you. Set minimum scores and explore which models meet every requirement.
Bars show the lowest and highest published scores for a model and benchmark variant. Dots mark range midpoints. Setups can differ; minimums are checked against the lower end of each range.
Choose one benchmark to see models with a single reported score. HLE and AIME evaluation setups are selected in your requirements.
Open weights · 4 of 6 benchmarks in selected setups
Compare memory fit, estimated speed and cost for this exact model.
FP8 · 8K context · 30 tok/s target. Uses FP8 because the current format is not listed for this model. Your other simulation settings are kept.
Evaluation settings not supplied
Aggregated / community report · MITEvaluation settings not supplied
Aggregated / community report · MITQwen reports its own earlier model in this comparison and explicitly identifies the full AIME 2026 I and II set; these values may differ from its original Qwen3.5 notes.
Model developer report · Apache-2.0Evaluation settings not supplied
Aggregated / community report · MITQwen reports its own earlier model in this comparison. Internal agent with bash and file-edit tools; temperature 1.0, top-p 0.95, 200K context window; attempt count unspecified.
Model developer report · Apache-2.0Switch setup to compare these results separately.
113 models in this selection. Select a model to inspect it above. Scores are percentages; a dash means no result for the selected setup. Sorting uses the lower end.
| Model | Your requirements | MMLU-Pro≥ 60% | GPQA≥ 50% | GSM8KNo minimum | AIME 2026AIME 2026, full setNo minimum | HLEText, no toolsNo minimum | SWE VerifiedNo minimum |
|---|---|---|---|---|---|---|---|
| Missing required score | 88% | — | — | — | — | 74% | |
| Meets all minimums | 87.8% | 88.4% | — | 93.3% | — | 76.2–76.4% | |
| Meets all minimums | 87.1% | 87.6% | — | — | — | 70.8–76.8% | |
| Meets all minimums | 86.7% | 86.6% | — | — | — | 72% | |
| Meets all minimums | 86.2% | 87.8% | — | 94.1% | — | 77.2% | |
| Meets all minimums | 86.1% | 85.5% | — | 92.6% | — | 72.4–75% | |
| Meets all minimums | 85.3% | 84.2% | — | 91% | — | 69.2–70% | |
| Meets all minimums | 85.2% | 86% | — | 92.7% | — | 73.4% | |
| Meets all minimums | 85% | 81% | — | — | 17.7% | 57.6% | |
| Meets all minimums | 85% | 82.4% | — | — | — | 70% | |
| Meets all minimums | 84.6% | 84.5% | — | — | — | 71.3% | |
| Missing required score | 84.4% | — | — | — | — | — | |
| Meets all minimums | 84.4% | 83.5% | — | — | 23.1% | 74.4% | |
| Meets all minimums | 84.3% | 85.7% | — | — | — | 73.8% | |
| Meets all minimums | 84.2% | 85.2% | — | — | — | 76.5% | |
| Meets all minimums | 84% | 71.5% | — | — | — | — | |
| Meets all minimums | 83.8% | 79.1% | — | — | — | — | |
| Meets all minimums | 83.73% | 79.23% | — | — | — | 53.73% | |
| Meets all minimums | 83.73% | 79.23% | — | — | — | 53.73% | |
| Meets all minimums | 82.5% | 81.7% | — | — | — | — | |
| Missing required score | 82% | — | — | — | — | 69.4% | |
| Meets all minimums | 81.94% | 75.44–75.57% | — | — | 11.72% | 51.56% | |
| Missing required score | 81.2–81.3% | — | — | — | — | — | |
| Meets all minimums | 81.02% | 74.43% | — | — | — | — | |
| Missing required score | 80.6% | — | — | — | — | — | |
| Meets all minimums | 79.8% | 76.1% | — | — | — | — | |
| Meets all minimums | 79.1% | 76.2% | — | — | — | — | |
| Missing required score | 78.3% | — | — | — | — | — | |
| Missing required score | 78.29% | — | — | — | — | — | |
| Missing required score | 78.1% | — | — | — | — | — | |
| Meets all minimums | 77.2% | 78.8% | — | — | — | — | |
| Missing required score | 75.87% | — | — | — | — | — | |
| Missing required score | 75.2% | — | — | — | — | — | |
| Missing required score | 75.2% | — | — | — | — | — | |
| Meets all minimums | 74% | 65.8% | — | — | — | — | |
| Missing required score | 72.1% | — | — | — | — | — | |
| Missing required score | 70.8% | — | — | — | — | 46.4% | |
| Meets all minimums | 69.6% | 62% | — | — | — | — | |
| Meets all minimums | 69.4% | 58.6% | — | — | — | — | |
| Missing required score | 64.4% | — | 89.3% | — | — | — | |
| Missing required score | 62.84% | — | — | — | — | — | |
| Missing required score | 61.6% | — | 96.8% | — | — | — | |
| Below a minimum | 61.5% | 38.4% | — | — | — | — | |
| Below a minimum | 60% | 43.4% | — | — | — | — | |
| Below a minimum | 55.3% | — | — | — | — | — | |
| Below a minimum | 52.47% | — | — | — | — | — | |
| Below a minimum | 48.9% | 38.1% | — | — | — | — | |
| Below a minimum | 47.4–47.87% | — | 86.2% | — | — | — | |
| Below a minimum | 47.24% | — | — | — | — | — | |
| Below a minimum | 44.25–48.3% | 30.4% | 84.5% | — | — | — | |
| Below a minimum | 40.73% | — | 79.9% | — | — | — | |
| Below a minimum | 36.6% | — | — | — | — | — | |
| Below a minimum | 29.7% | 11.9% | — | — | — | — | |
| Missing required score | — | 92.9% | — | — | — | — | |
| Missing required score | — | — | 79.2% | — | — | — | |
| Missing required score | — | — | — | — | 39.1% | — | |
| Missing required score | — | — | — | — | 44% | — | |
| Missing required score | — | — | — | — | — | 53.9% | |
| Missing required score | — | — | — | — | — | 62.4% | |
| Missing required score | — | — | — | — | — | 66% | |
| Missing required score | — | — | — | — | — | — | |
| Below a minimum | — | 27.3% | — | — | — | — | |
| Missing required score | — | 68.9% | — | — | 14% | 65.6% | |
| Missing required score | — | 75.6% | — | — | 19.5% | 72.4% | |
| Missing required score | — | — | 86% | — | — | — | |
| Missing required score | — | — | 79.6% | — | — | — | |
| Below a minimum | — | 38.89% | — | — | — | — | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | 80.5% | — | — | — | — | |
| Missing required score | — | — | — | 94.7% | — | 76% | |
| Missing required score | — | — | 91% | — | — | — | |
| Missing required score | — | — | 85.7% | — | — | — | |
| Missing required score | — | 85.2% | — | — | — | 75.8% | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | 71.2% | — | — | — | — | |
| Missing required score | — | 90.5% | — | — | 36.4% | 80.2% | |
| Missing required score | — | — | — | — | — | 74.8% | |
| Missing required score | — | 83.8% | — | — | — | — | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | 86.43% | — | — | — | — | |
| Missing required score | — | 88.83% | — | — | — | — | |
| Missing required score | — | 89.52% | — | — | 31.79% | 71.1% | |
| Missing required score | — | 89.9% | — | — | 31.44% | 71.07% | |
| Missing required score | — | 69.19–72.41% | — | — | 8.25–10.89% | 50.3–61.8% | |
| Missing required score | — | 67.1% | — | — | — | 47.9% | |
| Missing required score | — | 56.8% | — | — | — | 37.4% | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | 50.5% | — | 52.7% | — | — | |
| Below a minimum | — | 37.9% | — | 29.3% | — | — | |
| Missing required score | — | 60.1% | — | — | — | — | |
| Missing required score | — | 68.7% | — | — | — | — | |
| Missing required score | — | — | 89.5% | — | — | — | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | — | — | — | — | 70.6% | |
| Missing required score | — | 89.2% | — | — | — | — | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | — | — | — | — | 74.2% | |
| Missing required score | — | — | — | — | — | 74.1% | |
| Missing required score | — | 90.6% | — | — | — | 81.6% | |
| Missing required score | — | 90.3% | — | — | — | 80.2% | |
| Missing required score | — | 89.3% | — | — | — | 80.7% | |
| Missing required score | — | — | 50.57% | — | — | — | |
| Missing required score | — | — | — | — | — | 52.6% | |
| Missing required score | — | — | — | — | — | 42.2% | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | 56.1% | — | — | — | 59.8% | |
| Missing required score | — | 77.1% | — | — | — | 62.8% | |
| Missing required score | — | — | — | — | — | — | |
| Missing required score | — | — | — | — | — | 75% | |
| Missing required score | — | 75.2% | — | — | — | 59.2% | |
| Missing required score | — | 86% | — | — | — | 72.8% | |
| Missing required score | — | — | — | — | — | — |
We retain source and reuse terms for each score collection. This page reproduces numeric results and model labels. Benchmark questions, answers and outputs are excluded.
OpenEvals declares MIT for this aggregate dataset. The snapshot does not supply each result’s original publication or license, so upstream rights are not independently verified. This declaration does not cover benchmark questions, model weights, or third-party source material.
Revision d22dd54c048a · retrieved 2026-10-08The pinned results.csv repository revision identifies its dataset license as Apache-2.0. This supplement reproduces selected scalar results and model labels from that licensed results file, with attribution. The license basis applies to this result file, not to MMLU-Pro questions or model weights. The CSV's Data Source field is retained in reporting and setup notes; a Self-Reported row is not described as an independent TIGER-Lab run. The pinned repository tree contains only .gitattributes, README.md, and results.csv; no separate LICENSE or NOTICE file is present. The source README declares license: apache-2.0 and its exact copy is preserved in public/licenses.
Revision 8a87f5d0ae23 · retrieved 2026-10-08The pinned repository contains a root MIT LICENSE whose grant expressly covers software and associated documentation files; the score table is in the README. The GPQA-Diamond result is taken from the licensed README evaluation table; the repository's model-weight terms are not used as the score-data license basis.
Revision 4236a6af538f · retrieved 2026-10-08The pinned repository contains a root MIT LICENSE whose grant expressly covers software and associated documentation files; the score table is in the README. The MMLU-Pro result and its stated setup are from the licensed README, not inferred from the model-weight metadata.
Revision 2fe192450127 · retrieved 2026-10-08The pinned repository contains an explicit root Apache-2.0 LICENSE. Its README contains the selected score rows, with no separate README license or exclusion noted in the repository. The score table is treated as part of the licensed repository Work; this basis is the explicit repository license, not the model-weight license metadata.
Revision 6a9e13bd6fc8 · retrieved 2026-10-08The pinned repository contains an explicit root Apache-2.0 LICENSE. Its README contains the selected score rows, with no separate README license or exclusion noted in the repository. The score table is treated as part of the licensed repository Work; this basis is the explicit repository license, not the model-weight license metadata.
Revision 995ad96eacd9 · retrieved 2026-10-08The pinned model repository contains an explicit Modified MIT LICENSE covering the Software and associated documentation files. Selected scalar results and protocol labels are taken from its README, which is covered as repository documentation; this is not a reliance on model-weight metadata alone. The Modified MIT notice also requires prominent Kimi K2.6 display if the licensed material is used in products or services above its stated monthly-user or revenue thresholds.
Revision 7eb5002f6aad · retrieved 2026-10-08Selected scalar results for Google’s own E2B, E4B and 12B IT models were published in the Google Developers Gemma 4 model card, whose footer licenses page content under CC BY 4.0 except where otherwise noted; no benchmark-table exception was shown. The same values were verified in the linked immutable Google-authored Hugging Face README. Its revision pins the result values; the Google Developers publication supplies the documentation reuse basis. The local notice preserves attribution, license link, adaptation description and retrieval fingerprint. Model-weight license metadata is not treated as permission for result tables. No benchmark questions, third-party baseline results or artwork are reproduced.
Revision 3e22461f65e8 · retrieved 2026-10-16The pinned README expressly releases both the repository and MiniCPM model weights under Apache-2.0 and links the OpenBMB repository license, whose exact text and copyright notice are preserved locally at a pinned revision. Only OpenBMB’s internally reproduced results for its own MiniCPM5-2B model are imported. Rows marked with a dagger as originating from Artificial Analysis are excluded because OpenBMB’s license statement does not independently establish redistribution rights for those third-party results. No comparison-model rows, questions or artwork are reproduced.
Revision f97400052a43 · retrieved 2026-10-16The explicit root LICENSE at the pinned revision defines Model Materials to include associated documentation, and grants rights to deal in those materials, with license and applicable notices retained on distribution. This scope was checked in the license text rather than inferred from model-weight metadata. The selected scalars are NVIDIA’s own BF16 model results, described by NVIDIA as measured under its NeMo evaluation harness. No competitor rows, benchmark questions or figures are copied. The NVIDIA copyright notice and complete applicable license are preserved locally.
Revision a9904d24bcc1 · retrieved 2026-10-16The original resources/bench_53.png is published in Z.ai’s official GLM-5 GitHub repository under an explicit root Apache-2.0 LICENSE, with no separate restriction identified for this repository asset. Only Z.ai’s own HLE-with-tools numeric result is transcribed; the chart, logos and competitor results are not reproduced. The matching pinned GLM-5.3-Flash model card identifies the full-set protocol and evaluation settings. The figure’s repository license, not a model-weight license or the license of a linking page, establishes the result-publication reuse basis.
Revision 83c03d317378 · retrieved 2026-10-16The pinned README explicitly licenses this repository and model weights under MIT; the root LICENSE separately covers software and associated documentation. DeepSeek is the reporting party for both its own exact Flash and Flash-1017 checkpoints. Only four numerical HLE facts and necessary settings are transcribed with original explanations; competitor results, questions, prompts, images and source prose are excluded. Original copyright and permission notice are retained. README UTF-8 retrieval SHA-256: b72b796e4f4c739eca5d0ad360a7679b39118ca7eb64891f7151a527b292c42a; original LICENSE bytes SHA-256: f2c6c602815669d292889e5be8c802f2ed950653b77999b1584e8e6aed25d040. The documentation grant, not weight-license metadata alone, supplies the result reuse basis.
Revision 1bae0e108899 · retrieved 2026-10-18The NVIDIA-authored card's License/Terms of Use links the OpenMDW-1.1 governing agreement. The reviewed agreement defines Model Materials to include associated documentation and data and grants copyright and database reuse rights in those materials. This explicit linked documentation grant, not YAML weight-license metadata alone, covers the selected publisher numerical results in the accompanying card. No root LICENSE file is claimed. Only NVIDIA's own model results are transcribed, with original setup notes; third-party baseline columns, benchmark content, figures, logos and source prose are excluded. Full agreement and source-origin notices are preserved locally. README UTF-8 SHA-256: 3db3d81d4b89a9616f6a775181679497c1363a3b62b97c3cfdc73178378a3ce4; linked official agreement pinned at 9067ba54b2f84bc9e15a05734f570ec88e9f445d, original bytes SHA-256 2ab44b68365473c112f5092211a38f231cb23e50de68b75a13369adbd76a74df.
Revision b366b87f762e · retrieved 2026-10-19The NVIDIA-authored card's License/Terms of Use links the OpenMDW-1.1 governing agreement. The reviewed agreement defines Model Materials to include associated documentation and data and grants copyright and database reuse rights in those materials. This explicit linked documentation grant, not YAML weight-license metadata alone, covers the selected publisher numerical results in the accompanying card. The pinned root LICENSE is an empty stub (zero bytes, SHA-256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855), so it supplies no permission; the reviewed linked agreement supplies the documentation grant. Only NVIDIA's own model results are transcribed, with original setup notes; third-party baseline columns, benchmark content, figures, logos and source prose are excluded. Full agreement and source-origin notices are preserved locally. README UTF-8 SHA-256: 214a91779130828f913e5c1d9b982731fc529e3e87277c6cfaf23f66d1bb478f; linked official agreement pinned at 9067ba54b2f84bc9e15a05734f570ec88e9f445d, original bytes SHA-256 2ab44b68365473c112f5092211a38f231cb23e50de68b75a13369adbd76a74df.
Revision 567021037ff8 · retrieved 2026-10-19The Meta-authored repository has an explicit Apache-2.0 root LICENSE and the original README identifies Apache-2.0 without limiting it to model weights. This repository documentation grant covers the two selected publisher numerical results. Model-card license metadata alone is not the basis. Only Meta's own AIME 2026 and SWE-bench Verified observations are transcribed with original setup notes. AA-attributed GPQA/HLE and third-party comparison columns are excluded. The linked methodology report supplies factual protocol details; its content is not redistributed and the repository license is not claimed to license that separate PDF. No benchmark questions, answers, prompts, code, figures, logos or publisher prose are imported. README UTF-8 SHA-256: 2ba646dcbe5b3b97daedc59e64ef19505312fa99b6598cacc0b82d93f4d74163; root LICENSE SHA-256: cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30. Methodology PDF retrieved 2026-10-21 SHA-256: fb08920ab31c5df3e07e2c3b32856d80d65a48dd14864f42bce8a1ac38026c91. The complete root license and source-origin/change notice are preserved locally; no NOTICE file was present.
Revision 7af58019cf47 · retrieved 2026-10-21Reflection's original repository contains an explicit Apache-2.0 root LICENSE, and the README identifies that license without a weight-only limitation. This repository documentation grant covers the selected publisher-owned result table; the model-card metadata alone is not the basis. The card reports separate BF16, FP8 and NVFP4 instruction-tuned columns. All 15 supported values are transcribed, with original setup notes and unknown settings preserved. Both light/dark renderings and all three checkpoint cards agree; repeated renderings are not independent observations. Only Reflection's own results are imported. No unsupported benchmark, competitor result, question, answer, prompt, model code, image or source prose is copied. README SHA-256: 62d54c638f3bb62c15fcb782af39242584c65cd41e4d8618700a2213de48aacb; root LICENSE SHA-256: 43070e2d4e532684de521b885f385d0841030efa2b1a20bafb76133a5e1379c1. The complete license and original attribution/change notice are preserved locally. No NOTICE file is present in any of the three reviewed repositories. FP8 checkpoint revision 077872da1757aa43e0330ab924b834825c54ecbc, README SHA-256 081a22b31d9ead4f42054a6c9cc7dac29c88245926d1446db5f1d58069490416; NVFP4 checkpoint revision f8ecac0e16045babf60156f6ba996a77fcb48cf9, README SHA-256 d4b0193fb0d6f6cdec2530c63ac2fb0d250777324aa9d067b429e29fe12087e5; both root licenses are byte-identical to the BF16 repository's license.
Revision 50a18f8ccf28 · retrieved 2026-10-22The original publisher GitHub repository contains an explicit, unqualified Apache-2.0 root LICENSE covering its README result publication. That original README reports the selected model's own AIME 2026 and SWE-bench Verified results, which agree with the official Hugging Face card. The missing root LICENSE in the Hugging Face repository and its model-weight license label alone are not used as the documentation grant. The linked GitHub model-use disclaimer concerns use of Xing4.0 or derivative models; no model is executed or redistributed and no model-use clearance is claimed here. Only the two numerical facts and original protocol notes are imported; competitor scores, questions, answers, prompts, source code, figures and publisher prose are excluded. GitHub README SHA-256: 73e5d8257d484ac95f043598ed4d2fa4e76ea73a976908c141f5ad1bba605ce3; root LICENSE SHA-256: c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4. The official Hugging Face protocol card is pinned to a32a2e99b5877c6b7ad16e9954ec08488bcc5fa2, README SHA-256 fe1c3b68d2fdd27aae56a3a339391e79b4cb08ce981d2b2e6b795ab70921b992. Its factual settings are paraphrased; the GitHub license is not asserted to license every separate publication. Complete original license bytes, source attribution and change notice are retained; no NOTICE file was present.
Revision 88b5308955ec · retrieved 2026-10-23The publisher README's Governing Download Terms explicitly designate OpenMDW-1.1, whose Model Materials definition includes associated documentation and whose grant covers copyright and database rights. This is the result-publication basis, rather than a weight-license metadata label alone. There is no root LICENSE or NOTICE in the reviewed model repository; no empty or absent file is represented as a grant. Only four NVIDIA-owned BF16 results from the Nemotron 3.5 SuperGA BF16 column are transcribed. The linked official reproduction guide identifies this exact post-trained Super VL checkpoint and documents its NeMo Gym evaluations. Base, NVFP4, MMLU-ProX, other unsupported benchmarks and all competitor columns are excluded. No benchmark questions, prompts, answers, model code, figures, source prose or dataset contents are imported. README SHA-256: 5314d0e464ebb95a4e84a29736b0297a32a0c9eea814a65d8f1791e4cb3f91fb. The complete original OpenMDW agreement is retained (SHA-256 2ab44b68365473c112f5092211a38f231cb23e50de68b75a13369adbd76a74df). Its current official HTML was freshly reviewed, retrieval SHA-256 8507e68db9e2b8f1ddcabcb04bdc0a57817e46719f2025edd33628ba4e329392. Protocol facts come from NVIDIA-NeMo/Gym at 6360a71289037f7299578665b2d1421c758c5a6e; the separately Apache-2.0-licensed recipe documentation's origin/copyright notice and root license are also retained. The reproduction guide SHA-256 is d74ed3644cd71d0c64efe3753db277abea35f6ab9edbda7556b99b7a1b95023b; exact recipe fingerprints are in the maintenance evidence.
Revision fb0ee0bc6b3f · retrieved 2026-10-23The original BF16 model repository and the separate official NVFP4 GGUF repository both contain an unqualified Apache-2.0 root LICENSE and link that license in their cards; this original repository documentation grant, rather than metadata alone, covers the selected own-model result publication. The BF16 README states SWE-bench Verified 74.2, and its embedded original publisher chart explicitly labels BF16 74.2 versus NVFP4 74.1. The NVFP4 GGUF card explicitly points to the main card for its benchmarks; that exact released variant stays separate. Only these two own-model numerical facts and original factual protocol notes are imported. No competitor scores, unsupported benchmarks, image/chart, logo, code, questions, prompts, answers or source prose are republished. BF16 README SHA-256 43244381065d603bbfc650e399bb67aef8166a01675b2446ad552d680b1058dc; result image retrieval SHA-256 329b7954c7bd5c9181ab94ab840d0b7df270780e694f65179d32742a242020c4. NVFP4 GGUF card revision 6ec43e19b565fc8d560e2d04985d7724f20673bf, README SHA-256 8a1a130252d03f22883e1aa52aa5f6f90374f85fd80477ebd7b7eaf2602dfbcc. Both complete root licenses have SHA-256 43070e2d4e532684de521b885f385d0841030efa2b1a20bafb76133a5e1379c1 and are retained with attribution/change notice. Neither reviewed root has a NOTICE file; serving-kit notices do not apply because no kit code is imported.
Revision 2a247be0c0ae · retrieved 2026-10-24The original card explicitly selects OpenMDW-1.1 as Governing Download Terms. Its Model Materials definition includes associated documentation and its copyright/database grant requires retained agreement and origin notices; these are preserved locally. No root LICENSE or NOTICE exists in this model repository, and their absence is not treated as a grant. The pinned original publication is freshly reviewed, README SHA-256 8f43225c9036e7aa14cdba81e56107e0a1d8b6d28a59b559c92a6dc8f8463ff9. Current official license HTML was freshly reviewed (retrieval SHA-256 c112016456e04fca21c293248ecdca1127778b85c0194bf537d39175ba196563); the complete pinned agreement is unchanged, SHA-256 2ab44b68365473c112f5092211a38f231cb23e50de68b75a13369adbd76a74df. Only four own-model NVFP4 numerical results are imported; BF16 observations remain unchanged and separate. MMLU-ProX and AA-origin/unsupported rows, competitor columns, benchmark questions, prompts, answers, figures, code and source prose are excluded. The separately Apache-licensed NeMo Gym protocol documentation is pinned to ac51e85139714f4ce80398037f5e5abe862cc568; its guide is byte-identical to the previously reviewed guide, SHA-256 d74ed3644cd71d0c64efe3753db277abea35f6ab9edbda7556b99b7a1b95023b. Its original copyright/origin notice and full root license remain in the local notice. The NVFP4 card explicitly says non-model settings/harnesses were held fixed relative to BF16; the BF16 serving recipe is identified as such, and an exact NVFP4 launcher/kernel selection is not invented.
Revision da5b2650a835 · retrieved 2026-10-24Fresh review of the original publisher repository: the unqualified root Apache-2.0 LICENSE covers the repository work, including the embedded original results documentation; this is not reliance on a model-weight metadata label alone. Root LICENSE SHA-256 525c465ab952779d2d360a9ff36f7f49ae018a23d80302ff839a850a77784c36, original Copyright 2026 XHToken retained verbatim in the local notice; no root NOTICE is present. README SHA-256 c483630bf7e9c22a6583d127d74ecf0e3270aea0c358e770ac5b5f4dbb82bf39; original all_benchmark.svg SHA-256 0f766e6d6acc297031b56b104cb29c78ce235d4669c86dd76392792a41871983. Only five numerical facts from the publisher-own Spark-X2.6-4B column and necessary evaluation settings are paraphrased. No source figure, source prose, benchmark questions, prompts, answers, logos or competitor columns are redistributed. The separately labeled GPQA, SWE Pro/Multilingual and unsupported benchmark results are not relabeled or imported. AIME subset, HLE no-tools subset and unreported evaluation details remain unknown.
Revision 66e3ae7bee54 · retrieved 2026-10-25Fresh review of the exact original 1.7B publisher repository and its unqualified root Apache-2.0 LICENSE, covering the accompanying original result publication. This is not reliance on weight-license metadata or an intermediary. Original root LICENSE SHA-256 525c465ab952779d2d360a9ff36f7f49ae018a23d80302ff839a850a77784c36; full original terms and Copyright 2026 XHToken remain verbatim in the shared family notice, with separate 1.7B attribution appended. No root NOTICE is present. README SHA-256 39c5a5ea1acc40728eb1f86da6b45b9f354161e23882f724ddcf95209a2a8158; original all_benchmark.svg SHA-256 653c626922bd6eeb24ca2b4be55739252635be9c77ad563850fe8103202af359. Only five numerical facts from the exact Spark-X2.6-1.7B column and necessary evaluation settings are transcribed with original explanations. The source figure, competitor/reference columns, unsupported scores, logos, source prose and benchmark questions/prompts/answers are excluded. GPQA and GPQA Diamond, SWE variants and HLE tool settings remain distinct; unreported subsets/settings remain unknown.
Revision f18c032c7675 · retrieved 2026-10-28Fresh source-specific review of the original Qwen3.8-27B repository and its unqualified root Apache-2.0 LICENSE, Copyright 2026 Alibaba Cloud, covering the accompanying original README result publication. This is not reliance on model-weight metadata or an intermediary. Exact LICENSE SHA-256 bbedc3fda3305820b977265f01b8619d87570a6739de3a5582c3464840f1e57a; original bytes retained in the local notice. No root NOTICE is present in the pinned repository file list. README SHA-256 7c24f46af1e0024e6ce9b799df6f4865f9200acf3b6f92b6530c37248cc9e821. Only two necessary scalar facts from the publisher own exact Qwen3.8-27B column and relevant protocol facts are transcribed in original wording. No competitor rows, source prose, chart/image/logo, benchmark questions, prompts or answers are reproduced. HLE subset and tool use remain unknown. The conditional Qwen3.8-Flash-Next license is a separate source and its holdout remains unchanged.
Revision 8c0b3d30f3ae · retrieved 2026-11-06The pinned original Ricursive Intelligence repository includes an unqualified root Apache-2.0 LICENSE covering the accompanying README with its own evaluation results. No separate root NOTICE or README restriction was found. README SHA-256 ed70eb22e97599dedae352eacdd532f4bcf5ade5ac23f9c69fc70a9d552e73ae; LICENSE SHA-256 58d1e17ffe5109a7ae296caafcadfdbe6a7d176f0bc4ab01e12a689b0499d8bd, preserved verbatim in the local notice. The transcription uses only scalar facts and necessary setup details for the exact checkpoint. Comparison columns, source prose, images and benchmark content are excluded.
Revision da94cc013a8f · retrieved 2026-11-07The pinned original Ricursive Intelligence repository includes an unqualified root Apache-2.0 LICENSE covering the accompanying README with its own evaluation results. No separate root NOTICE or README restriction was found. README SHA-256 b58e1c75fa3a11798d8aefa2a6da63e65898151841ba3feece4fa5e24ed8ddd7; LICENSE SHA-256 58d1e17ffe5109a7ae296caafcadfdbe6a7d176f0bc4ab01e12a689b0499d8bd, preserved verbatim in the local notice. The transcription uses only scalar facts and necessary setup details for the exact checkpoint. Comparison columns, source prose, images and benchmark content are excluded.
Revision 6187d1108a25 · retrieved 2026-11-07The exact original README expressly declares Apache-2.0 for this model card, which includes the publisher’s two GPQA Diamond results. The card attributes POCKET quantization and documentation to VIDRAFT / FINAL-Bench. No root LICENSE, NOTICE or separate copyright notice appears in the pinned POCKET tree. The linked standard Apache-2.0 text is preserved locally; it is byte-identical to the starting checkpoint’s linked LICENSE at 153ae4374b077f007d3a232a9c0053a89d705131 (SHA-256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30). Pinned README SHA-256 bbd3e1c6de6aaee88c275ce1d80b542c335ff4c9698e1f7a4e626ef3c7207955. Only exact-file scalar scores and necessary settings are transcribed; no charts, prompts, questions or report prose are copied.
Revision ea81e8e592a2 · retrieved 2026-11-11The exact publisher repository declares Apache-2.0 at repository level and ships an unqualified root LICENSE. Its separate Licensing section describes the model and training-data terms; it does not impose a restriction on reuse of the accompanying result documentation. No root NOTICE is present. Tether Data, S.A. de C.V. d.b.a. Tether AI Research is credited as the author in the original card. README SHA-256 959811cca18e73a6b0552cedb335c9d4d9a9b48301a7250155fd117f9443c774; LICENSE SHA-256 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4. The local notice preserves the license verbatim. Only the publisher's two own-checkpoint numerical facts and necessary evaluation settings are transcribed.
Revision 24561e968164 · retrieved 2026-11-12The authors' original technical report is explicitly published under Creative Commons Attribution 4.0 on arXiv. This grant covers the paper's publisher-owned evaluation publication, independently of Hugging Face weight-license metadata. Tables 34-36 report the exact post-trained 7B, 3.7B and 0.9B checkpoints. Only their own numerical results are transcribed, with original protocol notes; comparison columns, baseline/pre-training/Uno results, questions, prompts, answers, charts, and source prose are excluded. The paper describes publisher evaluations under the AA Intelligence Index v4.1.1 setup; following that protocol is not treated as importing the restricted AA result feed. No AA third-party comparison column is imported. Local attribution identifies authors, source, license, changes, immutable version and retrieval fingerprint. HTML SHA-256: 0cb64470cb83d549b5571c5de5444a55016ce2632233b4245fc08954ec391241.
Revision arXiv:2611.2 · retrieved 2026-11-19The exact original publisher repository contains an unqualified root Apache-2.0 LICENSE and explicitly licenses this release, including its accompanying evaluation snapshot, under that license. The root grant is reviewed independently of weight-license metadata. The README table and assets/benchmarks.json identify this own-model result as a publisher-supplied chart; no independent reproduction is claimed. Only Agnes AI’s own GPQA Diamond scalar is transcribed. All third-party comparison values and their AA/NVIDIA sources, chart artwork, icons, questions, prompts and answers are excluded. The assets/THIRD_PARTY_NOTICES.txt applies to LobeHub icons, which are not redistributed. The complete root license and original attribution are retained locally. README SHA-256 9e79faef1b8f3d0a58a0dabf70744f62f335b4a2513d60a30c9103ccf222408c; LICENSE SHA-256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30.
Revision 6c06a374dd7f · retrieved 2026-11-19The original publisher repository contains an unqualified root Apache-2.0 LICENSE and an explicit release license section naming OVH LLM SAS as licensor. Its associated README reports evaluations run by OVHai LLM using its own cluster and code. This root repository/documentation grant is reviewed independently of model-weight metadata. Only the exact final SFT-plus-RL checkpoint’s own GPQA Diamond and AIME 2026 scalar facts are transcribed. All comparison-model columns, averages, unsupported benchmark variants, questions, answers, prompts, charts and logos are excluded. The complete root LICENSE and NOTICE are retained verbatim with attribution. README SHA-256 e881cc8044909906cca894a0cc058d1fcec5aea6b9f003c95e4b8aeb3a706222; LICENSE SHA-256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30; NOTICE SHA-256 3d70d061de18970ccae5b6286a481754815ddaf9e872abf9004c327a96ac7527.
Revision b40f8165ee22 · retrieved 2026-11-20Fresh review of the original Tencent publication: LICENSE section 1 expressly includes documentation in the Work, independently of the model-weight license label. Section 2 permits reproduction and public display; the complete license is retained. Section 10 adds separate-license requirements for qualifying large Model-as-a-Service businesses; this benchmark fact explorer supplies no inference or fine-tuning access. Only the final Hy4 column’s own HLE-Full (with tools) scalar is transcribed with original notes. All comparison columns, preview/base/FP8 checkpoints, unsupported benchmarks, questions, prompts, answers, charts, logos and source prose are excluded. No NOTICE file is present in the pinned repository listing. README SHA-256 d6c27b4943d03ec38e2ff0f0451d00419bb8cf8ea9403cb27db49ddeff854850; LICENSE SHA-256 52755a6786f2f4b83b48cd7246aa9e34f7aac98563e4e60808ab02390799fdbd; appendix SHA-256 7ce08997cd3c67a54db99d03bd1405c8079f3ab4b8abeddb342a5679834436c2.
Revision c10c0de5b6f8 · retrieved 2026-11-21Fresh review of the original publisher technical report accompanying the unqualified root Apache-2.0 LICENSE, independently of the model-weight metadata. No separate restriction or conflicting rights notice was found in the report; OVH LLM SAS authors and publishes its own evaluations. The complete root LICENSE and NOTICE are retained with attribution. Only three necessary scalar facts from the two own Olala rows of Table 34 are transcribed, with original setup notes from Section 10, Table 54 and Appendix G. All comparison-model rows, unsupported benchmark variants, aggregate rankings, questions, prompts, answers, charts, logos, PDF and source prose are excluded. The report revision is a publication pin, not the evaluated-weight revision. PDF SHA-256 b28eb953e1690cb76f3d38d2dea44e38c895b2b3c7a3c9ca2577ea57ddd91975; LICENSE SHA-256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30; NOTICE SHA-256 3d70d061de18970ccae5b6286a481754815ddaf9e872abf9004c327a96ac7527.
Revision f781f12498ad · retrieved 2026-11-22Broad academic knowledge and reasoning.
Prompting and reasoning budgets can differ between submissions.Graduate-level science questions.
Only Diamond results are grouped here. Inspect each record for reasoning and sampling settings.Grade-school mathematical reasoning.
This older benchmark has limited coverage in the snapshot.Competition-level mathematical problems.
Full-year and exam-I results are separate. Records with an unidentified edition remain under Set unspecified; sampling and reasoning budgets can differ.Difficult questions across specialist subjects.
Choose text-only or full-set results and whether tools were used. Unidentified legacy setups are kept separate.Resolving real software issues.
This measures a model with an agent setup. Agent tools and scaffolding can substantially change the result.