{"rid":"compute/mlperf","card":{"rid":"compute/mlperf","eid":"mlcommons","topic":"compute","slug":"mlperf","provider":"MLPerf","url":"https://mlcommons.org/benchmarks/","domain":"mlcommons.org","category":"cloud VM","tier":null,"verdict":"fails-identity","vorder":4,"grp":"rejected","confidence":"documented","binding":"write-gated","identity_kind":"institution","identity_basis":"stated","needs_card":0,"needs_phone":0,"needs_id":0,"needs_institution":1,"country":null,"crypto":null,"interfaces":null,"egress_metered":null,"per_object_bytes":null,"retention_days":null,"verified_at":"2026-10-22T00:00:00.000Z","verified_basis":"live","checked_on":"2026-10-22","stale_on":"2026-11-21","n_evidence":3,"usd_month":null,"usd_per_tb_month":null,"summary":"MLPerf awards no compute at all. The programme is a measurement, and the submitter supplies the system, quoted: 'The working group defines the AI model to ru...","duplicate_of":null,"n_unknown":10},"record":{"topic":"compute","slug":"mlperf","provider":"MLPerf","url":"https://mlcommons.org/benchmarks/","category":"cloud VM","free_tier":{"stated":"MLPerf awards no compute at all. The programme is a measurement, and the submitter supplies the system, quoted: 'The working group defines the AI model to run, the data set against which it gets run, sets rules on what changes to the model are allowed, and measures how fast a given hardware runs the model.' The gate is membership, quoted: 'Membership is required for most benchmark working groups (e.g., Training, Inference, Mobile). There are some public benchmark working groups that have no access requirements where non-members may submit to the benchmark by first signing a Non-member Test Agreement.' The submission requirements list, quoted: 'Join an appropriate MLCommons Working Group', 'A signed Contributor License Agreement', 'Membership in MLCommons OR a signed Non-member Test Agreement', 'A signed trademark license agreement'.","unit":"none","metered_on":"wall-clock","source_url":"https://mlcommons.org/benchmarks/","read_on":"2026-10-22","notes":"none"},"per_object_cap":{"stated":"not applicable","bytes":null,"source_url":"https://mlcommons.org/benchmarks/","notes":null},"retention":{"stated":"not applicable","days":null,"source_url":"https://mlcommons.org/benchmarks/","notes":null},"egress":{"terms":"not stated","metered":false,"free_allowance":null,"price_per_tb":null,"source_url":"https://mlcommons.org/benchmarks/"},"interface":["unknown"],"interface_detail":"inferred from the tooling the provider documents; not probed","tooling":"not stated","chaining":"Nil. The useful product of the record is the exclusion: a reader who sees MLPerf results quoted as free capacity should be told that the numbers are vendor submissions, and that no MLCommons programme pays for the machine under test.","limits":[{"kind":"write-gated","value":"two gates stack: the Training and Inference working groups require organisational membership, and even a non-member submission needs a signed test agreement and a trademark licence before a result can be published, so the constraint that fails first is the membership, not the hardware.","source":"https://mlcommons.org/benchmarks/"}],"limits_note":"two gates stack: the Training and Inference working groups require organisational membership, and even a non-member submission needs a signed test agreement and a trademark licence before a result can be published, so the constraint that fails first is the membership, not the hardware.","binding":"write-gated","binding_note":"two gates stack: the Training and Inference working groups require organisational membership, and even a non-member submission needs a signed test agreement and a trademark licence before a result can be published, so the constraint that fails first is the membership, not the hardware.","identity_cost":{"kind":"institution","detail":null,"country":null},"reliability_note":"no reliability event recorded","confidence":"documented","evidence":[{"url":"https://mlcommons.org/benchmarks/","date":"2026-10-22","proved":"'Membership is required for most benchmark working groups (e.g., Training, Inference, Mobile)'; 'There are some public benchmark working groups that have no access requirements where non-members may submit to the benchmark by first signing a Non-member Test Agreement'; 'Membership in MLCommons OR a signed Non-member Test Agreement'; 'A signed trademark license agreement'; page 200"},{"url":"https://mlcommons.org/benchmarks/inference-datacenter/","date":"2026-10-22","proved":"the suite page describes measurement only ('measures how fast systems can process inputs and produce results') and names no grant, credit, allowance or hardware pool; page 200"},{"url":"https://mlcommons.org/about-us/policies/","date":"2026-10-22","proved":"the policies page offers three agreements, 'Membership Agreement', 'Membership and Affiliate FAQ', 'Non-member Test Agreement', which is the only route for an unaffiliated submitter; page 200"}],"verdict":"fails-identity","verified_at":"2026-10-22T00:00:00.000Z","notes":"This is a refutation rather than a discovery, and it belongs because the brief lists MLPerf as a compute-awarding programme and it is not one. Three readings of the membership sentence were considered: (a) membership means an individual can join, (b) membership means a company pays dues, (c) non-members are fine through the test agreement. The page's own 'Membership and Affiliate' FAQ and the phrase 'if you do not think your organization would be a good fit as a Member' in the GitHub route point at (b), and (c) is true but ends in a signed agreement that a solo entrant can sign. So the honest verdict is fails-identity for the main suites and a narrow non-member path for the public groups; the hardware cost stands either way.","extras":{"wave2":{"class":"benchmark-programme","free_tier_kind":"none","shape":{"vcpu":null,"ram_gib":null,"gpu_class":null,"gpu_count":null}}}},"claims":[{"path":"$.free_tier.stated","kind":"free-quota","num":null,"unit":null,"per":null,"currency":null,"usd":null,"fx_date":null,"parse":"unparsed"},{"path":"$.limits[0].value","kind":"limit:write-gated","num":null,"unit":null,"per":null,"currency":null,"usd":null,"fx_date":null,"parse":"unparsed"}],"unknown":[["$.egress.metered","egress-unknown"],["$.egress.terms","not-stated"],["$.identity_cost.detail","process"],["$.interface","interface-unknown"],["$.per_object_cap.notes","process"],["$.per_object_cap.stated","not-applicable"],["$.reliability_note","none-observed"],["$.retention.notes","process"],["$.retention.stated","not-applicable"],["$.tooling","not-stated"]],"history":[],"drift":[]}