How every score on this site is made

Who grades

One independent builder, with AI sessions doing the reading, each one logged. No provider has a say and none pays for anything.

From what

The maker's own page: the help article, the pricing page, the terms, the docs. Never a blog, a review, a benchmark we did not run, or a rumor.

Against what

A published test with a pass rule, graded yes, partly or no (the shelves add "not stated", counted as zero). The tests are on the page the grade appears on.

The letter

A to F is the share of a product's tests it passes: A from 84%, B from 64%, C from 46%, D from 36%, F below that, with a plus or minus marking the top or bottom of a band (A 90% and up, A+ from 97%) and partly counted as half. The full bands are on /badges. The score out of its tests is always printed beside the letter. Makers can embed theirs from /badges; that never changes a grade.

How often

Every cell carries the day it was checked. Grades are re-read when a provider moves, and the whole Scorecard is re-checked on a weekly pass; a grade older than a month is a grade to doubt.

What a correction looks like

A dated line on the page, next to the old grade, never a silent edit. Report one through the form on any question page or on Did it change?

What we did not do

Run the product ourselves, unless the row says so. A docs grade says what the maker's page promises, not what the product does on a Tuesday.

The tests themselves, with their pass rules, are on the Scorecard; the shelves name theirs at the top of each page. The data behind every grade is on Open data.

An entry earns a place on the map only when it is public — a primary source anyone can read — touches continuity, and can be verified against that source. Major entries follow a fixed RFC format, so a claim shows its work rather than asserting it:

1. 01
   A hypothesis, stated plainly.
2. 02
   The supporting evidence, with sources.
3. 03
   The conflicting evidence — what cuts against it.
4. 04
   Adjacent work, cited, not absorbed.
5. 05
   The open questions that remain.
6. 06
   Where the abstraction breaks — the named counter-examples.

Published failures are a feature: a map that shows where its own abstraction breaks is more credible than one that always works.

Competitors, namespace collisions, and overlapping-language projects get the same cited, fair treatment as anyone else. No hit pieces, no passes. Editorial decisions — what appears on the map and how it is covered — are never purchased or influenced by any commercial relationship.

Mistakes are corrected in the open, never quietly rewritten. The published record supersedes rather than deletes — the same discipline the project studies. Corrections are welcome from anyone, and the dated field-updates record is where that history lives.

Every entry in the research library carries the original publication and a direct link to it, the authors or organization, the date, the venue or identifier, what it is, this site’s reading of its main contribution, which of the five obligations it speaks to, its limits or the questions it leaves open, its publication status (peer-reviewed, preprint, workshop, draft, in force, vendor-reported), a disclosure where the source has an interest in the result, and the date this site last checked it against the primary source. Entries are verified from the primary source, never from a secondary write-up. Nothing is reproduced by this project unless the entry says so.

The section’s own thinking is labeled on every page as one of four kinds of claim: established (accepted research or standards vocabulary, with a source), emerging (a convention forming in the field), ours (this site’s proposal, citable as such), or hypothesis (ours, still to be validated, with what would settle it). Nothing marked ours is presented as an industry standard, anywhere.

Before any editorial decision, three questions are asked:

1. 01
   Does this increase or decrease trust in the map?
2. 02
   Does this make the map more useful to someone trying to understand the field?
3. 03
   Would we still do this if there were no financial upside?

Anything that fails the third question does not belong on the map. The test governs the map surface only; it is what keeps the map from sliding into content marketing one reasonable step at a time.