EVIDENCE CENTER
Measured results.
Private tasks.
A technical buyer can examine the task counts, run conditions, scoring decisions and limitations without receiving the prompts or answers.
RECORDED RESULTS
What the completed runs support.
Every row carries its denominator and material limitation. A result describes the recorded run. It does not become a universal claim about every system or every reasoning task.
Structured reasoning
18 private tasks. Single attempt. Answers checked by reference programs.
17 of 18
on each of two recorded runs
One network timeout in each run remained in the denominator.
Hard structured variants
18 harder variants. Single attempt. Reference-program scoring.
16 of 18
Both misses were network timeouts. Run-to-run variation was not measured.
Constrained wordplay
50 source items. Clue and answer length supplied. No crossing letters.
7 intended
13 defensible alternatives
17 genuine misses
Human review excluded 13 items that were not fairly reachable under the stated conditions.
Riddle wordplay
60-item sample. Blind, single attempt.
46 of 60
The exact system identity was not recorded in the source file, so this remains contextual evidence.
WHY TASK AUDITS MATTER
A score can be exact
and still be wrong.
A naive exact-match score made the wordplay run look like 7 correct answers out of 50.
Human review changed the meaning of that number. Thirteen responses were defensible same-length answers. Thirteen source items could not be reached fairly without crossing letters. The final fair-item denominator was 37, with 17 genuine misses.
This is the commercial value of a task-quality audit. It separates model failure from ambiguous construction, incomplete information and an acceptance rule that is too narrow.
Request an audit scopeTHE EVALUATION STANDARD
Proof begins before the run.
Fresh commissioned work follows a controlled path from specification to sealed evaluation. The tasks remain private throughout.
- SpecifyDefine the capability, answer format, exclusions and intended difficulty.
- ConstructWrite the task, accepted answer, rationale and checker as one unit.
- ReviewTwo blinded reviewers examine construction, adversarial quality and checker alignment.
- AdjudicateAn independent adjudicator resolves disagreements before any task can enter a scored set.
- SealFreeze the accepted task and its review record before testing.
- ReportPublish counts, conditions, exclusions, scoring decisions and limitations.
MEASURED INVENTORY
Large enough to license.
Defined enough to inspect.
The inventory counts describe identified historical material. They are separate from a fresh held-out evaluation and are not presented as proof that every historical item is absent from training data.
Pun and wordplay pairs
A measured subset of the clue-to-answer total.
Distinct answers
Unique answer strings after normalization.
Puzzle records
Constructed records in the measured master library.
Complete crosswords
Family Time puzzles with a grid, clues and a riddle.
LIMITATIONS
What these figures do not claim.
- The structured results are single-attempt runs. They do not establish run-to-run stability.
- Network timeouts remain visible and are not silently removed from the denominator.
- The historical library is licensable inventory. It is not described as entirely unseen by models.
- Legacy wordplay records without an exact system identifier are contextual evidence, not headline benchmark claims.
- The prompts, answers, checkers and model responses remain private.
A BUYER-REVIEWABLE ENGAGEMENT
Test the quality of a set.
Or commission a fresh one.
The office responds with a written scope, deliverables and price. A private technical dossier accompanies the agreed work.
Advanced Learning Academy LLC
Private delivery. No public task samples.