Define the capability
Name the intended reasoning, target format, task volume, restrictions and permitted tools. Specify difficulty through construction requirements first. Model difficulty is measured only through a documented run.
METHOD AND VERIFICATION
Our engagement standard connects the intended reasoning, the information supplied, the accepted outcome and the way it is scored.
HOW TO READ THIS PAGE
This page describes the process and acceptance requirements to agree in a commission. It is not a published benchmark result or a claim that a particular model has been tested.
CONSTRUCTION AND REVIEW
Name the intended reasoning, target format, task volume, restrictions and permitted tools. Specify difficulty through construction requirements first. Model difficulty is measured only through a documented run.
Identify each item as newly commissioned or selected from the historical library. Human-authored tasks and generated instances are distinguished. A commissioned generator must identify its authored rules and instance-generation process.
Check that the instructions supply the information needed for the intended outcome. Flag ambiguity, contradictory constraints, missing assumptions and unintended alternative answers. Resolve or exclude affected items before acceptance.
For fixed-answer tasks, document the correct answer and permitted equivalent forms. For environments, document valid actions, outcomes and reward behavior. A single accepted answer does not imply a single valid solution path.
Require tests for accepted responses, incorrect responses, malformed input and boundary cases. Check that normalization does not admit wrong answers or reject valid equivalents. Version the checker with the set it grades.
Record the source category, set version and permitted use. Scope overlap checks against named accessible collections when needed. Publication checks reduce known exposure risk; they cannot establish what an undisclosed training corpus contains.
Agree acceptance tests and a defect-review window. Record exclusions, corrections and version changes so a revised set or checker can be rerun and identified accurately.
SCORING BOUNDARIES
A checker evaluates the submitted answer against the defined rule. It does not observe the model’s internal reasoning or prove a general intelligence claim.
An engine can score the same recorded state and actions repeatably. A seeded environment is reproducible; a model’s choices may still vary between runs.
An audit can inspect rubric clarity and scoring alignment. Professional-domain judgments may require the buyer’s subject specialists. Open-ended work is not automatically reducible to an answer key.
REPORT STRUCTURE
A proposed report outline, not a redacted client report. No sample task or invented result is included.
Task count, categories, set version, source category, exclusions and intended use.
Model and version, run date, prompt protocol, tool access, attempt limits, output limits and other material settings.
Checker version, acceptance rules, treatment of invalid outputs, partial credit if agreed and any review process.
Actual counts and denominators, results by category, repeated-run variation where measured and uncertainty estimates where justified.
What was tested, what was not tested, known issues and limits on comparisons with other runs.
Agreed private task files, checker, configuration and run records sufficient for the buyer’s permitted rerun. Access and retention follow the engagement.
A PRIVATE COMPANY ENGAGEMENT
The office responds with a proposed scope, deliverables and price. Work begins only after agreement.
Advanced Learning Academy LLC
Private delivery. No public task samples.