Keep the evaluation held out
Your evaluation set is built for you, delivered privately and not posted on this site.
FOR MODEL LABS
Public benchmarks can become familiar. A private, newly commissioned set gives your team a different basis for measuring reasoning.
Wordplay, constraint and multi-step deduction demand more than recalling a fact. Commission the task types your evaluation is missing.
We can run your model against a fresh set and report the result. A scored run is separately scoped within the paid pilot.
Your evaluation set is built for you, delivered privately and not posted on this site.
Delivery includes JSON, the answer key and a Python checker. Your team can rerun the grading without an editor in the loop.
Commission fresh sets on your release calendar, with the scope and refresh schedule agreed in advance.
A written agreement names the material, field of use, term and exclusivity. Training and evaluation rights are specified separately.
YOUR ENGAGEMENT OPTIONS
A held-out set and a scored report, with refreshes to match your program.
Selected verified library material, licensed for the use your team requires.
Original tasks constructed to your rubric, with answers and automated verification.
PRIVATE ENGAGEMENTS
The office will reply with a written scope and a price.