Interests & possibilities · Project walkthrough
Prepared · Not yet runResearch Evaluation Pilot
A prepared set of synthetic research tasks for examining evidence quality, uncertainty, and human review. The evaluation has not run yet.
Practising research judgment
A convincing answer still needs evidence. This pilot is designed to examine how a response handles source support, conflicting information, uncertainty, and the point where a person needs to review the work.
Start with a fixed packet
Each synthetic task supplies an invented source packet. The focus is judgment within that packet, rather than finding live information on the web.
Examine the response
The prepared rubric asks whether claims follow from the evidence, uncertainty is visible, and the response respects the task boundaries.
Record human review
The planned review compares the response with the supplied evidence and records gaps and limitations. Preparation is complete; evaluation results are still to come.
What would you check?
New fictional example · Not a pilot case or result
“Seven of twelve people preferred layout A in a mock exercise.”
A proposed summary says: “Layout A improves productivity.” Does the supplied statement support that claim?
Read a possible review
The statement describes a preference in a small mock exercise. It supplies no productivity measure, so the proposed claim goes beyond the evidence. A careful response would keep that distinction visible and ask for more evidence.
Where it stands
Prepared; evaluation has not run
The synthetic task pack and review rubric are prepared. No participant, model, or workforce baseline has run, so there are no scores, findings, or comparisons to report.
The planned evaluation concerns offline research judgment. It cannot establish live retrieval quality, production security, or operational readiness.