# Test one proposition from the paper

These are proposed investigations, not validated controls. Choose one question at a time so a result can be interpreted.

**Proposition number / local hypothesis / owner:** [fill in]

**Task / population / baseline / intervention / comparison:** [fill in]

**Criteria and decision thresholds set before testing:** [fill in]

| Paper proposition | Comparison | Measure and result that would weaken the local hypothesis |
| --- | --- | --- |
| 1. Lower latency helps where waiting interrupts useful work. | Matched tasks at equivalent quality, with latency as isolated as practical. | Task completion and engagement do not improve despite a large latency change. Record other system changes. |
| 2. Better models can reduce downstream effort. | Models on the same evidence, tools, and quality standard. | Higher task capability does not reduce review/recovery or improve accepted work. |
| 3. Effective context use matters beyond nominal capacity. | Full, curated, and retrieved evidence with equivalent access to needed facts. | Full context consistently performs as well or better at equal total cost, weakening the case for added context management in this setting. |
| 4. Traceable sources reduce checking effort. | Candidate work with and without usable source references, independently fact-checked. | References do not reduce review effort or missed errors, or misleading references worsen results. |
| 5. Assistance can preserve learning. | Task-oriented assistance versus explanation and practice. | No benefit on an announced delayed transfer task, or unacceptable extra cost. |
| 6. Unchecked generated summaries can propagate errors. | Fictional seeded-error packet across repeated summaries, with and without independent source checks. | Errors are reliably corrected without the proposed additional controls. This does not test model-weight degradation. |
| 7. Better evidence can improve acceptance capacity. | Same task mix and quality standard, different evidence presentation. | No reduction in review effort or defects. Check whether the constraint lies elsewhere. |
| 8. Model changes deserve workflow reassessment. | Versioned task set before and after documented model/alias changes. | Repeated changes leave relevant behavior stable, supporting lighter reassessment for the tested scope. |
| 9. Understanding helps recovery. | Comparable unfamiliar failures and handoffs, with understanding measured separately. | Recovery does not differ with demonstrated understanding. Experience, documentation, and assistance may confound the comparison. |

## Make a result interpretable

- Define the primary outcome, material errors, and follow-up period before running.
- Record sample selection and experience; match or randomize where practical.
- Preserve configuration, prompt, evidence, and evaluator versions.
- Include failures, blocked cases, retries, and missing follow-up.
- Use a reference independent of the candidate where possible. Agreement between two model outputs is not verification.
- Distinguish no observed benefit, insufficient measurement, and evidence against the hypothesis.

**Observed result / uncertainty / coverage / confounders:** [fill in]

**Does this support, weaken, or leave the hypothesis unresolved? Why?** [fill in]

**Next decision / authorized scope / owner / review date:** [fill in]

Results support only the tested scope. A small pilot does not validate all nine propositions or establish an industry-wide conclusion.