Metrics from the design
What each metric on the generate page counts, how it is obtained, and what the design aims for.
The two packs are measured together. A measured number comes from this run. A modeled number appears in the design only. Not in the MVP means the capability isn't built into this page.
| Metric | What it counts | Design target | Basis |
|---|---|---|---|
| Requirements covered, current and approved | Requirements, across both packs, that have evidence, whose evidence topics are current for the model version, and whose last change was approved in Git. | 100%, or a recorded decision for each gap | Measured |
| Evidence items with a named human approver from Git | Evidence items whose last commit merged through a pull request approved by a code owner who isn't the author or an agent. | 100% | Measured |
| Values in the packs that resolve from a key | Key references that resolve, checked by check_consistency. |
100% | Measured |
| Authority content found in downstream output | Matches for authority-only topic ids, titles and key values in the built downstream HTML. The same search runs on the authority pack as a control, so a result of 0 means the search can see the content. | 0 | Measured |
| Time from request to a pack ready for release | Seconds for the checks and both pack builds on the server. A person still has to review the findings. | Under 1 working day | Measured |
| People hours per pack | Not measured. The design models under 4 hours against 32 by hand. | Under 4 | Modeled |
| Packs sealed with a locked policy and a manifest hash | Packs here carry a manifest hash but are never sealed. Sealing needs a human release and a locked bucket. | 100% | Not in the MVP |
Result labels
- Meets target
- The measured value reaches the design target.
- Open
- The value is short of the target. The row's note says why, for example gaps with no recorded decision.
- Fails
- The value is worse than allowed. For the leak metric, any match is a failure.