Read the claim before the badge
A finding is a specific observation about the experience Flock reviewed. Its severity tells you how much it could matter. Its evidence status tells you what Flock could support. Those are different judgments.
Start with the claim, the affected page or step, and the evidence. Then decide whether to investigate, fix, or defer it. A convincing explanation can still be wrong about the cause.
Severity: how much does this matter?
Persona findings use five severity levels. Lower numbers mean higher priority:
- 0 — Critical
- 1 — Major
- 2 — Moderate
- 3 — Minor
- 4 — Cosmetic
The model assigns severity while reviewing the experience. Its instructions anchor Critical to release- or task-blocking consequences and Cosmetic to low-priority polish. The intermediate labels help order findings; they are not measurements of revenue loss, affected customers, or the likelihood of an incident.
There is no calibrated numerical formula for severity. A Critical label does not make an observation verified, and a Cosmetic label does not make it false. Severity also does not apply a fixed deduction to clarity, trust, frustration, or the other review scores.
Use severity to prioritize inspection. Use the evidence and your product context to decide what to do.
Verified: a specific claim has supporting proof
For reviews using Flock’s current material-assertion verification, Verified means the specific claim and the supporting details it depends on passed the required checks. The evidence must support the claim, the required supporting details must be accounted for, and none of those essential details can be contradicted.
The scope matters. Evidence that a particular control lacks an accessible name supports that claim about that control in the captured state. It does not establish that the whole flow is inaccessible, that every customer encounters it, or that the suggested code change will fix it.
A screenshot’s presence alone does not make every statement about it verified. Read the verification explanation and inspect the evidence relevant to the claim. Suspected causes and suggested repairs still need engineering judgment and testing.
Directional: a lead to investigate
Directional means the observation does not have sufficient support to present it as a verified finding. Reasons can include missing screenshot evidence, unavailable or unsuccessful verification, a confidence requirement that was not met, or an observation outside the reportable journey. In current material verification, an unsupported claim remains directional; contradictory evidence excludes it from verified findings.
Directional observations can be useful. An unclear next step, for example, may deserve a closer look even when Flock cannot prove the claimed consequence. Reproduce the situation and gather the missing evidence before treating the proposed fix as necessary.
Do not read Directional as “minor.” Evidence status and severity are separate. Directional observations do not block the GitHub gate as verified findings.
Older results can use unverified in their raw output. The dashboard may also show Legacy evidence when current claim proof is unavailable. Treat those historical observations as something to reproduce and validate; a completed historical run does not supply missing proof.
Confidence: whose confidence, in what?
A finding can carry model confidence as well as evidence-related confidence. Model confidence is the model’s estimate of its own judgment. It has not been calibrated here into a measured probability that the finding is correct.
For current material verification, evidence confidence of 1 records that the required proof was supported. It is a complete-proof signal, not “100% certain in the real world.” A directional material claim has no such evidence-confidence value.
Older review formats use other eligibility rules, including confidence thresholds and agreement across personas. There is no universal rule that a displayed confidence of 70% makes a finding Verified. Read the status and explanation attached to the actual result.
Finding confidence also does not tell you how certain a clarity or trust rating is. Impression ratings are a separate part of the review.
Shared, unique, and affected personas
A Shared verified finding groups matching, scoped findings from at least two distinct personas. A Unique verified finding does not meet that shared grouping condition. Repeated mentions from the same persona do not count as additional affected personas.
These labels describe the synthetic review’s coverage. They do not measure prevalence among your customers. “Three personas affected” does not mean three customers, three independent human studies, or a known percentage of your market. Directional observations do not receive the same Shared or Unique classification.
The review summary can group observations from individual runs. Do not add the summary’s findings to the underlying run totals as though they were separate issues.
Evidence and technical findings
Depending on the review and what was captured, evidence can include screenshots, page or step references, DOM information, console errors, network records, or traces. Inspect the available artifact rather than assuming every kind was collected for every finding.
Technical findings describe captured browser or runtime signals. A console error or failed request is useful evidence, but does not automatically prove the cause of a persona’s frustration or establish a customer-facing failure. Correlate it with the relevant action and observed result.
If an image, artifact, or verification result is missing, that is a coverage limit. It is not evidence that the issue did not occur. A partial review can still contain useful individual results; it should also tell you what remains unread, failed, or unavailable.
Addressing a finding is not verifying a fix
Address records a fix attempt and keeps the finding pending until rerun verification. For a directional observation, begin by reproducing and validating the claim. A repair prompt is guidance for that work, not proof that the proposed change is correct.
Dismissing or deferring a finding records a triage decision. It does not change what the original evidence proved. Keep the reason with the decision so the next person can understand it.
After a change, check the relevant behavior again under comparable conditions. Keep the target, persona, goal, and saved sign-in consistent where practical. A finding that was not observed in a later review is narrower evidence than a guarantee that it can never recur.
What a completed review or passing gate means
Execution status, analysis readiness, and task success answer different questions. A run can finish while its summary is still pending. One persona can fail to reach a goal even when the review completes. An unknown outcome remains unknown.
Likewise, zero verified findings is not a clean bill of health. Check which pages and personas were covered, what evidence is available, and whether analysis is complete.
The GitHub Preview Gate evaluates the configured comparison and policy. A first successful PR review can establish its baseline; it is not evidence of improvement against an earlier run. Read the check’s explanation, including blockers, warnings, and incomplete or unavailable comparison state. See the GitHub App guide for setup and gate behavior.
A useful review explanation
Whether you are reading the dashboard or asking your agent, look for four things: the specific claim, its priority, the evidence that supports it, and what remains uncertain. The next step should follow from those facts: validate an observation, investigate a cause, implement a scoped repair, or recover missing coverage.
Your connected agent can search Flock’s public documentation for these definitions, then retrieve the actual review with your workspace permissions. Documentation explains the terms; the review’s own evidence determines what can be said about your result.