Verification architecture

Verification adapters

Flows layers human Verify Gates with a small set of browser-side probes. Probes are labeled by method in the Build Evidence Report. They are not a substitute for a security audit or full E2E suite.

Browse routes Independent validation case study

Honesty rules

A checked box alone is weak proof — attach an evidence note.
Automated probes are labeled automated-* in exports. Manual stays manual.
Opaque CORS responses are ok: null, not success theater.
Route gate completion ≠ production-ready claim.
Definition of Done is never auto-checked by step completion.
Unauthenticated authz pass means one path/method returned 401/403 — not a two-account ownership matrix. Two-account results disclose their source (UI visibility ≠ cross-account API ≠ RLS). not that every role is correct.

Adapter catalog

Status is product truth: shipped means available in the run UI today.

Deploy URL probe

automated-probe · Run header · Probe next to Deployment URL

Shipped

Does: GET the deployment base URL from the browser. Reports HTTP status when CORS allows it.
Pass criteria: HTTP 2xx when readable
Limits: CORS often yields an opaque response (ok: null). Not TLS, uptime, or content validation. Not a production health audit.

API path probe

automated-api-probe · Run header · path input + Probe API

Shipped

Does: GET `deployBase + path` (e.g. `/api/health`). Stores recent path results on the run.
Pass criteria: HTTP 2xx when readable
Limits: Same CORS limits as deploy probe. Does not parse response body or assert JSON shape.

Authz probe

automated-authz-probe · Run header · protected path + GET/POST + Probe authz

Shipped

Does: Sends an unauthenticated request (`credentials: omit`) to a path you expect to be protected.
Pass criteria: HTTP 401 or 403 (unauth correctly rejected)
Limits: Fail = 2xx (unauth allowed). 404 = path wrong, not a security pass. Opaque CORS = inconclusive. Not a full role matrix, CSRF, session fixation, or privilege-escalation suite.

Browser persistence check

automated-persistence-check · Run header · Check browser persistence

Shipped

Does: Write / read / delete a token in `localStorage` in this browser.
Pass criteria: Read matches write
Limits: Proves browser storage only — not your app database, server sessions, or multi-device sync.

App persistence probe (create→read→refresh)

automated-app-persistence-probe · Run header · create path + read path + Probe app persistence

Shipped

Does: POST JSON with a unique `{{token}}`, GET read, then refresh-read again. Detects false success when create claims success but token is missing. Badge “persistence passed” only when create→read→refresh all find the token.
Pass criteria: Token found on read and refresh; falseSuccessDetected is not true
Limits: Needs CORS that allows the browser to read the response. Auth-required endpoints return honest 401/403 notes. Not multi-tenant isolation, new browser session, cascade delete, or concurrent write testing. Single-session only unless multi-session adapter is run.

Two-account authorization matrix

automated-two-account-authz-probe · Run · More verification tools · Run two-account matrix evaluator

Shipped

Does: Evaluates Account A vs B allow/deny matrix (A reads/changes A allowed; B reads/changes/deletes A denied; participant admin denied; unauthenticated mutation denied). Discloses evidence source trust level.
Pass criteria: All decisive cells match expected allow/deny
Limits: Live Authorization headers for A/B required for remote HTTP matrix; evaluator can validate outcomes. UI hide/redirect alone is weaker (trust levels 1–2). Not full RLS audit — test SELECT/INSERT/UPDATE/DELETE separately for Supabase.

Generated edge-case tests

generated-from-signals · Run · More verification tools · Generated edge-case tests

Shipped

Does: From route/app signals (forms, auth, roles, DB lists), generates plain-language tests with executable assertions. User reviews/approves — does not invent the security checklist.
Pass criteria: N/A — generation; execution is separate probes/harness
Limits: Detection is heuristic from route text. Not a substitute for a security audit.

Project binding + unrelated-repo warning

binding-validation · Run header · Project fields

Shipped

Does: Tracks goal, repository, start commit, builder, model, environment, stack, deploy target, route hash. Warns when a soccer/team goal is bound to an unrelated repo (e.g. Sitemapper). Override requires a recorded reason.
Pass criteria: Binding completeness; no blocking mismatch (or override recorded)
Limits: Heuristic mismatch detection. Soft gate in UI — strong gate available via canStartStepOne().

Trust tiers on checks

trust-model · Evidence export schema v2 · completion disclosure banner

Shipped

Does: Classifies each check: self-attested → observed → automated → cross-account → external → repeatedly verified. Manual checkboxes cannot be presented as high-trust automated proof. Route may complete with banner: independent verification incomplete.
Pass criteria: N/A — honesty model
Limits: Does not auto-upgrade manual checks.

Content / body text probe

automated-content-probe · Run header · path + expected text + Probe content

Shipped

Does: GET `deployBase + path` and assert a substring appears in the response body when CORS allows reading it.
Pass criteria: Expected text found in readable body
Limits: Not a DOM, click, or visual regression test. Opaque CORS = inconclusive. Case-sensitive substring match only.

External Playwright smoke script

flows-external-playwright-smoke · Download `/adapters/playwright-smoke.mjs` from run header or verification docs

Shipped (external)

Does: Optional Node + Playwright script you run locally against a base URL; prints JSON pass/fail for visible body text.
Pass criteria: Exit 0 when text is visible after navigation
Limits: Not embedded in the Flows web app. Requires local Playwright install. One URL/text check — not a full E2E suite. Not multi-tenant or role testing.

Evidence strength score

heuristic · Run header · Evidence strength · also in evidence export

Shipped

Does: Scores 0–100 from gate checks, evidence notes, DoD, probes, and timeline density (thin / moderate / strong).
Pass criteria: N/A — awareness only
Limits: Not a certification. Strong ≠ production-ready. Does not replace independent test suites.

Continuous build recording

local timeline · Run header · Build timeline · full log in evidence MD/JSON

Shipped

Does: Appends operator events (open run, copy prompt/repair, checks, step complete, probes, DoD, exports) to a local timeline (max 200).
Pass criteria: N/A — audit trail density
Limits: Browser localStorage only — not remote CI, not multi-device, not tamper-proof. Cleared when the run is reset. Not a substitute for git history or server logs.

Portable run pack (team handoff)

file export/import · Run · Build Evidence Report menu · Export/Import run pack; also Runs dashboard import

Shipped

Does: Exports full local progress (gates, notes, probes, timeline) as JSON; import replaces progress for the same route slug.
Pass criteria: N/A — handoff tool
Limits: Not real-time collaboration, not access control, not multiplayer. Receiving browser must have the same route (library or custom). Custom generated routes only import if that draft already exists locally.

Verify Gate + evidence notes

manual (default) · Each step · checklist + per-check evidence

Shipped

Does: Human attests each check. Optional free-text evidence note. Complete stays soft-locked without notes (warns).
Pass criteria: All boxes checked; stronger if notes attached
Limits: Checkbox alone is weak proof (self-attested). AI does not grade its own work. Manual is never high-trust automated proof.

Definition of Done

manual · End of run · DoD section

Shipped

Does: Route-level residual checklist, separate from step gates.
Pass criteria: All DoD items checked by operator
Limits: Never auto-checked when steps complete. Route gates ≠ production-ready claim.

Multi-session / concurrency adapter

—

Not shipped (P1)

Does: Would test two tabs, two accounts concurrent edit/delete, lost updates.
Pass criteria: —
Limits: Evidence export currently labels single-session tested vs multi-session not claimed.

In-product Playwright DOM attach

—

Not shipped

Does: Would run headless checks inside the Flows product and auto-fill Verify Gates.
Pass criteria: —
Limits: Not planned as a browser-side embed. Use the external smoke script or your own CI. Product surface remains Cloudflare Flows (not third-party SPA hosts).

Evidence pack

From any run, export Markdown, JSON (schemaVersion 1), clipboard, or ZIP. Exports include project URLs, probe results, step states, check evidence, and Definition of Done — with an explicit note that gate completion is not independent production verification.

Markdown / JSON / ZIP — full Build Evidence Report
Export history — local list on Runs
Independent benchmark — youth soccer management (59 plan checks passed · Tier 1 self-checked · not independent reliability)

How to use probes in a run

Set Deployment URL (and optional repository / goal) in the run header.
Probe deploy, then a health path, then a protected path with authz (expect 401/403).
For APIs that allow CORS reads: run app persistence (create path → read path) and confirm the token.
Content-probe a public page for expected UI copy when CORS allows body reads.
Optional: download playwright-smoke.mjs and run DOM checks on your machine.
Stamp probe summaries into step notes when you want them in the evidence trail.
Run browser persistence only if you care about localStorage in this browser.
Still complete Verify Gates with evidence notes — probes do not auto-check boxes.
Watch Evidence strength as a heuristic; export the pack when the route (and DoD) is finished.