What Flows just did
Based on your request to build Jailbreak Regression Suite, Flows created 3 implementation stages and 9 checks.
You asked for: Build a jailbreak regression suite: a corpus of known jailbreaks with expected refusals, automated evaluation, and CI gating on every model/prompt change.. Flows generated 3 stages because the application requires A corpus of jailbreak attempts with the expected safe behavior for each; Automated evaluation classifying agent responses as refused/complied; Runs pinned to model version and prompt version for attribution. Each stage tells your AI builder what to do, tests whether it actually worked, and gives a repair instruction when it fails. These checks are designed to catch: Fixing a jailbreak once with no test, so the next model update reopens it; Naive keyword grading that counts a hedged compliance as a refusal; Suite tied to one model, breaking silently on provider upgrades.
Example failure
The form reported success, but no database record was created.
What Flows does
Flows detects the failure, generates a repair instruction, reruns the test, and stores the passing evidence.
Stages — Plan → Build → Test → Fix → Prove → Ship
3 stages · 9 checks · progressive disclosure — open a stage for details
Copy to external builder
Copies a prompt for Claude/Cursor/etc. Records prompt_copied only — not execution, commits, or checks.
Execute through Oort
Requires Sign in with Oort + a provider connection (BYOK). Keys stay on Oort. A model name in the dropdown is not the same as a live connection.
Next
Check off: Attacks scoped to real safety boundaries
Add a proof note on each check — a checkbox alone is weak proof.
Probes & advanced tools are under “More verification tools”. Experience level: Settings.
Project (saved with proof)
Project binding: Unbound — complete project binding before trusting this run · missing repositoryUrl, repositoryStartCommit, environment, stack
More verification tools (probes, timeline, adapters)Show
App persistence (create → read → assert token)
Body template: {"probe":"{{token}}","title":"flows-probe-{{token}}"}. Needs CORS-readable public create/list endpoints (or inconclusive). Badge only when create→read→refresh finds the token (detects false success).
Two-account authorization (User B cannot see/change User A)
Authorization evidence source: Cross-account ownership test (trust level 5/6). These are not equivalent. Unauthenticated probe alone is weaker (source: unauthenticated API probe). Tokens stay in this browser session only — not uploaded to Flows servers.
Generated edge-case tests (review / approve)
- api_probe Submitting the form empty is rejected with a clear error
- api_probe Each required field can be omitted and fails validation
- api_probe Malformed email is rejected
- api_probe Malformed phone is rejected when phone is collected
- api_probe Duplicate submission does not create two records
- api_probe Double-click submit creates only one record
- false_success_probe Failed request does not show success
- browser_automation Refresh during submission does not leave corrupt state
- api_probe Oversized input is rejected
- api_probe Unsafe strings do not execute or break storage
- api_probe Create then refresh still shows the record
- api_probe Edit then refresh keeps the edit
- api_probe Delete then refresh removes the record
- api_probe Nonexistent record ID fails cleanly
- cross_account_probe Cross-user record identifier is rejected
Generated from this route’s signals — you approve; you do not invent security tests.
Content probe (GET body contains text)
In-product content probe reads response text when CORS allows — not full DOM/click automation. Playwright script is optional and external.
Proof strength
thin · 0/100
Heuristic from notes + probes + gates — not a production certification.
Verification adapters
- ● Deploy URL probe · idle
- ● API path probe · idle
- ● Authz probe (unauth → 401/403) · idle
- ● Browser persistence · idle
- ● App persistence (create→read) · idle
- ● Content / body text probe · idle
- ○ In-product Playwright DOM · not embedded (download external script)
Filled circles = available here. Empty = not shipped. Probes are not a production audit. Adapter docs
Step 1 of 3
Not startedAssemble the jailbreak corpus
Collect the attacks that matter for your agent's boundaries.
Flows keeps plan, checks, failures, and next steps connected
Your AI tool writes or changes the code
Repository grounds the plan only after inspect
Checks always carry a validation tier
How this plan was created
Plan from project details
Flows creates steps using the goal, requirements, stack, platform, and details you provide.
- Project description
- Selected template / route
- User-entered stack
- Constraints and notes
- Jailbreak Regression Suite
No repository inspection implied. Connect and inspect a codebase to ground steps in real files.
Codebase access
Checking GitHub…
Or paste a repository URL manually
Use these instructions in Claude, ChatGPT, Cursor, Emergent, or another builder.
Build the jailbreak corpus. Sources: jailbreaks found in your own red-teaming, publicly documented technique families (persona/role-play, hypothetical framing, token smuggling, many-shot priming, refusal-suppression phrasing), and any real attempts seen in usage. Scope to YOUR agent's actual safety boundaries - what should it refuse or handle carefully given its capabilities (destructive actions, sensitive data, out-of-scope requests)? For each entry record: id, category, the attack (single or multi-turn), and the EXPECTED SAFE BEHAVIOR (hard refusal, safe partial help, or redirect - be specific about what safe looks like for this case). Aim for breadth across categories over volume within one.
Project context
Copies the project goal, technical details, current step, previous progress, and checks so your AI tool understands what it is working on.
Expected after this step
A categorized jailbreak corpus with per-case expected behavior.
Should not happen
- ✕Fixing a jailbreak once with no test, so the next model update reopens it
- ✕Naive keyword grading that counts a hedged compliance as a refusal
- ✕Suite tied to one model, breaking silently on provider upgrades
- ✕No process, so found jailbreaks live in a chat log, not the suite
Verify gate — prove this step works before continuing
Do not move on until every check is true. Add proof notes — a checked box alone is weak proof. Checks are manual by default; deploy/API probes live under “More verification tools” and do not auto-check boxes.
Do not continue if…
- !Fixing a jailbreak once with no test, so the next model update reopens it
- !Naive keyword grading that counts a hedged compliance as a refusal
- !Suite tied to one model, breaking silently on provider upgrades
- !No process, so found jailbreaks live in a chat log, not the suite
Repair path — if this step fails
Use the Repair Prompt when a verify gate fails. Loop: Detected → Diagnosed → Repair issued → Changed → Retested → Resolved
Show repair prompt textShow
The corpus tests generic model safety irrelevant to my agent. Refocus on boundaries that matter here - if the agent has no capability an attack would abuse, that attack is noise; prioritize jailbreaks that reach real actions or data.
Your notes for this step
Definition of Done
Final route-level requirements. Manual checks — separate from per-step verify gates. Completing step gates does not auto-check these. Route completion is not a production-ready claim.