Ryvx

Docs / Verification

Verification

PoC or it didn't happen.

This is the actual differentiator: not that Ryvx is more autonomous than the next tool, but that what it reports is backed by something real. A finding isn't an LLM's opinion that a vulnerability probably exists — it's a working exploit, a deterministic score, and a trail proving both.

The PoC gate

create_finding hard-rejects anything without a working poc_script_code— no unconfirmed suspicions get filed. This isn't a soft prompt instruction; it's a validation gate the tool enforces before a finding is ever recorded, and it has already caught a live hallucination bug from a local model that tried to report a finding without one.

Deterministic severity, not a self-reported score

Severity is a CVSS 3.1 base score computed by a hand-rolled formula against the finding's reported vector — not a number the LLM makes up about its own finding. Deduplication tries an LLM comparison against every existing finding first, then falls back automatically to a difflib similarity check on any failure, so dedup only ever gets smarter, never blocks a scan outright if the LLM call itself fails.

The evidence trail

Every tool call any agent makes, on every target, is appended to an audit log — agent, tool, redacted arguments, result preview, model, and timestamp — a full chain-of-custody trail for defending a scan's actions after the fact.

A finding's own lifecycle is tracked too: ryvx advance moves a finding forward one remediation step at a time (DISCOVERED through INVESTIGATING, EVIDENCE_COLLECTED, and on to VERIFIED_FIXED or a reopen back to DISCOVERED), recording an evidence bundle at each transition. A screenshot is captured automatically the moment a finding with an http(s):// target is filed, so a bundle exists even if nobody ever runs advance by hand.

Differential verification: where it stands

Ryvx also has a differential-verification mechanism aimed at reducing false positives further — comparing behavior across variants of a request to confirm something like an IDOR is real rather than coincidental. Being direct about where this one actually is: it still has no verification rate worth quoting. Three known bugs in it were fixed on 2026-08-12 — target-URL selection now skips citation domains and prefers a path-bearing URL over a bare root, a scope allowlist is mandatory rather than optional, and an IDOR check compares JSON leaf values across a three-way differential instead of raw response bytes — but it has never been run against a real multi-run suite. There is a working mechanism here and, honestly, no coverage number yet.

Holding the tool to its own standard

"PoC or it didn't happen" applies to what this project says about itself, too. Benchmark recall numbers against DVWA and Juice Shop were published, then withdrawn a few hours later once an audit found the runs behind them had been cut off before their agents finished investigating — and the credited found/missed lists didn't even match what those runs had actually produced. The fix wasn't just quieter publishing: the benchmark harness now refuses to score a run it can detect was cut short, checking a cancelled_agents field before scoring anything at all. Nothing replaces the withdrawn numbers until an untruncated run exists. See the full account for what that audit found.