A reproducible framework for security verification using multiple AI models as adversarial reviewers, plus systematic model-level stress testing for exhaustive architectural analysis, plus structured upstream tracing for severity triage, plus an explicit contract for what the reviewers are given and what a confirmed finding must prove.
MMASA provides the first empirical demonstration that cross-provider AI auditing identifies 2.5x more security issues than any single-model assessment. The methodology applies Reason's Swiss cheese model to AI code review: by using models from different providers with misaligned failure modes, vulnerabilities that slip through one model's analysis are caught by another.
Version 3.1 added Adversarial Model Stress Testing (AMST), a systematic methodology for exhaustive security analysis of entire application architectures.
Version 4.0 added Upstream Reachability Verification (URV), a triage technique applied to every finding from the prior four techniques. URV traces upstream callers and parallel paths to determine actual severity (active vs latent) and to surface adjacent bugs the original audit missed.
Version 4.2 carries the Semantic Deception category into the canonical blind spot taxonomy. It was documented in the methodology paper in V3.0 and never catalogued in the taxonomy, so the single most impactful blind spot of the original campaign was missing from the list practitioners are told to review before every audit. Now Category 13, items #43 to #45.
Version 4.1 added the input contract and the verification standard. Every prior version specified what to ask the models and left unstated what to give them and what a confirmed finding must prove. V4.1 makes the Cycle 2 context packet a required input, moves review from increment scope to branch scope, requires a reproduction trace before any finding is marked confirmed, and adds state-machine interrogation as a lens distinct from attack simulation.
Key Contributions:
- Four-pass assessment methodology (Pattern Scan, Attack Surface, Architecture Review, Reconciliation)
- Adversarial Model Stress Testing (AMST), exhaustive model enumeration + domain-grouped adversarial analysis
- Upstream Reachability Verification (URV) for severity triage and adjacent-bug discovery
- A required input contract for the adversarial pass, a reproduction-trace standard for confirmation, and a published prompt set with a JSON output contract
- First published taxonomy of 45 AI security auditing blind spots across 13 failure mechanisms (expanded from 42 in V4.1)
- AMST-7 adversarial question framework for structured attack thinking
- Five-technique security stack integrating preventive, detective, architectural, and triage assessment
- Empirical results: 60+ findings across multiple assessment campaigns, plus a documented case study where URV inverted the severity ranking and surfaced three previously unflagged active bugs
- CI guard patterns for automated regression prevention
MMASA V4.2 defines a layered security methodology where each technique covers vulnerability classes the others miss. Technique 5 (URV) is structurally different from Techniques 1 to 4: it does not find bugs from scratch, it triages findings from the others.
+------------------------------------------------------------+
| MMASA SECURITY METHODOLOGY V4.2 |
+------------------------------------------------------------+
| |
| TECHNIQUE 1: SAFE IMPLEMENTATION FRAMEWORK (SIF) |
| +-- When: Every code change (pre-implementation) |
| +-- How: Gate system with adversarial questions |
| +-- Finds: Known vulnerability patterns in new code |
| +-- Gap: Only examines what's being changed |
| |
| TECHNIQUE 2: FOUR-PASS MULTI-MODEL REVIEW |
| +-- When: Every code change (post-implementation) |
| +-- How: Model A, Model B, Model A adversarial pipeline |
| +-- Finds: Provider-specific blind spots (2.5x improve) |
| +-- Gap: Bounded by the context packet supplied |
| |
| TECHNIQUE 3: ADVERSARIAL MODEL STRESS TEST (AMST) |
| +-- When: Milestone checkpoints (pre-launch, quarterly) |
| +-- How: Exhaustive enumeration + domain-grouped review |
| +-- Finds: Interaction bugs, temporal gaps, missing code |
| +-- Gap: No runtime verification |
| |
| TECHNIQUE 4: AUTOMATED SCANNING (CI/CD) |
| +-- When: Every commit (automated) |
| +-- How: SAST tools, custom rules, regression guards |
| +-- Finds: Known patterns, regressions |
| +-- Gap: No context, no interaction analysis |
| |
| TECHNIQUE 5: UPSTREAM REACHABILITY VERIFICATION (URV) NEW |
| +-- When: After every finding from techniques 1-4 |
| +-- How: Trace upstream callers + parallel paths |
| +-- Finds: True severity, blast radius, adjacent bugs |
| +-- Gap: Does not find new bugs; triages existing ones |
| |
+------------------------------------------------------------+
- Select independent models from at least two different providers (e.g., Anthropic Claude + OpenAI GPT)
- Assemble the context packet (V4.1), the diff plus callers, table writers, sibling surfaces, and the commit list. See docs/prompts.md Section 3
- Run at branch scope, not per commit (V4.1), whenever the change moves an authorization, consent, or lifecycle boundary
- Execute passes sequentially with escalating abstraction
- Interrogate the state machine (V4.1), AMST-7 against every model the change touches
- Reconcile independently, confirming a finding only when a reproduction trace can be written against real lines
- Apply URV to every finding (V4.0), determine actual severity and surface adjacent bugs
- Generate CI guards from confirmed findings
- Document blind spots, analyze what each model missed
- Enumerate all models in the system (filesystem, not documentation)
- Group into domains by association and access path (8-15 models per group)
- Apply AMST-7 questions to each model in each domain group
- Apply URV to every flagged finding (V4.0), triage severity through wiring trace
- Cross-reference findings across domains for interaction vulnerabilities
- Synthesize and prioritize by impact, fix effort, and URV-corrected severity
- Update blind spot taxonomy with newly discovered categories
After any finding from Techniques 1 to 4:
- Identify the suspect file and line
- Trace direct callers via grep
- Trace callers' callers at least two to three hops upstream
- Identify entry points (controllers, jobs, agent tools, webhooks)
- Verify wiring at the entry point (route registered? YAML config? cron schedule?)
- Look for parallel paths with the same shape but different wiring status
- Reclassify severity: Active (wired), Latent (broken but unreachable), Adjacent (parallel sibling)
See the full documentation:
- docs/methodology.md, four-pass review framework, with URV as Section 3.6 and the V4.1 input contract as Section 3.7
- docs/prompts.md, NEW, the prompts, the context packet, the JSON output contract, and the framework-semantics false-positive table
- docs/amst.md, Adversarial Model Stress Testing methodology
- docs/urv.md, NEW, Upstream Reachability Verification methodology
- docs/blind-spots.md, Complete 45-item blind spot taxonomy
- docs/coverage-matrix.md, Technique coverage comparison
| Category | Count | Core Failure | Source |
|---|---|---|---|
| Existence vs. Reality | 4 | Assuming described code actually exists/functions | V1.0 |
| Default & Inference | 4 | Security depending on implicit behavior | V1.0 |
| Association & Scope | 4 | ORM associations including unintended records | V1.0 |
| Pipeline & Process | 5 | Split paths or incomplete coverage | V1.0 |
| Output & Export | 4 | Data export vulnerabilities | V1.0 |
| Schema & Complexity | 4 | Documents ≠ schema, subsystem complexity | V2.0 |
| Test Execution | 4 | Tests exist ≠ tests run, wrong framework | V2.0 |
| Temporal & State | 3 | Expiration gaps, state cascade absence | V3.0 (AMST) |
| Interaction & Scale | 2 | Combinatorial exhaustion, cross-domain effects | V3.0 (AMST) |
| Absence & Staleness | 2 | Inverse existence gaps, config staleness | V3.1 (AMST-7) |
| Reachability & Wiring | 3 | Severity misclassification through file-only inspection | V4.0 (URV) |
| Scope Starvation | 3 | The reviewer reasoned correctly about a slice; the defect was outside it | V4.1 (branch-scope experiment) |
| Semantic Deception | 3 | The code is wired, executes, and does not do what its name promises | V3.0 (forensic audit), catalogued V4.2 |
When stress-testing each model in a system, ask:
| # | Question | What It Catches |
|---|---|---|
| 1 | What if this action is triggered twice? | Race conditions, double-spend, idempotency gaps |
| 2 | What if this is accessed from outside its intended scope? | Cross-tenant, wrong role, wrong state |
| 3 | What if a prerequisite is removed after this record exists? | State cascade failures, orphaned records |
| 4 | Are immutability and uniqueness enforced at the database level? | Soft guards, bypassable readonly methods |
| 5 | What happens when time-sensitive data expires? | Consent expiration, token reuse, stale cache |
| 6 | Do callbacks fail gracefully? | Side effect chains, async race conditions |
| 7 | What if an optional association is nil? | Missing context, nil comparison bypasses |
When triaging any finding from Techniques 1 to 4:
| # | Step | Purpose |
|---|---|---|
| 1 | Identify the suspect file and line | Capture precise location from prior technique |
| 2 | Trace direct callers | Grep for every invocation of the suspect symbol |
| 3 | Trace callers' callers (2-3 hops minimum) | Reach an actual entry point, not a midpoint |
| 4 | Identify entry points | Route, scheduled job, agent tool, webhook |
| 5 | Verify wiring at the entry point | Is the entry point actually exposed? |
| 6 | Look for parallel paths | Other files with same shape, different wiring |
| 7 | Reclassify severity | Active, Latent, or Adjacent |
Four changes, each traceable to a specific class of miss. Full detail in docs/prompts.md.
1. The context packet is a required input to the adversarial pass. A diff alone starves the adversary; it cannot find an interaction with code it was never shown. Supply the diff, every caller and reader of each changed symbol, every writer of each changed table (jobs and importers especially), every sibling surface exposing the same entity, and on a multi-commit branch the commit list.
2. Four new adversarial questions, appended to the established seven:
| # | Question | What It Catches |
|---|---|---|
| 8 | Is every WRITE to a protected object at least as guarded as the READ of it? | A mutation shipped in one commit and an authorization layer in another |
| 9 | Does any surface serialize this entity under weaker rules than its siblings? | Parity gaps across controller, serializer, LLM tool, dashboard, export |
| 10 | What LEGITIMATE user flow does this change break or make permanently impossible? | Non-exploit defects that no attacker persona asks about |
| 11 | What existing code outside the diff reads or writes what this change touched? | Importers, jobs, and seeds that a migration silently changed |
3. State-machine interrogation as a distinct cycle. Run AMST-7 against every model a change touches, plus the inverse question. Lifecycle and composition defects surface here and are invisible to per-diff attack simulation. This is AMST at change scope; it does not replace a milestone sweep.
4. A reproduction trace is required before CONFIRMED. Concrete inputs and starting state, the call sequence, the wrong outcome, every step citing a real line. If the trace cannot be written, the finding is PLAUSIBLE or REFUTED. A fourth verdict, BY-DESIGN, keeps correct observations of intentional behavior out of the false-positive count.
Run the branch, not the increment. Reviewing each commit as it lands misses composition defects by construction. Whenever a change moves an authorization, consent, or lifecycle boundary, re-run the adversarial pass against the whole branch with that boundary named.
| Vulnerability Class | SIF | Four-Pass | AMST | CI Scanning | URV |
|---|---|---|---|---|---|
| Auth bypass | ✅ | ✅ | ✅ | ✅ | n/a |
| Unscoped queries | ✅ | ✅ | ✅ | ✅ | n/a |
| Sensitive data in logs | ✅ | ✅ | ✅ | ✅ | n/a |
| Race conditions | ✅ | Partial | ✅ | ❌ | n/a |
| Consent/authz bypass | ✅ | ✅ | ✅ | ❌ | n/a |
| Temporal expiration gaps | ❌ | Partial | ✅ | ❌ | n/a |
| Cross-domain interaction | ❌ | Partial | ✅ | ❌ | n/a |
| Missing code (should exist) | Partial | Partial | ✅ | ❌ | n/a |
| State cascade failures | ❌ | Partial | ✅ | ❌ | n/a |
| Combinatorial identity | ❌ | ❌ | ✅ | ❌ | n/a |
| Cross-commit composition gap | ❌ | ✅ | ❌ | ❌ | n/a |
| Outside-the-diff writer effects | ❌ | ✅ | Partial | ❌ | n/a |
| Cross-surface parity gap | ❌ | ✅ | Partial | ❌ | n/a |
| Broken legitimate flow | ❌ | ✅ | Partial | ❌ | n/a |
| No-op safety method | Partial | ❌ | ❌ | ❌ | n/a |
| Model name confusion | Partial | ❌ | ✅ | ✅ | n/a |
| Severity misclassification | ❌ | ❌ | ❌ | ❌ | ✅ |
| Adjacent-bug discovery | ❌ | ❌ | ❌ | ❌ | ✅ |
| Multi-hop reachability | ❌ | ❌ | Partial | ❌ | ✅ |
URV's unique contribution is in the bottom three rows, vulnerability classes that no other technique reliably catches. The four V4.1 rows above them depend on the context packet: a review run on a bare diff catches none of them. The Four-Pass column's Partial marks in the AMST block come from the state-machine cycle and apply to touched models only.
A ✅ in this matrix means a technique catches the class once the class is known, not that it would have discovered it. Fail-open authorization reads ✅ across four techniques on the auth-bypass row above, and it was invisible to all four passes of the original assessment until execution-flow tracing found it in 25+ controller actions. See coverage-matrix.md for the full caveat.
- 35 findings across 7 commits
- 85+ files modified
- 2,000+ tests, zero regressions
- 2.5x improvement over single-model assessment
- 26 findings across 19 domain-grouped rounds
- 6 critical issues missed by three prior independent audits
- 400+ data models analyzed in approximately 4 hours
- 5 new blind spot categories discovered
- 1 starting finding from Four-Pass Review
- 3 previously unflagged active bugs surfaced via parallel-path enumeration
- 1 severity inversion (the original Critical/Active finding reclassified as Latent)
- 3 new blind spot entries documented (Category 11: Reachability & Wiring)
- Full case study in urv.md Section 3
- 1 production feature branch, roughly 3,100 lines across 12 commits
- 3 review passes at three points in the branch's life, two frontier providers as adversaries
- 11 real vulnerabilities confirmed and fixed by those passes
- 8 further confirmed defects found afterward by an independent multi-agent review, including an authorization bypass where a state-changing endpoint was reachable by staff explicitly forbidden from reading the record
- 6 of the 8 traced to missing context or a missing lens rather than to model capability
- 3 new blind spot entries documented (Category 12: Scope Starvation)
- Boundary: one codebase, one branch, three adversarial passes against one review run. Directional evidence, not a controlled study; effect sizes are unmeasured, and no improvement in detection rate is claimed. Full evidence table in prompts.md Section 10
| Finding Type | Four-Pass | AMST | URV | Both/All |
|---|---|---|---|---|
| Pattern-based (auth, scoping, PII) | ✅ | ✅ | n/a | Redundant coverage |
| Interaction bugs | ❌ | ✅ | n/a | AMST-only |
| Temporal/expiration gaps | ❌ | ✅ | n/a | AMST-only |
| Missing code | ❌ | ✅ | n/a | AMST-only |
| Provider-specific blind spots | ✅ | ❌ | n/a | Four-Pass-only |
| Severity misclassification | ❌ | ❌ | ✅ | URV-only |
| Adjacent bugs | ❌ | ❌ | ✅ | URV-only |
This methodology builds on:
- Vijayaraghavan et al. (2026), Multi-agent organizational intelligence and the "Team of Rivals" architecture
- Reason (1990), Swiss cheese model of layered defenses
- Isotopes AI (2026), The Orchestration Imperative for AI Agents
The convergence of multiple independent research efforts on the same principles provides validation that multi-model orchestration is a generalizable pattern. AMST extended this by applying adversarial thinking not just across models (inter-model) but within a single model's analysis of the full architecture (intra-model, structured enumeration).
V4.0 extends this further by recognizing that finding bugs is only half the work: triaging them for actual severity and surfacing adjacent ones requires its own structured technique. URV provides that triage step.
V4.1 addresses a limit that model diversity cannot reach. Misaligned failure modes protect against reviewers reasoning badly; they do not protect against every reviewer being handed the same incomplete slice. When the defect lives outside what all reviewers were shown, the holes in the layers align perfectly regardless of how different the layers are. The remedy is not another layer but a wider input.
If you use this methodology in your work, please cite:
@misc{barrera_castanon_mmasa_2026_v4_2,
author = {Barrera Castañón, Luis},
title = {{MMASA}: Multi-Model Adversarial Security Assessment},
year = {2026},
month = {8},
url = {https://github.com/LBCplus/Multi-Model-Adversarial-Security-Assessment},
note = {Version 4.2, includes the review input contract, reproduction-trace standard, and the full 45-item blind spot taxonomy}
}
This methodology framework, blind spot taxonomy, and documentation are licensed under CC BY 4.0. You are free to share and adapt this material for any purpose with appropriate attribution.
Luis Barrera Castañón, Ed.D. Founder & CEO, Lived Experience Inc. ORCID: 0009-0005-1652-0099
This methodology was developed with assistance from Claude (Anthropic) and ChatGPT (OpenAI). See the full methodology document, the AMST document, the URV document, and the prompts document for AI assistance disclosures.