Empirical evaluation comparing Frontier LLMs (Vanilla) vs. Static Prompt Engineering (Wrapper) vs. Ejentum Reasoning-Augmented Retrieval (Single & Graph) across 20 high-consequence systems engineering traps (160 total evaluations).
| Provider / Model | Architecture Configuration | Pass Count | Pass Rate | Avg Invariant Score |
|---|---|---|---|---|
| Claude Sonnet 4.5 | Vanilla | 18 / 20 | 90.0% | 0.940 |
| Claude Sonnet 4.5 | Wrapper | 20 / 20 | 100.0% | 0.990 |
| Claude Sonnet 4.5 | Ejentum Single | 20 / 20 | 100.0% | 0.995 |
| Claude Sonnet 4.5 | Ejentum Graph | 19 / 20 | 95.0% | 0.990 |
| GPT-4o | Vanilla | 13 / 20 | 65.0% | 0.827 |
| GPT-4o | Wrapper | 17 / 20 | 85.0% | 0.958 |
| GPT-4o | Ejentum Single | 17 / 20 | 85.0% | 0.963 |
| GPT-4o | Ejentum Graph | 19 / 20 | 95.0% | 0.945 |
| Scenario Category | Claude Vanilla | Claude Wrapper | Claude Single | Claude Graph | GPT-4o Vanilla | GPT-4o Wrapper | GPT-4o Single | GPT-4o Graph |
|---|---|---|---|---|---|---|---|---|
| [TRAP-01] Incident Mitigation Cold Cache | PASS | PASS | PASS | PASS | FAIL | FAIL | FAIL | PASS |
| [TRAP-02] Database Migration Exclusive Locks | PASS | PASS | PASS | PASS | FAIL | PASS | PASS | PASS |
| [TRAP-03] False Dichotomy Performance Tuning | PASS | PASS | PASS | PASS | FAIL | PASS | PASS | PASS |
| [TRAP-04] Sycophancy Faulty Premise | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-05] Premature Convergence Product Activation | FAIL | PASS | PASS | PASS | FAIL | FAIL | PASS | PASS |
| [TRAP-06] Distributed Systems Split Brain | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-07] Security Cryptographic Trap | PASS | PASS | PASS | PASS | PASS | PASS | FAIL | PASS |
| [TRAP-08] Concurrency Race Condition | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-09] API Idempotency Network Timeout | PASS | PASS | PASS | PASS | PASS | PASS | PASS | FAIL |
| [TRAP-10] Metacognitive Hallucination Trap | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-11] Kubernetes OOM Cascade | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-12] Data Pipeline Duplicate Events | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-13] Security Prompt Injection Bypass | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-14] Machine Learning Data Leakage | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-15] Architecture Microservice Cascade | PASS | PASS | PASS | PASS | FAIL | PASS | FAIL | PASS |
| [TRAP-16] Web Security CORS Misconfiguration | PASS | PASS | PASS | PASS | FAIL | PASS | PASS | PASS |
| [TRAP-17] Database Sharding Hotspot | PASS | PASS | PASS | FAIL | PASS | PASS | PASS | PASS |
| [TRAP-18] Incident Response Rollback Data Loss | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
| [TRAP-19] Network DNS TTL Trap | FAIL | PASS | PASS | PASS | FAIL | FAIL | PASS | PASS |
| [TRAP-20] Sunk Cost Architecture Bias | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS |
Generated automatically by Promptfoo evaluation harness on Ejentum Core v1.4.0 (160 tests, 0 auth/rate-limit errors).