TrialMatch
Precision Oncology Protocol Matching & VerificationSanity Context MCP: Connected
DEV x Sanity 2026
10-Patient Gold Standard Clinical Benchmark
100 TRIALSAutomated comparative evaluation: Structured Sanity Agent vs Naive Keyword vs Bare LLM.
Arm 1: Structured Sanity Agent
Avg Clinical Precision
100%
Zero false positives
Safety Violations
0
Strict protocol checks
Hallucinations: 0Auditability: 100% GROQ Trace
Arm 2: Naive Flat Keyword
Avg Clinical Precision
60%
Disqualified patient matches
Safety Violations
152
Exclusion clauses ignored
Hallucinations: 0 (In-corpus)Auditability: 0% (Opaque scan)
Arm 3: Bare LLM (Zero DB)
Avg Clinical Precision
0%
Zero database verification
Hallucinated NCT IDs
22
Fake or defunct IDs
Safety Violations: 22Auditability: 0% (No DB access)
Showing 10 of 10 gold-standard test profiles
| Test Case & Profile | Arm 1: Structured Sanity | Arm 2: Naive Keyword Search | Arm 3: Bare LLM (Zero DB) |
|---|---|---|---|
TC-01Texas Lung Biomarker: EGFR Prior: Platinum Chemotherapy Audit Details | Precision: 100%(2 trials) Safety Violations: 0 Rule: RULE-EXCLUSION-OVERRIDE | Precision: 55%(69 matches) FAILED: 31 Safety Violations FAILED: Disease mismatch. Matched query term "lung" inside negative exclusion criteria ("History of active lung is excluded"). | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT05607550, NCT03974022 |
TC-02California Colorectal Biomarker: KRAS Prior: Prior FOLFOX Allowed Audit Details | Precision: 100%(2 trials) Safety Violations: 0 Rule: RULE-BIOMARKER-SPECIFICITY & Organ Site Metastatic Protocol Hierarchy | Precision: 93%(14 matches) FAILED: 1 Safety Violations FAILED: Patient disqualified by hepatic exclusion. Protocol prohibits active liver metastases. | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT04793958, NCT05198934 |
TC-03Florida Breast Biomarker: HER2 Prior: Prior Anti-HER2 Targeted Therapy Audit Details | Precision: 100%(3 trials) Safety Violations: 0 Rule: RULE-BIOMARKER-SPECIFICITY | Precision: 53%(72 matches) FAILED: 34 Safety Violations FAILED: Disease mismatch. Matched query term "breast" inside negative exclusion criteria ("History of active breast is excluded"). | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT06018337, NCT04556773 |
TC-04New York Melanoma Biomarker: BRAF Prior: Any Audit Details | Precision: 100%(1 trials) Safety Violations: 0 Rule: RULE-PHASE-APPROPRIATENESS | Precision: 10%(30 matches) FAILED: 27 Safety Violations FAILED: Trial phase violation. Returned PHASE4 early-phase trial despite strict PHASE3 requirement. | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT04511013, NCT05352672 |
TC-05Ohio Ovarian Biomarker: BRCA Prior: Chemotherapy Allowed (Sensitive) Audit Details | Precision: 100%(1 trials) Safety Violations: 0 Rule: RULE-WASHOUT-WINDOWS & Recurrence Sensitivity Hierarchy | Precision: 92%(13 matches) FAILED: 1 Safety Violations FAILED: Relapse interval mismatch. Trial restricted to platinum-resistant disease; patient has platinum-sensitive recurrence. | Precision: 0%(4 trials) HALLUCINATED: 4 Invalid NCT IDs Produced unverified: NCT03574922, NCT03462342 |
TC-06Pennsylvania Prostate Biomarker: BRCA Prior: Prior Enzalutamide Permitted Audit Details | Precision: 100%(0 trials) Safety Violations: 0 Rule: RULE-EXCLUSION-OVERRIDE | Precision: 40%(20 matches) FAILED: 12 Safety Violations FAILED: Disease mismatch. Matched query term "prostate" inside negative exclusion criteria ("History of active prostate is excluded"). | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT04644835, NCT04497116 |
TC-07Massachusetts Pancreatic Biomarker: KRAS Prior: Chemo Naive (No Prior Systemic Therapy) Audit Details | Precision: 100%(0 trials) Safety Violations: 0 Rule: RULE-BIOMARKER-SPECIFICITY | Precision: 7%(14 matches) FAILED: 13 Safety Violations FAILED: Genomic polarity inversion. Trial requires activating mutation; patient is documented wild-type. | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT03563248, NCT03825705 |
TC-08North Carolina Glioblastoma Biomarker: EGFR Prior: Newly Diagnosed (No Prior Systemic Chemo) Audit Details | Precision: 100%(2 trials) Safety Violations: 0 Rule: RULE-EXCLUSION-OVERRIDE & Disease Chronology Staging | Precision: 80%(45 matches) FAILED: 9 Safety Violations FAILED: Disease chronology mismatch. Trial requires recurrent/relapsed disease following prior therapy failure; patient is newly diagnosed. | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT04797468, NCT02977780 |
TC-09Illinois Leukemia Biomarker: EGFR Prior: Relapsed/Refractory Post 7+3 Induction Audit Details | Precision: 100%(1 trials) Safety Violations: 0 Rule: RULE-EXCLUSION-OVERRIDE | Precision: 96%(46 matches) FAILED: 2 Safety Violations FAILED: Disease mismatch. Matched query term "leukemia" inside negative exclusion criteria ("History of active leukemia is excluded"). | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT03625505, NCT04240067 |
TC-10Washington Cancer Biomarker: FGFR Prior: Prior Anti-PD-1/L1 Immunotherapy Required Audit Details | Precision: 100%(0 trials) Safety Violations: 0 Rule: RULE-EXCLUSION-OVERRIDE & Prior Immunotherapy Exposure Stratification | Precision: 76%(92 matches) FAILED: 22 Safety Violations FAILED: Disease mismatch. Matched query term "cancer" inside negative exclusion criteria ("History of active cancer is excluded"). | Precision: 0%(2 trials) HALLUCINATED: 2 Invalid NCT IDs Produced unverified: NCT05727020, NCT04172675 |