TrialMatch

Precision Oncology Protocol Matching & Verification
The Duel (Live Match)3-Arm Benchmark
Sanity Context MCP: Connected
DEV x Sanity 2026

10-Patient Gold Standard Clinical Benchmark

100 TRIALS

Automated comparative evaluation: Structured Sanity Agent vs Naive Keyword vs Bare LLM.

Arm 1: Structured Sanity Agent

100% DETERMINISTIC
Avg Clinical Precision
100%
Zero false positives
Safety Violations
0
Strict protocol checks
Hallucinations: 0Auditability: 100% GROQ Trace

Arm 2: Naive Flat Keyword

HIGH TOXICITY HAZARD
Avg Clinical Precision
78%
Disqualified patient matches
Safety Violations
60
Exclusion clauses ignored
Hallucinations: 0 (In-corpus)Auditability: 0% (Opaque scan)

Arm 3: Bare LLM (Zero DB)

HALLUCINATES IDS
Avg Clinical Precision
0%
Zero database verification
Hallucinated NCT IDs
20
Fake or defunct IDs
Safety Violations: 20Auditability: 0% (No DB access)
Showing 10 of 10 gold-standard test profiles
Test Case & ProfileArm 1: Structured SanityArm 2: Naive Keyword SearchArm 3: Bare LLM (Zero DB)
TC-01Texas
Lung
Biomarker: EGFR
Prior: Platinum Chemotherapy
Audit Details
Precision: 100%(2 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE
Precision: 99%(69 matches)
FAILED: 1 Safety Violations
FAILED: Patient banned by chemo exclusion in Rule 14. Trial explicitly bars prior platinum chemotherapy.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04596395, NCT04847926
TC-02California
Colorectal
Biomarker: KRAS
Prior: Prior FOLFOX Allowed
Audit Details
Precision: 100%(2 trials)
Safety Violations: 0
Rule: RULE-BIOMARKER-SPECIFICITY & Organ Site Metastatic Protocol Hierarchy
Precision: 93%(14 matches)
FAILED: 1 Safety Violations
FAILED: Patient banned by hepatic exclusion. Protocol prohibits active liver metastases.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04185971, NCT03785249
TC-03Florida
Breast
Biomarker: HER2
Prior: Prior Anti-HER2 Targeted Therapy
Audit Details
Precision: 100%(3 trials)
Safety Violations: 0
Rule: RULE-BIOMARKER-SPECIFICITY
Precision: 89%(72 matches)
FAILED: 8 Safety Violations
FAILED: Quantitative expression mismatch. Protocol requires HER2-positive (IHC 3+), patient is HER2-low (IHC 1+/2+).
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04556770, NCT05436214
TC-04New York
Melanoma
Biomarker: BRAF
Prior: Any
Audit Details
Precision: 100%(1 trials)
Safety Violations: 0
Rule: RULE-PHASE-APPROPRIATENESS
Precision: 13%(30 matches)
FAILED: 26 Safety Violations
FAILED: Trial phase violation. Returned PHASE4 early-phase trial despite strict Phase 3 requirement.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04547903, NCT02224781
TC-05Ohio
Ovarian
Biomarker: BRCA
Prior: Chemotherapy Allowed (Sensitive)
Audit Details
Precision: 100%(1 trials)
Safety Violations: 0
Rule: RULE-WASHOUT-WINDOWS & Recurrence Sensitivity Hierarchy
Precision: 92%(13 matches)
FAILED: 1 Safety Violations
FAILED: Relapse interval mismatch. Trial restricted to platinum-resistant disease; patient has platinum-sensitive recurrence.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT05123456, NCT05987654
TC-06Pennsylvania
Prostate
Biomarker: BRCA
Prior: Prior Enzalutamide Permitted
Audit Details
Precision: 100%(0 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE
Precision: 95%(20 matches)
FAILED: 1 Safety Violations
FAILED: Patient banned by hormonal exclusion. Trial bars patients previously treated with enzalutamide.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04719277, NCT03834519
TC-07Massachusetts
Pancreatic
Biomarker: KRAS
Prior: Chemo Naive (No Prior Systemic Therapy)
Audit Details
Precision: 100%(0 trials)
Safety Violations: 0
Rule: RULE-BIOMARKER-SPECIFICITY
Precision: 21%(14 matches)
FAILED: 11 Safety Violations
FAILED: Genomic polarity inversion. Trial requires activating KRAS mutation; patient is KRAS wild-type.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT05123456, NCT05234567
TC-08North Carolina
Glioblastoma
Biomarker: EGFR
Prior: Newly Diagnosed (No Prior Systemic Chemo)
Audit Details
Precision: 100%(2 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE & Disease Chronology Staging
Precision: 78%(45 matches)
FAILED: 10 Safety Violations
FAILED: Disease chronology mismatch. Trial requires recurrent glioblastoma following radiation failure; patient is newly diagnosed.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Zero auditable database query.
TC-09Illinois
Leukemia
Biomarker: EGFR
Prior: Relapsed/Refractory Post 7+3 Induction
Audit Details
Precision: 100%(1 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE
Precision: 100%(46 matches)
FAILED: 0 Safety Violations
Matches disqualified patient via negative criteria.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT05934123, NCT06015789
TC-10Washington
Cancer
Biomarker: FGFR
Prior: Prior Anti-PD-1/L1 Immunotherapy Required
Audit Details
Precision: 100%(0 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE & Prior Immunotherapy Exposure Stratification
Precision: 99%(92 matches)
FAILED: 1 Safety Violations
FAILED: Immunotherapy exclusion. Protocol bars patients with prior immune checkpoint inhibitor therapy.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04747970, NCT04845575