TrialMatch

Precision Oncology Protocol Matching & Verification
Sanity Context MCP: Connected
DEV x Sanity 2026

10-Patient Gold Standard Clinical Benchmark

100 TRIALS

Automated comparative evaluation: Structured Sanity Agent vs Naive Keyword vs Bare LLM.

Arm 1: Structured Sanity Agent

100% DETERMINISTIC
Avg Clinical Precision
100%
Zero false positives
Safety Violations
0
Strict protocol checks
Hallucinations: 0Auditability: 100% GROQ Trace

Arm 2: Naive Flat Keyword

HIGH TOXICITY HAZARD
Avg Clinical Precision
60%
Disqualified patient matches
Safety Violations
152
Exclusion clauses ignored
Hallucinations: 0 (In-corpus)Auditability: 0% (Opaque scan)

Arm 3: Bare LLM (Zero DB)

HALLUCINATES IDS
Avg Clinical Precision
0%
Zero database verification
Hallucinated NCT IDs
22
Fake or defunct IDs
Safety Violations: 22Auditability: 0% (No DB access)
Showing 10 of 10 gold-standard test profiles
Test Case & ProfileArm 1: Structured SanityArm 2: Naive Keyword SearchArm 3: Bare LLM (Zero DB)
TC-01Texas
Lung
Biomarker: EGFR
Prior: Platinum Chemotherapy
Audit Details
Precision: 100%(2 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE
Precision: 55%(69 matches)
FAILED: 31 Safety Violations
FAILED: Disease mismatch. Matched query term "lung" inside negative exclusion criteria ("History of active lung is excluded").
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT05607550, NCT03974022
TC-02California
Colorectal
Biomarker: KRAS
Prior: Prior FOLFOX Allowed
Audit Details
Precision: 100%(2 trials)
Safety Violations: 0
Rule: RULE-BIOMARKER-SPECIFICITY & Organ Site Metastatic Protocol Hierarchy
Precision: 93%(14 matches)
FAILED: 1 Safety Violations
FAILED: Patient disqualified by hepatic exclusion. Protocol prohibits active liver metastases.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04793958, NCT05198934
TC-03Florida
Breast
Biomarker: HER2
Prior: Prior Anti-HER2 Targeted Therapy
Audit Details
Precision: 100%(3 trials)
Safety Violations: 0
Rule: RULE-BIOMARKER-SPECIFICITY
Precision: 53%(72 matches)
FAILED: 34 Safety Violations
FAILED: Disease mismatch. Matched query term "breast" inside negative exclusion criteria ("History of active breast is excluded").
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT06018337, NCT04556773
TC-04New York
Melanoma
Biomarker: BRAF
Prior: Any
Audit Details
Precision: 100%(1 trials)
Safety Violations: 0
Rule: RULE-PHASE-APPROPRIATENESS
Precision: 10%(30 matches)
FAILED: 27 Safety Violations
FAILED: Trial phase violation. Returned PHASE4 early-phase trial despite strict PHASE3 requirement.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04511013, NCT05352672
TC-05Ohio
Ovarian
Biomarker: BRCA
Prior: Chemotherapy Allowed (Sensitive)
Audit Details
Precision: 100%(1 trials)
Safety Violations: 0
Rule: RULE-WASHOUT-WINDOWS & Recurrence Sensitivity Hierarchy
Precision: 92%(13 matches)
FAILED: 1 Safety Violations
FAILED: Relapse interval mismatch. Trial restricted to platinum-resistant disease; patient has platinum-sensitive recurrence.
Precision: 0%(4 trials)
HALLUCINATED: 4 Invalid NCT IDs
Produced unverified: NCT03574922, NCT03462342
TC-06Pennsylvania
Prostate
Biomarker: BRCA
Prior: Prior Enzalutamide Permitted
Audit Details
Precision: 100%(0 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE
Precision: 40%(20 matches)
FAILED: 12 Safety Violations
FAILED: Disease mismatch. Matched query term "prostate" inside negative exclusion criteria ("History of active prostate is excluded").
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04644835, NCT04497116
TC-07Massachusetts
Pancreatic
Biomarker: KRAS
Prior: Chemo Naive (No Prior Systemic Therapy)
Audit Details
Precision: 100%(0 trials)
Safety Violations: 0
Rule: RULE-BIOMARKER-SPECIFICITY
Precision: 7%(14 matches)
FAILED: 13 Safety Violations
FAILED: Genomic polarity inversion. Trial requires activating mutation; patient is documented wild-type.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT03563248, NCT03825705
TC-08North Carolina
Glioblastoma
Biomarker: EGFR
Prior: Newly Diagnosed (No Prior Systemic Chemo)
Audit Details
Precision: 100%(2 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE & Disease Chronology Staging
Precision: 80%(45 matches)
FAILED: 9 Safety Violations
FAILED: Disease chronology mismatch. Trial requires recurrent/relapsed disease following prior therapy failure; patient is newly diagnosed.
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT04797468, NCT02977780
TC-09Illinois
Leukemia
Biomarker: EGFR
Prior: Relapsed/Refractory Post 7+3 Induction
Audit Details
Precision: 100%(1 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE
Precision: 96%(46 matches)
FAILED: 2 Safety Violations
FAILED: Disease mismatch. Matched query term "leukemia" inside negative exclusion criteria ("History of active leukemia is excluded").
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT03625505, NCT04240067
TC-10Washington
Cancer
Biomarker: FGFR
Prior: Prior Anti-PD-1/L1 Immunotherapy Required
Audit Details
Precision: 100%(0 trials)
Safety Violations: 0
Rule: RULE-EXCLUSION-OVERRIDE & Prior Immunotherapy Exposure Stratification
Precision: 76%(92 matches)
FAILED: 22 Safety Violations
FAILED: Disease mismatch. Matched query term "cancer" inside negative exclusion criteria ("History of active cancer is excluded").
Precision: 0%(2 trials)
HALLUCINATED: 2 Invalid NCT IDs
Produced unverified: NCT05727020, NCT04172675