Ignite Creation Date:
2026-03-26 @ 3:15 PM
Ignite Modification Date:
2026-03-30 @ 2:59 AM
Study NCT ID:
NCT07414966
Status:
NOT_YET_RECRUITING
Last Update Posted:
2026-02-17
First Post:
2026-02-09
Is NOT Gene Therapy:
True
Has Adverse Events:
False
Brief Title:
Scalable Clinical Oversight of Large Language Models Via Uncertainty Triangulation
Sponsor:
Organization:
Raw JSON
{'hasResults': False, 'derivedSection': {'miscInfoModule': {'versionHolder': '2026-03-25'}, 'conditionBrowseModule': {'meshes': [{'id': 'D003327', 'term': 'Coronary Disease'}], 'ancestors': [{'id': 'D017202', 'term': 'Myocardial Ischemia'}, {'id': 'D006331', 'term': 'Heart Diseases'}, {'id': 'D002318', 'term': 'Cardiovascular Diseases'}, {'id': 'D014652', 'term': 'Vascular Diseases'}]}}, 'protocolSection': {'designModule': {'phases': ['NA'], 'studyType': 'INTERVENTIONAL', 'designInfo': {'allocation': 'RANDOMIZED', 'maskingInfo': {'masking': 'NONE'}, 'primaryPurpose': 'DIAGNOSTIC', 'interventionModel': 'CROSSOVER'}, 'enrollmentInfo': {'type': 'ESTIMATED', 'count': 7}}, 'statusModule': {'overallStatus': 'NOT_YET_RECRUITING', 'startDateStruct': {'date': '2026-02-19', 'type': 'ESTIMATED'}, 'expandedAccessInfo': {'hasExpandedAccess': False}, 'statusVerifiedDate': '2026-02', 'completionDateStruct': {'date': '2026-02-28', 'type': 'ESTIMATED'}, 'lastUpdateSubmitDate': '2026-02-14', 'studyFirstSubmitDate': '2026-02-09', 'studyFirstSubmitQcDate': '2026-02-14', 'lastUpdatePostDateStruct': {'date': '2026-02-17', 'type': 'ACTUAL'}, 'studyFirstPostDateStruct': {'date': '2026-02-17', 'type': 'ACTUAL'}, 'primaryCompletionDateStruct': {'date': '2026-02-28', 'type': 'ESTIMATED'}}, 'outcomesModule': {'primaryOutcomes': [{'measure': 'Mean physician review time per case (minutes)', 'timeFrame': 'Through study completion, an average of 2 hours.', 'description': 'Mean time spent by each clinician reviewing and rendering a diagnostic decision per case under each arm. Measured in minutes.'}], 'secondaryOutcomes': [{'measure': 'Diagnostic accuracy (%)', 'timeFrame': 'Through study completion, an average of 2 hours.', 'description': 'Proportion of correct CHD subtype classifications (STEMI, NSTEMI, unstable angina, chronic coronary syndromes) under each arm.'}, {'measure': 'Computational Return on Investment (ROI)', 'timeFrame': 'Through study completion, an average of 2 hours.', 'description': 'Ratio of physician time savings (valued at standardized minute-wages from Sanming healthcare reform benchmarks) to computational cost of SCOUT inference, stratified by clinician seniority level.'}]}, 'oversightModule': {'oversightHasDmc': True, 'isFdaRegulatedDrug': False, 'isFdaRegulatedDevice': False}, 'conditionsModule': {'keywords': ['artificial intelligence'], 'conditions': ['Coronary Heart Disease (CHD)']}, 'descriptionModule': {'briefSummary': 'This prospective, multi-reader, randomized crossover trial evaluates SCOUT (Scalable Clinical Oversight via Uncertainty Triangulation), a model-agnostic meta-verification framework that selectively defers unreliable large language model (LLM) predictions to clinicians by triangulating three orthogonal uncertainty signals: model heterogeneity, stochastic inconsistency, and reasoning critique. The trial assesses whether SCOUT-assisted review can reduce physician review time compared with standard manual review of AI-generated diagnoses while maintaining non-inferior diagnostic accuracy in coronary heart disease (CHD) subtyping.', 'detailedDescription': 'Background: Large language models are increasingly deployed in clinical workflows, yet requiring clinician review of every AI output negates the efficiency gains that motivate their adoption. SCOUT addresses this efficiency-safety paradox through algorithmic meta-verification.\n\nThe SCOUT framework triangulates three orthogonal external signals to determine case-level uncertainty: (1) Model Heterogeneity - whether a structurally different auxiliary LLM agrees with the primary model; (2) Stochastic Inconsistency - whether repeated sampling from the same model yields divergent outputs; (3) Reasoning Critique - whether an external checker model identifies logical flaws in the chain-of-thought reasoning.\n\nIn this crossover trial, 7 clinicians of varying seniority (2 junior residents, 3 senior residents, 2 attending physicians) each review all 110 cases under both standard manual review and SCOUT-assisted review workflows. The study evaluates workflow efficiency (primary endpoint) and diagnostic accuracy (secondary endpoint).'}, 'eligibilityModule': {'sex': 'ALL', 'stdAges': ['ADULT', 'OLDER_ADULT'], 'minimumAge': '18 Years', 'healthyVolunteers': False, 'eligibilityCriteria': 'Inclusion Criteria:\n\n* Board-certified or in-training cardiologists at Fuwai Hospital\n* Spanning three experience strata: junior residents, senior residents, attending physicians\n\nExclusion Criteria:\n\n* Clinicians involved in the development or optimization of the SCOUT framework\n* Clinicians involved in the gold-standard adjudication process'}, 'identificationModule': {'nctId': 'NCT07414966', 'acronym': 'SCOUT', 'briefTitle': 'Scalable Clinical Oversight of Large Language Models Via Uncertainty Triangulation', 'organization': {'class': 'OTHER_GOV', 'fullName': 'China National Center for Cardiovascular Diseases'}, 'officialTitle': 'Prospective Evaluation of a Model-Agnostic Meta-Verification Framework (SCOUT) for Scalable Clinical Oversight of Large Language Model Outputs in Coronary Heart Disease Diagnosis: A Multi-Reader, Randomized, Crossover Trial', 'orgStudyIdInfo': {'id': '2025-2702-1'}}, 'armsInterventionsModule': {'armGroups': [{'type': 'ACTIVE_COMPARATOR', 'label': 'Control (Standard Manual Review)', 'description': 'Physicians manually review all cases in the control set (n=54) with access to AI predictions and reasoning. No selective deferral.', 'interventionNames': ['Diagnostic Test: Standard Manual Review Workflow']}, {'type': 'EXPERIMENTAL', 'label': 'Experimental (SCOUT-Assisted Review)', 'description': 'Physicians process the intervention set (n=56) through the SCOUT framework. Low-uncertainty cases are auto-accepted; high-uncertainty cases undergo physician review with full audit trail.', 'interventionNames': ['Diagnostic Test: SCOUT-Assisted Review Workflow']}], 'interventions': [{'name': 'SCOUT-Assisted Review Workflow', 'type': 'DIAGNOSTIC_TEST', 'description': "SCOUT-Assisted Review (Intervention Arm): Physicians review 56 cases processed through the SCOUT framework. For cases classified as low-uncertainty (D(x)=0), the AI prediction is auto-accepted without physician review. For high-uncertainty cases (D(x)=1), the physician reviews the case with access to the main model's chain-of-thought reasoning and the meta-verification audit results. The main model is DeepSeek-V3.1 with chain-of-thought prompting.", 'armGroupLabels': ['Experimental (SCOUT-Assisted Review)']}, {'name': 'Standard Manual Review Workflow', 'type': 'DIAGNOSTIC_TEST', 'description': "Physicians perform a full manual review of 54 cases using raw medical records with access to the AI model's predictions and reasoning, but without SCOUT uncertainty stratification or selective deferral.", 'armGroupLabels': ['Control (Standard Manual Review)']}]}, 'contactsLocationsModule': {'centralContacts': [{'name': 'Xiaojin Gao, Dr.', 'role': 'CONTACT', 'email': 'sophie_gao@sina.com', 'phone': '+86 010 88322415'}]}, 'ipdSharingStatementModule': {'infoTypes': ['STUDY_PROTOCOL', 'SAP', 'ICF', 'CSR', 'ANALYTIC_CODE'], 'timeFrame': 'Beginning 1 months after publication of the primary results and available for up to 60 months.', 'ipdSharing': 'YES', 'description': 'De-identified individual participant data underlying the results reported in this study will be made available.', 'accessCriteria': 'Data are available from the corresponding author upon reasonable request. Requestors will need to provide a methodologically sound research proposal and sign a data use agreement.'}, 'sponsorCollaboratorsModule': {'leadSponsor': {'name': 'China National Center for Cardiovascular Diseases', 'class': 'OTHER_GOV'}, 'responsibleParty': {'type': 'SPONSOR'}}}}