Ignite Creation Date:
2025-12-25 @ 5:05 AM
Ignite Modification Date:
2025-12-26 @ 4:08 AM
Study NCT ID:
NCT06814327
Status:
ACTIVE_NOT_RECRUITING
Last Update Posted:
2025-08-28
First Post:
2024-11-18
Is NOT Gene Therapy:
False
Has Adverse Events:
False
Brief Title:
Assessment of Organ Failure Risk Predictions in ICU
Sponsor:
Organization:
Raw JSON
{'hasResults': False, 'derivedSection': {'miscInfoModule': {'versionHolder': '2025-12-24'}, 'conditionBrowseModule': {'meshes': [{'id': 'D012769', 'term': 'Shock'}, {'id': 'D012131', 'term': 'Respiratory Insufficiency'}, {'id': 'D051437', 'term': 'Renal Insufficiency'}], 'ancestors': [{'id': 'D010335', 'term': 'Pathologic Processes'}, {'id': 'D013568', 'term': 'Pathological Conditions, Signs and Symptoms'}, {'id': 'D012120', 'term': 'Respiration Disorders'}, {'id': 'D012140', 'term': 'Respiratory Tract Diseases'}, {'id': 'D007674', 'term': 'Kidney Diseases'}, {'id': 'D014570', 'term': 'Urologic Diseases'}, {'id': 'D052776', 'term': 'Female Urogenital Diseases'}, {'id': 'D005261', 'term': 'Female Urogenital Diseases and Pregnancy Complications'}, {'id': 'D000091642', 'term': 'Urogenital Diseases'}, {'id': 'D052801', 'term': 'Male Urogenital Diseases'}]}}, 'protocolSection': {'designModule': {'studyType': 'OBSERVATIONAL', 'designInfo': {'timePerspective': 'PROSPECTIVE', 'observationalModel': 'COHORT'}, 'enrollmentInfo': {'type': 'ACTUAL', 'count': 499}, 'patientRegistry': False}, 'statusModule': {'overallStatus': 'ACTIVE_NOT_RECRUITING', 'startDateStruct': {'date': '2024-11-18', 'type': 'ACTUAL'}, 'expandedAccessInfo': {'hasExpandedAccess': False}, 'statusVerifiedDate': '2024-11', 'completionDateStruct': {'date': '2026-05-15', 'type': 'ESTIMATED'}, 'lastUpdateSubmitDate': '2025-08-27', 'studyFirstSubmitDate': '2024-11-18', 'studyFirstSubmitQcDate': '2025-02-03', 'lastUpdatePostDateStruct': {'date': '2025-08-28', 'type': 'ESTIMATED'}, 'studyFirstPostDateStruct': {'date': '2025-02-07', 'type': 'ACTUAL'}, 'primaryCompletionDateStruct': {'date': '2025-05-15', 'type': 'ACTUAL'}}, 'outcomesModule': {'otherOutcomes': [{'measure': 'Exploratory analysis of clinician prediction of renal failure within 48 hours compared to published ML models', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': "This exploratory outcome compares the area under the receiver operating characteristic curve (auROC) for two methods of predicting renal failure within 48 hours of each assessment time point: (1) ICU clinicians' risk estimates, and (2) previously published machine learning (ML) models applied retrospectively. For each assessment, we compute the auROC separately for clinicians and for the ML model. The difference in auROCs is evaluated using the same methodological framework as the primary outcome."}, {'measure': 'Exploratory analysis of clinician mortality prediction compared to published ML models', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': "All-cause mortality prediction by McNemar's test. Evaluated and tested using the performance of respective assessments by clinicians (binary response) and corresponding (paired) predictions by a machine learning model (probability prediction reduced to a binary response) trained on historical data for the three individual binary outcomes of all-cause mortality \\<28-day, \\<6-months, \\<12-months. For each horizon separately the machine-based probabilities are thresholded to match the sensitivity of the clinicians, a McNemar's test is performed to test for a significant difference in predictive capabilities and p-values will be adjusted to account for multiple testing if necessary."}, {'measure': 'Exploratory analysis of predictive accuracy of treating physicians versus treating nurses', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': 'Evaluated and tested for the same as the primary, secondary and first exploratory outcomes (i.e., circulatory, respiratory, and renal failure risks) using the respective assessments by treating physicians and treating nurses with paired time-point assessments.'}, {'measure': 'Comparison of predictive accuracy of treating versus non-treating physicians', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': 'Evaluated and tested for the same as the primary, secondary and first exploratory outcomes (i.e., circulatory, respiratory, and renal failure risks) using the respective assessments by clinicians (treating) and solely relying on EHR data (non-treating) with paired time-point assessments.'}, {'measure': 'Calibration analysis of clinician prediction scores', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': "Calibration analysis of clinician prediction scores. The study assesses the prediction capabilities of a collective of clinicians. However, humans might be ill-calibrated amongst each other with respect to providing probability estimates. Assessing the collective's performance without calibration of the individuals amongst each other might underestimate the actual predictive capabilities of the clinicians if they were well-calibrated. We propose to re-assess the primary and secondary outcomes but additionally perform a risk score calibration amongst physicians."}, {'measure': 'Exploratory analysis of patterns (e.g., systematic errors/biases) of predictive performance in human and machine assessors', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': '1. Influencing factors for the prediction capabilities of clinicians such as clinician seniority, clinician role (incl. nurse vs physician), time of day, weekday, workload, order of patients seen and other collected information on participating clinicians.\n2. Systematic errors and biases of both human expert assessments and machine learning based predictions on patient age, diagnostic group, or any of the collected patient-specific or clinician-related collected variables.'}], 'primaryOutcomes': [{'measure': 'Clinician prediction of circulatory failure within 8 hours compared to published ML models', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': "This outcome compares the area under the receiver operating characteristic curve (auROC) for two methods of predicting circulatory failure within 8 hours of each assessment time point: (1) ICU clinicians' risk estimates, and (2) previously published machine learning (ML) models applied retrospectively. For each assessment, we compute the auROC separately for clinicians and for the ML model for the same time points and patients. The difference in auROC (clinician minus ML) is the main measure of interest, evaluated under a non-inferiority framework with a margin of 0.025."}], 'secondaryOutcomes': [{'measure': 'Clinician prediction of respiratory failure within 24 hours compared to published ML models', 'timeFrame': 'Assessments are collected within the first 72 hours following admission.', 'description': "This outcome compares the area under the receiver operating characteristic curve (auROC) for two methods of predicting respiratory failure within 24 hours of each assessment time point: (1) ICU clinicians' risk estimates, and (2) previously published machine learning (ML) models applied retrospectively. For each assessment, we compute the auROC separately for clinicians and for the ML model. The difference in auROCs (clinician minus ML) is the main measure of interest, evaluated using the same methodological framework as the primary outcome."}]}, 'oversightModule': {'isUsExport': False, 'oversightHasDmc': False, 'isFdaRegulatedDrug': False, 'isFdaRegulatedDevice': False}, 'conditionsModule': {'keywords': ['Machine learning', 'Intensive care', 'artificial intelligence'], 'conditions': ['Circulatory Failure', 'Respiratory Failure', 'Renal Failure']}, 'descriptionModule': {'briefSummary': 'During this observational study, the investigators aim to assess the ability of ICU clinicians to predict the risk of impending organ failure and retrospectively compare it to the performance of previously published machine learning models. The central hypothesis of this study is that the treating physician can predict impending organ failure in adult ICU patients with similar accuracy as the best previously publishes machine learning models.', 'detailedDescription': "In this observational study, clinician's (physicians and nurses) assessment of the estimated imminent organ failure risk in an ICU setting are prospectively collected. Circulatory failure is investigated in the primary objective, and respiratory failure, renal failure, and mortality are investigated in secondary objectives. These assessments investigate the predictive performance and influencing factors for clinician prediction. The assessments will be collected in questionnaires and be performed by the clinicians directly involved in the patient treatment and by clinicians who are not actively responsible for the patient treatment. Furthermore, this study aims to benchmark these risk assessments made by healthcare professionals against retrospectively generated AI risk scores for the same patients and timepoints. The AI risk scores will be calculated retrospectively from a set of models from a systematic search of the current literature. The AI models that will be employed for this analysis will be identified as indicated by a systematic review protocol and must satisfy the following two criteria: they do not require any data beyond what is routinely collected during an ICU stay and may be accessed as open source. Such a comparison is vital for the understanding of the relative accuracy and reliability of AI-based predictions in the context of organ failure risk compared to human performance. The data and findings from this study are anticipated to provide evidence for the clinical utility of AI-based risk scores and pave the way for future research into the optimization of AI systems for healthcare applications."}, 'eligibilityModule': {'sex': 'ALL', 'stdAges': ['ADULT', 'OLDER_ADULT'], 'minimumAge': '18 Years', 'samplingMethod': 'PROBABILITY_SAMPLE', 'studyPopulation': 'Adult Intensive Care Unit at the University Hospital Bern', 'healthyVolunteers': False, 'eligibilityCriteria': "Inclusion Criteria:\n\n* patient minimum age of 18 years\n* emergency admission to the ICU\n* arterial line in place\n\nExclusion Criteria:\n\n* documented refusal (on the general consent form) to participate to clinical research\n* patients with neurologic conditions that impair the patient's level of consciousness (including, but not limited to stroke, traumatic brain injury, intracranial hemorrhage, CNS infections; except polytrauma)\n* patients on mechanical circulatory support systems (IABP, VA-ECMO, Impella, VAD) or extracorporeal membrane oxygenation (VV-ECMO) at any time during their ICU stay;\n* patients receiving end-of-life care or are admitted for the sole purpose of evaluating organ donation"}, 'identificationModule': {'nctId': 'NCT06814327', 'acronym': 'AI4ICU-Obs', 'briefTitle': 'Assessment of Organ Failure Risk Predictions in ICU', 'organization': {'class': 'OTHER', 'fullName': 'ETH Zurich'}, 'officialTitle': 'Prospective Assessment of Risk Predictions of Organ Failure in the Intensive Care Unit - Comparing Accuracy of Human and AI Risk Predictions', 'orgStudyIdInfo': {'id': 'BASEC-01046'}}, 'armsInterventionsModule': {'armGroups': [{'label': 'Adult ICU patients', 'description': 'Adult ICU patients'}]}, 'contactsLocationsModule': {'locations': [{'zip': '3010', 'city': 'Bern', 'state': 'Canton of Bern', 'country': 'Switzerland', 'facility': 'University Hospital Inselspital, Berne', 'geoPoint': {'lat': 46.94809, 'lon': 7.44744}}], 'overallOfficials': [{'name': 'Martin Faltys, Dr. med.', 'role': 'PRINCIPAL_INVESTIGATOR', 'affiliation': 'Insel Gruppe AG, University Hospital Bern'}]}, 'ipdSharingStatementModule': {'ipdSharing': 'UNDECIDED'}, 'sponsorCollaboratorsModule': {'leadSponsor': {'name': 'ETH Zurich', 'class': 'OTHER'}, 'collaborators': [{'name': 'Insel Gruppe AG, University Hospital Bern', 'class': 'OTHER'}], 'responsibleParty': {'type': 'SPONSOR'}}}}