Ignite Creation Date:
2025-12-25 @ 3:19 AM
Ignite Modification Date:
2026-01-01 @ 4:52 AM
Study NCT ID:
NCT06624605
Status:
COMPLETED
Last Update Posted:
2024-10-03
First Post:
2024-09-30
Is NOT Gene Therapy:
True
Has Adverse Events:
False
Brief Title:
Enhancing Interdisciplinary Understanding of Ophthalmology Notes Through a Local Large Language Model
Sponsor:
Organization:
Raw JSON
{'hasResults': False, 'derivedSection': {'miscInfoModule': {'versionHolder': '2025-12-24'}, 'conditionBrowseModule': {'meshes': [{'id': 'D003142', 'term': 'Communication'}], 'ancestors': [{'id': 'D001519', 'term': 'Behavior'}]}}, 'protocolSection': {'designModule': {'phases': ['NA'], 'studyType': 'INTERVENTIONAL', 'designInfo': {'allocation': 'RANDOMIZED', 'maskingInfo': {'masking': 'NONE'}, 'primaryPurpose': 'HEALTH_SERVICES_RESEARCH', 'interventionModel': 'PARALLEL', 'interventionModelDescription': 'Randomized study assessing the efficacy of Large Language Model-generated Plain Language Summaries of Ophthalmology notes in increasing non-ophthalmologist note understanding. Notes were randomized to intervention (Summary + Original Note) or control (Original note only) arms. Non-ophthalmology providers were sent surveys assessing note comprehension for both control and treatment arm notes, and Ophthalmologists were sent surveys for intervention arm notes regarding Summary accuracy and other attributes. Conducted as a Quality Improvement initiative.'}, 'enrollmentInfo': {'type': 'ACTUAL', 'count': 851}}, 'statusModule': {'overallStatus': 'COMPLETED', 'startDateStruct': {'date': '2024-02-01', 'type': 'ACTUAL'}, 'expandedAccessInfo': {'hasExpandedAccess': False}, 'statusVerifiedDate': '2024-10', 'completionDateStruct': {'date': '2024-05-31', 'type': 'ACTUAL'}, 'lastUpdateSubmitDate': '2024-10-01', 'studyFirstSubmitDate': '2024-09-30', 'studyFirstSubmitQcDate': '2024-10-01', 'lastUpdatePostDateStruct': {'date': '2024-10-03', 'type': 'ACTUAL'}, 'studyFirstPostDateStruct': {'date': '2024-10-03', 'type': 'ACTUAL'}, 'primaryCompletionDateStruct': {'date': '2024-05-31', 'type': 'ACTUAL'}}, 'outcomesModule': {'otherOutcomes': [{'measure': 'Presence of Errors, Type of Errors, and Potential Harm of Errors', 'timeFrame': 'Assessed at a single time point &amp;amp;amp;amp;lt;24 hr following enrollment and again at a single time point 8 weeks after enrollment', 'description': 'Presence of any incorrect or missing information in the Plain Language Summary, error rates, types of errors, and potential harm rating.\n\nTypes of errors were reported as:\n\nHPI Exam findings Diagnosis Plan Explanation of the patient\\&amp;#39;s overall eye health Other N/A\n\nLikelihood of harm was graded on the following scale:\n\nLow risk Medium risk High risk N/A\n\nPotential harm rating was graded on the following scale:\n\nMild or moderate harm Death or severe harm'}], 'primaryOutcomes': [{'measure': 'Non-Ophthalmologist Note Comprehension and Satisfaction', 'timeFrame': 'From enrollment to 8 weeks after enrollment', 'description': 'Assessed via survey responses evaluating understanding of the patient\\&amp;amp;amp;#39;s ophthalmology diagnosis, satisfaction with the note, and overall preference between Standard Ophthalmology Note and Standard Ophthalmology Note + Plain Language Summary.\n\nDegree of understanding was assessed on the following Likert scale:\n\nNot at all Neutral Moderately A great deal\n\nSatisfaction was graded on the following Likert scale:\n\nNot satisfied at all Somewhat unsatisfied Neutral Somewhat satisfied Satisfied\n\nPreference between note types was assessed on the following Likert scale:\n\nStandard Note, a great deal Standard Note, somewhat Neutral Plain Language Summary, somewhat Plain Language Summary, a great deal'}], 'secondaryOutcomes': [{'measure': 'Ophthalmologist Evaluation of Summary Accuracy and Time burden', 'timeFrame': 'Assessed at a single time point <24 hr after enrollment', 'description': 'Assessed via survey responses regarding Plain Language Summary accuracy in reflecting the Standard Ophthalmology Note findings, time spent reviewing/editing, and perceived burden.\n\nAccuracy in reflecting findings was graded on the following Likert scale:\n\nNot at all Neutral A little A great deal\n\nTime spent reviewing/editing was reported using the following options:\n\n\\<1 minute\n\n1. minute\n2. minutes\n3. minutes\n4. minutes\n5. or more minutes\n\nPerceived burden was graded on the following Likert scale:\n\nNot at all Neutral A little A great deal'}, {'measure': 'Semantic and Linguistic Quality of Plain Language Summaries', 'timeFrame': 'Assessed at a single time point &lt;24 hr following enrollment', 'description': 'Flesch Reading Ease\n\nScale: 0 (poor) to 100 (good)'}, {'measure': 'Semantic and Linguistic Quality of Plain Language Summaries', 'timeFrame': 'Assessed at a single time point &lt;24 hr following enrollment', 'description': 'Flesch-Kincaid Grade Level\n\nScale: 0 (most difficult to read, highest grade level) to 80 (easiest to read, lowest grade level)'}, {'measure': 'Semantic and Linguistic Quality of Plain Language Summaries', 'timeFrame': 'Assessed at a single time point &lt;24 hr following enrollment', 'description': 'Semantic Measure of Gobbledygook (SMOG) Index\n\nScale: 0 (easiest to read) to 11 (most difficult to read)'}, {'measure': 'Semantic and Linguistic Quality of Plain Language Summaries', 'timeFrame': 'Assessed at a single time point &amp;lt;24 hr following enrollment', 'description': 'BERTScore Measure of Linguistic Similarity\n\nScale: -1 (least similar) to +1 (most similar)'}, {'measure': 'Semantic and Linguistic Quality of Plain Language Summaries', 'timeFrame': 'Assessed at a single time point &amp;lt;24 hr following enrollment', 'description': 'SBERT Cosine Similarity Measure of Linguistic Similarity\n\nScale: -1 (least similar) to +1 (most similar)'}]}, 'oversightModule': {'isUsExport': False, 'oversightHasDmc': False, 'isFdaRegulatedDrug': False, 'isFdaRegulatedDevice': False}, 'conditionsModule': {'conditions': ['Communication', 'Artificial Intelligence (AI)', 'Artificial Intelligence Technology', 'Interdisciplinary Communication']}, 'descriptionModule': {'briefSummary': 'This prospective, randomized controlled trial evaluated the efficacy of adding Large Language model (LLM)-generated Plain Language Summaries (PLSs) to Standard Ophthalmology Notes (SONs) in enhancing comprehension among non-ophthalmology providers. The study utilized surveys to assess non-ophthalmology providers\\' comprehension and satisfaction with the notes and ophthalmologists\\' evaluation of PLS accuracy, safety, and time burden. An objective semantic and linguistic analysis of the PLSs was also conducted.'}, 'eligibilityModule': {'sex': 'ALL', 'stdAges': ['ADULT', 'OLDER_ADULT'], 'minimumAge': '18 Years', 'healthyVolunteers': False, 'eligibilityCriteria': 'Inclusion Criteria:\n\n* Evaluation by an Ophthalmologist in the Mayo Clinic Department of Ophthalmology in Rochester, MN in either the inpatient or outpatient setting between Feb 1, 2024 - May 31, 2024.\n\nExclusion Criteria:\n\n* No in-network non-Ophthalmologist care providers in the medical record\n* Procedure-only visits containing only a procedure note'}, 'identificationModule': {'nctId': 'NCT06624605', 'briefTitle': 'Enhancing Interdisciplinary Understanding of Ophthalmology Notes Through a Local Large Language Model', 'organization': {'class': 'OTHER', 'fullName': 'Mayo Clinic'}, 'officialTitle': 'Enhancing Interdisciplinary Understanding of Ophthalmology Notes Through Local Large Language Model', 'orgStudyIdInfo': {'id': '24-000550'}}, 'armsInterventionsModule': {'armGroups': [{'type': 'NO_INTERVENTION', 'label': 'Control', 'description': 'Participants receive standard ophthalmology notes without any additions.'}, {'type': 'EXPERIMENTAL', 'label': 'Intervention', 'description': 'Participants receive standard ophthalmology notes with an appended LLM-generated plain language summary, reviewed and edited by the ophthalmologist. The plain language summary is clearly marked as generated as part of a Quality Improvement initiative.', 'interventionNames': ['Other: Large Language Model-generated Plain Language Summary of Ophthalmology notes']}], 'interventions': [{'name': 'Large Language Model-generated Plain Language Summary of Ophthalmology notes', 'type': 'OTHER', 'description': 'Prospective, randomized Quality Improvement study with real-world implementation of Large Language Model-generated Plain Language Summaries of Ophthalmology notes.', 'armGroupLabels': ['Intervention']}]}, 'contactsLocationsModule': {'locations': [{'zip': '55905', 'city': 'Rochester', 'state': 'Minnesota', 'country': 'United States', 'facility': 'Mayo Clinic', 'geoPoint': {'lat': 44.02163, 'lon': -92.4699}}], 'overallOfficials': [{'name': 'John J Chen, MD, PhD', 'role': 'STUDY_DIRECTOR', 'affiliation': 'Mayo Clinic'}]}, 'ipdSharingStatementModule': {'infoTypes': ['STUDY_PROTOCOL', 'CSR', 'ANALYTIC_CODE'], 'timeFrame': 'Beginning after publication with no end date', 'ipdSharing': 'YES', 'description': 'All collected IPD.', 'accessCriteria': 'The underlying data supporting the findings of this study can be made available to clinical investigators and researchers upon request. Written requests for data sharing including an analysis plan will be required before approval. These requests will be individually assessed in consultation with the study team leads and co-investigators as appropriate. If other investigators are interested in performing additional analyses, these requests can be made to the corresponding author (PDT) and analyses will be performed in collaboration with the Mayo Clinic. In all cases, any data and materials to be shared will be released via a Material Transfer Agreement. Individual-level data will be available and data sharing will ensure that the rights and privacy of individuals participating in the research always remains protected. The anticipated time frame to respond to initial data requests is 1 month.'}, 'sponsorCollaboratorsModule': {'leadSponsor': {'name': 'John J Chen', 'class': 'OTHER'}, 'responsibleParty': {'type': 'SPONSOR_INVESTIGATOR', 'investigatorTitle': 'Principal Investigator', 'investigatorFullName': 'John J Chen', 'investigatorAffiliation': 'Mayo Clinic'}}}}