Ignite Creation Date:
2025-12-24 @ 4:57 PM
Ignite Modification Date:
2025-12-30 @ 6:14 AM
Study NCT ID:
NCT07187050
Status:
COMPLETED
Last Update Posted:
2025-09-22
First Post:
2025-09-02
Is NOT Gene Therapy:
True
Has Adverse Events:
False
Brief Title:
LLM-Assisted vs Manual Writing for Clinical Documentation: Effects on Time and Quality
Sponsor:
Organization:
Raw JSON
{'hasResults': False, 'derivedSection': {'miscInfoModule': {'versionHolder': '2025-12-24'}}, 'protocolSection': {'designModule': {'phases': ['NA'], 'studyType': 'INTERVENTIONAL', 'designInfo': {'allocation': 'RANDOMIZED', 'maskingInfo': {'masking': 'SINGLE', 'whoMasked': ['OUTCOMES_ASSESSOR']}, 'primaryPurpose': 'HEALTH_SERVICES_RESEARCH', 'interventionModel': 'PARALLEL', 'interventionModelDescription': 'This is a prospective, randomized, open-label, blinded-endpoint (PROBE) trial.'}, 'enrollmentInfo': {'type': 'ACTUAL', 'count': 21}}, 'statusModule': {'overallStatus': 'COMPLETED', 'startDateStruct': {'date': '2025-02-18', 'type': 'ACTUAL'}, 'expandedAccessInfo': {'hasExpandedAccess': False}, 'statusVerifiedDate': '2025-09', 'completionDateStruct': {'date': '2025-07-16', 'type': 'ACTUAL'}, 'lastUpdateSubmitDate': '2025-09-18', 'studyFirstSubmitDate': '2025-09-02', 'studyFirstSubmitQcDate': '2025-09-18', 'lastUpdatePostDateStruct': {'date': '2025-09-22', 'type': 'ESTIMATED'}, 'studyFirstPostDateStruct': {'date': '2025-09-22', 'type': 'ESTIMATED'}, 'primaryCompletionDateStruct': {'date': '2025-03-14', 'type': 'ACTUAL'}}, 'outcomesModule': {'primaryOutcomes': [{'measure': 'Average time spent creating each document', 'timeFrame': 'On one study day within 2 weeks after enrollment', 'description': 'In the CocktailAI \\& Modification group and the Control group, the time spent creating documents is measured in seconds. In the CocktailAI group, the time required for document generation is measured in seconds.'}], 'secondaryOutcomes': [{'measure': 'Document quality assessment', 'timeFrame': 'On one study day within 2 weeks after enrollment', 'description': 'Blinded to group allocation, ophthalmology experts evaluate the documents using pre-specified criteria defined before study initiation. These criteria are developed based on six domains: Medical Accuracy, Language, Conciseness, Presence of Hallucinations, Validity for Clinical Use, and Possibility of Harm. Most domains are assessed on a three-point scale, whereas Presence of Hallucinations is evaluated dichotomously (present or absent). In addition to these domain-specific ratings, experts provide a subjective overall score on a 10-point scale (with higher scores indicating better quality) and are asked to guess which study group the document belongs to.'}]}, 'oversightModule': {'oversightHasDmc': False, 'isFdaRegulatedDrug': False, 'isFdaRegulatedDevice': False}, 'conditionsModule': {'keywords': ['clinician-in-the-loop', 'Large Language Model', 'Clinical Documentation', 'Electronic Health Record'], 'conditions': ['Clinician-in-the-loop', 'Clinical Documentation', 'Large Language Model']}, 'descriptionModule': {'briefSummary': 'The goal of this clinical trial is to learn whether an LLM-assisted writing workflow can reduce the time to complete hospital discharge summaries and discharge referrals and maintain or improve document quality compared with writing from scratch by clinicians. The study used six simulated patient records (no real patient data).\n\nThe main questions it aims to answer are:\n\n* Does the LLM-assisted writing workflow reduce the time needed to complete each document compared with manual writing?\n* Does the LLM-assisted writing workflow improve (or at least maintain) document quality compared with manual writing, as rated by blinded experts?\n\nResearchers will compare LLM-assisted versus manual writing to see if the LLM-assisted approach is faster and has equal or better quality. LLM-only drafts (unedited first drafts) will be evaluated as a separate third group to understand the baseline quality of LLM output without clinician edits.\n\nParticipants will create two documents-a discharge summary and a discharge referral-for each of six simulated cases. Those assigned to CocktailAI \\& Modification group will use an LLM assistant (called CocktailAI) to generate a first draft for each document and then review and edit it to finalize; those assigned to the control group will write each document from scratch without LLM assistance.'}, 'eligibilityModule': {'sex': 'ALL', 'stdAges': ['CHILD', 'ADULT', 'OLDER_ADULT'], 'healthyVolunteers': False, 'eligibilityCriteria': 'Inclusion Criteria:\n\n* Ophthalmologists at Kyoto University Hospital\n* Junior residents, senior residents, graduate students, board-certified ophthalmologists\n* Physicians who confirm that they do not routinely use CocktailAI for clinical documentation and provide informed consent after receiving an explanation of the study.'}, 'identificationModule': {'nctId': 'NCT07187050', 'briefTitle': 'LLM-Assisted vs Manual Writing for Clinical Documentation: Effects on Time and Quality', 'organization': {'class': 'OTHER', 'fullName': 'Kyoto University, Graduate School of Medicine'}, 'officialTitle': 'Evaluating an LLM-Assisted Workflow for Clinical Documentation: A Pilot Randomized Controlled Trial on Time and Quality', 'orgStudyIdInfo': {'id': 'clinician-in-the-loop_workflow'}}, 'armsInterventionsModule': {'armGroups': [{'type': 'EXPERIMENTAL', 'label': 'CocktailAI & Modification arm', 'description': 'Participants log into the study web application using a unique ID and their name. For each simulated patient case, they press "Start" to unlock the case record and initiate timing. After viewing the patient information, clinicians use CocktailAI to generate a first draft of the discharge summary. They then review and edit the draft and submit the final document by pressing "Submit." For the discharge referral of the same case, clinicians follow the same procedure. Copy-and-paste from the case record is permitted. This sequence is repeated for six simulated cases.', 'interventionNames': ['Other: Template-Based LLM Assistant']}, {'type': 'ACTIVE_COMPARATOR', 'label': 'Control arm', 'description': 'Participants log into the study web application using a unique ID and their name. For each simulated patient case, they press "Start" to unlock the case record and initiate timing. After viewing the patient information, clinicians write the discharge summary from scratch and submit the document by pressing "Submit." They then write the discharge referral for the same case from scratch and submit it. Copy-and-paste from the case record is permitted. This sequence is repeated for six simulated cases.', 'interventionNames': ['Other: Manual Writing']}, {'type': 'ACTIVE_COMPARATOR', 'label': 'CocktailAI arm', 'description': 'The LLM assistant generates drafts of discharge summaries and discharge referrals directly from the simulated patient records, with no clinician review or edits. These drafts are the unedited first drafts produced in the CocktailAI \\& Modification arm (captured before any clinician edits) and are included as a separate group.', 'interventionNames': ['Other: Template-Based LLM Assistant']}], 'interventions': [{'name': 'Template-Based LLM Assistant', 'type': 'OTHER', 'description': 'This study uses CocktailAI, a template-based LLM assistant co-developed by the Department of Ophthalmology and Visual Sciences, Kyoto University Graduate School of Medicine, and Fitting Cloud Inc. (Kyoto, Japan). It is designed to extract relevant information from EHRs using LLMs and embed the extracted content into predefined templates. In this trial, the inputs are six simulated patient records (no real patient data). Text generation uses Gemini-2.0-flash-lite. Templates for discharge summaries and discharge referrals are pre-defined by a team member.', 'armGroupLabels': ['CocktailAI & Modification arm', 'CocktailAI arm']}, {'name': 'Manual Writing', 'type': 'OTHER', 'description': 'The same document templates are provided; however, all LLM instruction prompts are removed in advance. Clinicians manually write the documents, following the template structure, for each of the six simulated cases.', 'armGroupLabels': ['Control arm']}]}, 'contactsLocationsModule': {'locations': [{'zip': '606-8507', 'city': 'Kyoto', 'country': 'Japan', 'facility': 'Department of Ophthalmology and Visual Sciences Kyoto University Graduate School of Medicine 54 Shogoin, Kawahara, Sakyo', 'geoPoint': {'lat': 35.02107, 'lon': 135.75385}}]}, 'sponsorCollaboratorsModule': {'leadSponsor': {'name': 'Kyoto University, Graduate School of Medicine', 'class': 'OTHER'}, 'collaborators': [{'name': 'Fitting Cloud Inc.', 'class': 'UNKNOWN'}], 'responsibleParty': {'type': 'PRINCIPAL_INVESTIGATOR', 'investigatorTitle': 'Principal Investigator', 'investigatorFullName': 'Tomohiro Takayama', 'investigatorAffiliation': 'Kyoto University, Graduate School of Medicine'}}}}