Ignite Creation Date:
2024-10-26 @ 3:35 PM
Last Modification Date:
2024-10-26 @ 3:35 PM
Study NCT ID:
NCT06514248
Status:
COMPLETED
Last Update Posted:
None
First Post:
2024-07-05
Brief Title:
Clinical Evaluation of the V5-LU01 AI Software in Thoracic CT for V5med Inc
Sponsor:
None
Organization:
None
Study Overview
Official Title:
Evaluation of V5 Lung AI Software With A Two-Arm Comparative Reader Study
Status:
COMPLETED
Status Verified Date:
2024-07
Last Known Status:
None
Delayed Posting:
No
If Stopped, Why?:
Not Stopped
Has Expanded Access:
No
If Expanded Access, NCT#:
N/A
Has Expanded Access, NCT# Status:
N/A
Acronym:
None
Brief Summary:
STUDY DESIGN
This is a two-arm retrospective multi-reader multi-case MRMC randomized reader study
OBJECTIVE
Primary The primary objective of this clinical study is to prove that radiologists performance aided with V5med Lung AI is superior to the unaided for detecting qualified lung nodules
Secondary The secondary objective of this clinical study is to prove that the radiologists reading time is not significantly increased when aided with V5med Lung AI
Addition Objectives To prove that the agreement eg in kappa correlation coefficient between experts and radiologists Lung-RADS score aided with V5med Lung AI is superior to the unaided
NUMBER OF SUBJECTS
Retrospective CT studies from approximately 350 patients will be included in the study with approximately 170 true positive cases and 180 normal cases
PRIMARY ENDPOINTS
Scores given by the radiologists with and without V5med Lung AI will be recorded and compared to the true status of the study-cases The frequency of the scores for each method Aided Unaided will be tabulated and LROC curves constructed along with sensitivity specificity PPV NPV and clinical actions Additionally machine nodule detection rate and false positives per patient on normal cases will be measured
PATIENT POPULATION
The study will target approximately two hundred 200 patients whose CT lung nodules were shown to be cancer and one hundred and eighty 150 patients who have no lung nodule greater than 4mm The patient population will be consistent with the national lung cancer screening protocols
This is a two-arm retrospective multi-reader multi-case MRMC randomized reader study
OBJECTIVE
Primary The primary objective of this clinical study is to prove that radiologists performance aided with V5med Lung AI is superior to the unaided for detecting qualified lung nodules
Secondary The secondary objective of this clinical study is to prove that the radiologists reading time is not significantly increased when aided with V5med Lung AI
Addition Objectives To prove that the agreement eg in kappa correlation coefficient between experts and radiologists Lung-RADS score aided with V5med Lung AI is superior to the unaided
NUMBER OF SUBJECTS
Retrospective CT studies from approximately 350 patients will be included in the study with approximately 170 true positive cases and 180 normal cases
PRIMARY ENDPOINTS
Scores given by the radiologists with and without V5med Lung AI will be recorded and compared to the true status of the study-cases The frequency of the scores for each method Aided Unaided will be tabulated and LROC curves constructed along with sensitivity specificity PPV NPV and clinical actions Additionally machine nodule detection rate and false positives per patient on normal cases will be measured
PATIENT POPULATION
The study will target approximately two hundred 200 patients whose CT lung nodules were shown to be cancer and one hundred and eighty 150 patients who have no lung nodule greater than 4mm The patient population will be consistent with the national lung cancer screening protocols
Detailed Description:
Overview of the Study
A reader study with sixteen participating radiologists will be conducted A location-specific receiver operating characteristic LROC curve will be used to evaluate radiologists detection performance in detecting qualified lung nodules on chest CT scans with and without the use of the V5med Lung AI system The reading time for each case will also be recorded
The 16 study radiologists will be split into two groups X and Y The 350 chest CT scans will be split into two subsets A and B Group X radiologists will interpret subset A without AI and subset B with V5med Lung AI Group Y radiologist will interpret subset B without AI and subset A with AI After a minimum of one month each radiologist group will interpret the same images again but in the reverse order with regard to study modes without AI and with AI Please see Figures 1 and 2 for detailed arrangement of the radiologist interpretation sessions
During the baseline reading session the radiologist will mark each location of qualified nodules and assign a score If at least one suspicious nodule is observed the radiologist will also indicate a Lung-RADS score ie 2 3 4A 4B or 4X at the end of case interpretation
During the second reading ie aided reading session the radiologist will be presented with a chest CT with CADe marks displayed on the left window and the same chest CT without CADe marks displayed on the right window If the radiologist confirms a nodule the radiologist will mark the location on the chest CT displayed on the right window This may or may not correspond to the locations of the CADe marks as displayed on the left window The same as in the baseline study the radiologist will assign a score level of suspicion to each suspicious location and provide a Lung-RADS score for the chest CT case if at least one location on the CT is marked
Based on the suspicious locations and their scores level of suspicion assessed by study radiologists LROC curves can be constructed for both the baseline and the aided reads and the significance of any difference will be calculated The LOS score will also be used to calculate sensitivity specificity PPV and NPV
Number and types of Chest CT cases
Retrospective chest CT image series from approximately 350 cases will be included in the study Approximately two hundred 200 cases will have at least one nodule size greater than 4mm These cases will have radiology-prove report andor biopsy-confirmed cancers Also included as nodule images are those where a nodule was not detected at that time but was detected and acted on based on a subsequent CT These are the prior images where the nodule can be identified in the same location as on the current image confirmed by the radiologist expert panel using a majority of three as the decision criterion In addition approximately one hundred and fifty 150 cases will not have a nodule size greater than 4mm
The selected chest CT cases for the 2-arm comparative reader study randomly selected from a larger pool of CT cases will be enriched in the following way
1 Lung nodules cancers and benign nodules will be nodule size ranging from 4mm to 30 mm The proportion of nodules 20 mm or less may take majority of the cases with nodules since this lung lesion size range is where we expect the major impact of this software to be
2 Non-Solid ground glass nodules will be added to the study dataset based on availability to determine the performance of the system on non-solid nodules For this group to have sufficient cases we may include more benign non-malignant non-solid nodules
3 In this project VIRGINIA TECH study team will perform a machine standalone test of the V5med Lung AI algorithm followed by a reader performance evaluation study V5med Technologies will provide a system configured with the operating point set to be used for the reader studies and a configuration for an open system to be used for machine testing and to generate free-response ROC FROC curve
The Design of Two-Arm Comparative Study
The First Arm Study
Perform a baseline reader study without any special tool The reader is asked to mark all potential nodules not less than 4mm For each mark the size of region of interest ROI reader score level of suspicion and time spent will be recorded for the statistical analyses
The Second Arm Study
Perform another reader study using the V5med Lung AI as the aided read The reader is asked to mark all potential nodules not less than 4mm assisted by V5med Lung AI Some or majority of ROIs parameters will be pre-filled by V5med Lung AI The reader will make final determination for each location The marked location LOS score Lung-RADS score and time spent will be recorded for the statistical analyses and compared to the first arm study
The primary study hypothesis is that the detection of lung nodules with the use of V5med Lung AI and associated functions is superior to the detection of lung nodules without the use of V5med Lung AI as measured by the area under the LROC curve
A reader study with sixteen participating radiologists will be conducted A location-specific receiver operating characteristic LROC curve will be used to evaluate radiologists detection performance in detecting qualified lung nodules on chest CT scans with and without the use of the V5med Lung AI system The reading time for each case will also be recorded
The 16 study radiologists will be split into two groups X and Y The 350 chest CT scans will be split into two subsets A and B Group X radiologists will interpret subset A without AI and subset B with V5med Lung AI Group Y radiologist will interpret subset B without AI and subset A with AI After a minimum of one month each radiologist group will interpret the same images again but in the reverse order with regard to study modes without AI and with AI Please see Figures 1 and 2 for detailed arrangement of the radiologist interpretation sessions
During the baseline reading session the radiologist will mark each location of qualified nodules and assign a score If at least one suspicious nodule is observed the radiologist will also indicate a Lung-RADS score ie 2 3 4A 4B or 4X at the end of case interpretation
During the second reading ie aided reading session the radiologist will be presented with a chest CT with CADe marks displayed on the left window and the same chest CT without CADe marks displayed on the right window If the radiologist confirms a nodule the radiologist will mark the location on the chest CT displayed on the right window This may or may not correspond to the locations of the CADe marks as displayed on the left window The same as in the baseline study the radiologist will assign a score level of suspicion to each suspicious location and provide a Lung-RADS score for the chest CT case if at least one location on the CT is marked
Based on the suspicious locations and their scores level of suspicion assessed by study radiologists LROC curves can be constructed for both the baseline and the aided reads and the significance of any difference will be calculated The LOS score will also be used to calculate sensitivity specificity PPV and NPV
Number and types of Chest CT cases
Retrospective chest CT image series from approximately 350 cases will be included in the study Approximately two hundred 200 cases will have at least one nodule size greater than 4mm These cases will have radiology-prove report andor biopsy-confirmed cancers Also included as nodule images are those where a nodule was not detected at that time but was detected and acted on based on a subsequent CT These are the prior images where the nodule can be identified in the same location as on the current image confirmed by the radiologist expert panel using a majority of three as the decision criterion In addition approximately one hundred and fifty 150 cases will not have a nodule size greater than 4mm
The selected chest CT cases for the 2-arm comparative reader study randomly selected from a larger pool of CT cases will be enriched in the following way
1 Lung nodules cancers and benign nodules will be nodule size ranging from 4mm to 30 mm The proportion of nodules 20 mm or less may take majority of the cases with nodules since this lung lesion size range is where we expect the major impact of this software to be
2 Non-Solid ground glass nodules will be added to the study dataset based on availability to determine the performance of the system on non-solid nodules For this group to have sufficient cases we may include more benign non-malignant non-solid nodules
3 In this project VIRGINIA TECH study team will perform a machine standalone test of the V5med Lung AI algorithm followed by a reader performance evaluation study V5med Technologies will provide a system configured with the operating point set to be used for the reader studies and a configuration for an open system to be used for machine testing and to generate free-response ROC FROC curve
The Design of Two-Arm Comparative Study
The First Arm Study
Perform a baseline reader study without any special tool The reader is asked to mark all potential nodules not less than 4mm For each mark the size of region of interest ROI reader score level of suspicion and time spent will be recorded for the statistical analyses
The Second Arm Study
Perform another reader study using the V5med Lung AI as the aided read The reader is asked to mark all potential nodules not less than 4mm assisted by V5med Lung AI Some or majority of ROIs parameters will be pre-filled by V5med Lung AI The reader will make final determination for each location The marked location LOS score Lung-RADS score and time spent will be recorded for the statistical analyses and compared to the first arm study
The primary study hypothesis is that the detection of lung nodules with the use of V5med Lung AI and associated functions is superior to the detection of lung nodules without the use of V5med Lung AI as measured by the area under the LROC curve
Study Oversight
Has Oversight DMC:
None
Is a FDA Regulated Drug?:
None
Is a FDA Regulated Device?:
None
Is an Unapproved Device?:
None
Is a PPSD?:
None
Is a US Export?:
None
Is an FDA AA801 Violation?:
None