This document shows the comparison results for Cheyne / Stokes scoring between the automatic analysis in Noxturnal and manually scored recordings by an RPSGT.
For this comparison 9 studies were used that represented patients with minimal Cheyne / Stokes breathing to patients with very severe Cheyne / Stokes breathing.
The Automatic scoring was compared to the scoring results from an RPSGT.
The following plots compare the duration of Cheyne / Stokes markers as percentage of recording duration.
The first plot plots the duration percentages from the automatic scoring against the same values from the manual scoring.
The second plot shows the Bland-Altman comparison.
From the 9 recorings we get a correlation of 99% with a Bland-Altman plot showing good agreement between the two. On average the manual scoring is scoring 2.3% more of each recording as Cheyne / Stokes with the 95% confidence interval ranging from +8.5% for the manual scoring down to -3.8% with a standard deviation of 3.1%. The two methods (manual and automatic) are substantially equivalent.
Comparison on using automatic vs. manual scoring in the classification of patient population
To estimate whether a patient needs further investigation or treatment for Cheyne Stokes breathing a threshold may be used.
There is a practice of using a threshold of 1/3 of the study to establish a diagnosis of primary Cheyne-Stokes. We compared the classification of automatic vs. manual scoring using this threshold as well as investigating thresholds of ¼ 1/5 and 1/2 . From these thresholds we calculated the sensitivity, specificity and true and negative predictive values.
By comparing the Cheyne-Stokes percentage values to a threshold and determining positive/negative results from that comparison, we calculated True Positives (TP), True Negatives (TN), False Positives (FP) and False Negatives (FN). From those values we calculated: Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) as comparison parameters between automatic and manual scoring of events.
We define a true positive (TP), true negative (TN), false positive (FP) and false negative (FN) in the following manner:
- TP = both automatic and manual percentage values above or equal to threshold.
- TN = both automatic and manual percentage values below threshold
- FN = automatic below threshold but manual above or equal to threshold
- FP = automatic above or equal to threshold but manual below threshold.
From the TP, TN, FN and FP we can calculate Sensitivity, Specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV) from the formulas:
- Sensitivity = TP / (TP + FN)
- Specificity = TN / (TN + FP)
- PPV = TP / (TP + FP)
- NPV = TN / (TN + FN)
Cheyne-Stokes percentage threshold of 33.3%
By using a percentage threshold of 33.3% the following results were obtained from the comparison:
- True Positives = 6
- False Positives = 0
- True Negatives = 3
- False Negatives = 0
The following table summarizes the results:
As seen in table 1, the classification is correct in all cases resulting in a 100% Sensitivity, Specificity, PPV and NPV.
Other Cheyne-Stokes percentage thresholds
We evaluated other Cheyne-Stokes percentage thresholds. Since one-third (33.3%) seems in general use, we also looked at one-fourth (25%), one fifth (20%), one sixth (16.6%) and half (50%) as well as 10%, 20% and 40%.
- Thresholds 25%, 40% and 50% gave perfect results for all values, same as for the 33.3% threshold.
Thresholds 20%, 16.6% and 10% gave one false positive which, because of how few a recordings were used, resulted in a specificity values of 66.7%, 66.7% and 50% respectively with Sensitivity of 100% in all cases.
The comparison between the automatic scoring of Cheyne-Stokes in Noxturnal and manual scoring of the same markers revealed that the Noxturnal 3.1 application scored Cheyne-Stokes in a substantially equivalent manner a human (RPSGT) scorer.
The correlation and Bland & Altman plots indicate substantial equivalence and the recognized threshold of 33.3% Cheyne-Stokes resulted in a 100% Sensitivity, Specificity, PPV and NPV.