Technical Summary of Updates in SomaHRD v1.5
This article outlines the primary changes implemented in the SomaHRD v1.5 workset, focusing on the metrics for Genomic Instability (GI) and the logic for sample classification.
It is meant as an overview of the changes and pipeline performances. If you need regulatory documentation of the performances, please reach out to our Customer Care team through the "Contact us" button.
1. New Metric for Result Certainty: GI Dispersion
The primary update in v1.5 is the replacement of the "GI Confidence Interval" with a new metric called GI Dispersion. This metric provides a more detailed assessment of the consistency of the internal prediction scores.
-
How it worked in v1.4: The confidence interval was based on a limited range (the 40th to 60th percentiles) of the 100 internal prediction scores.
-
How it works in v1.5: GI Dispersion is now calculated using the Shannon entropy of all 100 scores. A lower dispersion value indicates strong agreement among the scores (a high-confidence result), while a higher value indicates the scores are more spread out (greater uncertainty).
2. Refined Logic for "Non-conclusive" Results
The rules for flagging a sample as Non-conclusive have been refined to be more specific, incorporating the new GI Dispersion metric.
-
Previous Logic (v1.4): A sample was flagged as Non-conclusive if its GI confidence interval crossed the decision threshold of 0.5.
-
New Logic (v1.5): A sample is now flagged as Non-conclusive when two conditions are met simultaneously:
-
The GI Dispersion is high (greater than 2.7).
-
The GI Probability score falls within a specific range (0.37 to 0.59).
-
In addition, a new warning flag has been introduced to highlight borderline cases. This flag is applied to samples that have a high GI Dispersion (greater than 2.6) but do not meet the full criteria for a Non-conclusive status.
3. Validation Protocol and Performance Metrics
The performance of the GiS module was evaluated on the PAOLA cohort by comparing its classifications against Myriad’s reference status. A total of 329 samples were included for the Positive/Negative comparison after excluding samples with a Myriad status of "Non-conclusive".
Sensitivity
Sensitivity measures the system's ability to accurately detect the targeted analyte. It is calculated as the proportion of true positives (TP) correctly identified while minimizing the false negatives (FN).
In the specific case of the GiS score, the sensitivity is determined using a threshold-based classification. We used reference samples with known positive or negative GiS statuses for performance assessment. The module categorized outputs as:
-
Positive: GiS score was above 0.5.
-
Negative: GiS score was 0.5 or lower.
-
Non-conclusive: Internal criteria for this classification were met.
For the purpose of this evaluation, "non-conclusive" outputs were treated as false positives, and cases exhibiting the signature's characteristics but scoring below the positive threshold were considered false negatives.
The data shows an increase in Sensitivity in v1.5, from 95.6% to 96.5%.
Precision
Precision measures the ability of the system to detect only true events while minimizing false positives (FP. It is calculated as the proportion of positive results that are truly correct, reflecting the system's capacity to minimize false positives (FP).
For the GiS score, precision is calculated as the proportion of cases scoring above the threshold that truly exhibit the signature-related features. Cases that do not display these features but still receive a high score are considered false positives.
The data shows a stable precision between v1.4 and v1.5.