Research Article Volume 5 Issue 1
Progenity, USA
Correspondence: Jay Stoerker, Vice President of Research and Development, Progenity, USA, Tel 248 536 3248
Received: January 05, 2017 | Published: February 1, 2017
Citation: Stoerker J, Tarrier B, Reeder T, et al. On the importance of clinical follow-up in the establishment of non-invasive prenatal testing (NIPT) for laboratory developed tests. MOJ Proteomics Bioinform. 2017;5(1):14-20. DOI: 10.15406/mojpb.2017.05.00147
Non-invasive prenatal genetic testing (NIPT) is an advance in the detection of fetal chromosomal aneuploidies that analyzes cell-free DNA (cfDNA) in the blood of a pregnant woman. NIPT has quickly spread across the globe after introduction into clinical practice earlier in this decade. Virtually all professional societies currently recommend that NIPT be used as a screening test rather than a diagnostic method. Due to the extraordinarily high specificity and sensitivity, it is clearly an attractive alternative to conventional serum biochemical screening tests-the current standard of care.
We performed a clinical validation study of our Laboratory Developed Test (LDT) assessing equivalency to the verifi™ test by Illumina, Inc. The method uses next generation sequencing (NGS), and features interpretation based on the calculation of normalized chromosomal values (NCV). In our assay, samples from 3334 consecutive patients which had been tested with Illumina’s verifi test, were blinded by an independent third party and retested by the method developed for the Progenity LDT to be offered in our commercial laboratory. As an additional level of validation, we sought and achieved follow up contact on 1681 of the original Illumina cases, to further independently confirm our results and theirs.
In the analysis of equivalence between the two tests, 111 trisomy 21 samples, 43 trisomy 18 samples and 21 trisomy 13 samples were identified by both tests, indicating an equivalency of >99.9%. Although the results between the verifi “gold standard” test and our LDT proved identical (i.e., 100% concordance), the verifi test is not promoted by Illumina as having a 100% sensitivity and specificity. Therefore, certain discrepancies can only be appreciated by clinical follow up. In the NCV analyses, categories are created for “affected” or “suspected trisomy” based on the magnitude of the NCV value. Our findings suggest that the very few discrepancies were between the “aneuploidy suspected” categories of the verify reference standard and our LDT’s confirmed “not affected” or “affected” categories. Our conclusion is that NIPT LDTs must be validated by a process including clinical follow-up, not just equivalency to a reference testing method.
Keywords: cell free DNA, cfDNA, LDT, NGS, NIPT, in vitro, molecular pathology, accuracy, precision, analytical sensitivity, nano chemistry
NIPT, non-invasive prenatal genetic testing; cfDNA, cell-free DNA; LDT, laboratory developed test; NGS, next generation sequencing; NCV, normalized chromosomal values; NIPT, non-invasive prenatal testing; CAP, college of american pathology
The observation by Lo et al.,1 that cell-free fetal (cff) DNA fragments could be isolated and analyzed from the blood of pregnant women was made in 1997.1 Even though most references define cff DNA as fetal, studies have shown that its source is from apoptotic cells in the placenta.2 Digital PCR methods have demonstrated that a mixture of placental and maternal DNA in the maternal blood changes as pregnancy progresses, generally resulting in greater fractions of the fetal DNA component over time.3 In 2008, fetal aneuploidy detection using a two-step approach of shotgun sequencing of DNA followed by counting statistics was achieved.4 Shortly thereafter, non-counting methods based on SNPs were shown to be effective in NIPT.5 Multiple clinical trials were performed after 2010 that showed reliable detection of trisomies 21 and 18 after week 10 of pregnancy.6 The published results from these trials in peer-reviewed journals led to the commercial release of non-invasive prenatal testing (NIPT) for autosomal aneuploidy in 2011. Four laboratories in the U.S. (Sequenom, Inc.; Ariosa, Inc.; Natera, Inc.; and Illumina, Inc.) have been prominent in developing this field, and expanding it to include sex chromosome anomalies (SCA) and sub-chromosomal anomalies, also called microdeletions.7 Clinical performance in these CLIA-certified, a College of American Pathology (CAP)-accredited laboratory appears to be on par with prior published clinical validation studies, with high sensitivities and specificities and very high negative predictive values.8–10
Recently, Illumina has begun to develop and sell in-vitro diagnostic kits for NIPT and to license third-party laboratories wishing to develop and sell their own laboratory-developed (LDT) NIPT tests.11 An LDT is a type of in vitro diagnostic test that is designed, manufactured, and used within a single laboratory.12 By any reasonable definition, the practice of NIPT by NGS is “high complexity.” It is also “high risk” in that results impact pregnancy management, and some patients may choose pregnancy termination, with or without recommended diagnostic follow-up.12 Nevertheless, the FDA has not yet exercised direct “enforcement discretion” in the oversight of NIPT in the US.12 Rather, it has allowed introduction of new NIPT assays in the usual regulatory context of an LDT; namely, through CAP and CLIA. Regulatory standards applying to next-generation sequencing (NGS) for clinical tests are at an early stage of development in the U.S. A total of only 18 laboratory checklist requirements for the NGS analytic, “wet bench,” process and bioinformatics processes have been included within the College of American Pathology (CAP) molecular pathology checklist,13 contrasted to 79 requirements for molecular pathology, and a total of 223.
Implementing a new NIPT with proper regulatory compliance requires “…establishment of method performance specifications should provide evidence that the accuracy, precision, analytical sensitivity, and analytical specificity of the procedure is adequate to meet the clients’ needs as determined by the laboratory director...” as directed by CLIA 88 §493.1253 (b) (2). Agreements with Illumina allow for this. This regulation can be interpreted by the laboratory director as allowing a test which has shown a predetermined acceptable level of performance as measured against a reference method. However, the assay validation sections of the CAP checklists (COM.40300, COM.40400 and COM.40450) may create confusion in that “…validation may consist of data from manufacturer”.14 Laboratories new to the field may be tempted to quote Illumina performance information solely, such as the MELISSA study,15 which has been validated by clinical outcome data, in a transitive connection to their own tests after a method comparison study. However, CLIA requires the laboratories performing an LDT to cite their own validation results, not only that of the manufacturer or predicate assay. Our approach has complied with this directive, but has been different in adopting additional clinical follow-up within our validation process.
We present an assay validation here that is designed not only to show equivalence to a reference test in performance characteristics, but also to prove that the test is reliable using independent outcome data achieved by clinical follow-up. After a CLIA-compliant process of feasibility testing, optimization, and verification studies, we recruited a cohort of remnant samples (4032 of our clinical reference samples sent for commercial processing by Illumina) to perform our blinded clinical validation. This approach to validation allowed us to demonstrate both analytical and clinical equivalency to the reference Illumina test, while also refining our understanding of the strengths and weaknesses of our own unique version of the assay.
The structure and rationale of the study is illustrated in Figure 1. The study was designed to perform both a direct analytical comparison of our LDT with the reference test, as well as an independent metric of clinical utility. The reference laboratory for the LDT validation was Illumina, Inc.’s Verinata Health laboratory in Redwood City, CA. This laboratory stored plasma remnants of approximately 1mL from properly consented clinical specimens after the prescribed commercial clinical test had been reported to the ordering healthcare provider as part of the existing Progenity-Illumina test send out (TSO) commercial arrangement. Storage at -80°C for up to 6months was maintained in 96-well plates as the sample cohort accumulated. Samples for the study were collected between June and September 2015. A smaller, secondary study was completed to show the equivalency of fresh versus frozen plasma. “All testing was performed on material whole-blood samples received in cfDNA BCT™ tubes (Streck, Omaha, NE) received within 5 days from blood draw and accessioned with a complete test requisition form (TRF) authorized by an ordering healthcare provider. The TRF also included a patient informed consent and patient signature, which were required for testing”.9 Therefore, IRB approval was not required for this study.
Figure 1 Schematic of the overall validation process is shown here.
The equivalency feature is created by the transferal of blinded samples from the reference laboratory (Verinata) to Progenity, while at the same time transmitting the reference test results to the adjudicator (White Crow Innovation, LLC Ann Arbor MI). The clinical follow-up feature was performed by Integrity Genomics (Charleston SC), which received test results and demographic information from Verinata. Privacy of patient information sent to White Crow Innovation was ensured in that WCI never had access to patient demographic information. Verinata sent test results with only an internally meaningful 2D bar code. Progenity sent WCI results with the same 2D bar code as identification. Integrity Genetics had access to the patient demographic information, and the 2D bar code, but did not transmit the names and other confidential information to WCI.
The Progenity laboratory received the samples from the reference lab blinded, with only an untraceable 2D barcode for sample identification. Samples were processed and analyzed at the Progenity laboratory (Ann Arbor, MI) and the results sent to White Crow Innovations, LLC (WCI; Ann Arbor, MI), an independent consulting firm, for adjudication. Illumina sent its verifi test results and 2D barcodes from the original verifi assay to WCI. Illumina also sent demographic and contact information, as well as the clinical results of the verifi assay performed at the Illumina laboratory to WCI’s partner, the independent genetic counseling firm, Integrity Genomics, Inc. (IG; Charleston, SC) for clinical follow-up.
A total of 4032 remnant plasma samples (42 different 96-well plates) were sent to the Progenity, Inc. laboratory by the Illumina, Inc. laboratory. These samples had been accumulated and stored at Progenity’s request after initial analysis by Illumina and reporting to physicians by Progenity, as noted above. To ensure the sample results were blinded to Progenity, samples were given 2D barcode identification numbers, randomized, and were sent directly to Progenity by Illumina. Of the 4032 samples, Illumina provided aneuploidy data on 4031. Information could not be located on the single remaining sample, so it was eliminated from the validation. Progenity classified 645 samples as “Administrative Cancellation (AC)” due to pre-testing quality issues with the remnants, and these samples were also removed from the validation. Of the remaining 3386 samples, 52 were eliminated due to the physician practice response being inadvertently communicated directly to Progenity rather than Integrity Genomics (IG), thereby breaking the blind for those samples (15 of the 645 AC samples were also incorrectly faxed, but had already been eliminated due to AC status). The final number of samples used for the blinded Progenity-Illumina equivalency results was 3334.
Each specimen collected to run the reference assay used 10ml of peripheral venous blood collected into a Cell-Free DNA BCT® tube (Streck, Inc.; Omaha NE). Blood was separated into plasma and an extra aliquot (1ml) of plasma from samples submitted for testing using the verifi NIPT assay was set aside (under informed consent) at -80°C for future testing in the validation study. This aliquot of plasma was identified on a Matrix 2D Storage Tube (Thermo Fisher Scientific, Inc.; Hudson, NH), separate from any patient information. WCI acted as an independent, third-party consulting firm assisting in the design, blinding, and reporting of the validation study. IG was contracted by WCI to obtain clinical outcomes from patients in the study. 4031 blinded samples were shipped to Progenity from Verinata for use in validation with only the 2D barcode as an identifier. These samples were run through the verifi® by Progenity assay and test results were reported to WCI for comparison against the original test result reported by Illumina in the analytical portion of the validation. To further tie the result with a clinical outcome, ordering healthcare providers were contacted through IG in an attempt to discover if the result reported by the verifi® assay was clinically correct in the clinical outcomes secondary portion of the overall validation. cfDNA was extracted from the plasma samples using a buffer system including chaotropic salts and an AcroPrep™ DNA Filter Plate silica matrix (Pall Inc.; Port Washington NY) for reversible binding of DNA fragments. cfDNA fragments were then prepared into shotgun sequencing libraries using a modification of the Illumina TruSeq Nano chemistry including custom indexed-adapters (Integrated DNA Technologies, Inc.; Coralville IA). Library preparation steps include end-repair, a-tailing, adapter ligation, and PCR enrichment. Following PCR, libraries were purified using SPRI beads and quantitated using a QuantiFluor® dsDNA binding fluorescent dye (Promega; Madison WI). Libraries falling below a lower-limit at this step fail QC. Libraries passing QC are normalized to an equimolar concentration and pooled in preparation for sequencing. Massively parallel sequencing with short reads was used to analyze the libraries on the Illumina HiSeq™ 2500 system (Illumina, Inc.; San Diego CA). Sequencing reads were aligned to the human genome and aligned reads were assigned to specific bins. A proprietary bioinformatics algorithm was used to calculate the normalized chromosome value (NCV) which is used to determine aneuploidies for chromosomes 13, 18, 21, and the sex chromosomes.
Data analysis started with CASAVA/BCL2FASTQ de-multiplexing, followed by sample alignment to the hg19 genome reference using Bowtie 1.1.21., retaining only unique alignments with no more than one mismatch. The resulting mapped sequences for each sample were binned, normalized, and summarized into NCVs according to a robust, proprietary algorithm based upon established methods2. For each sample, NCV scores for autosomal chromosomes 13, 18, and 21 were evaluated by the algorithm to identify values exceeding empirically determined fetal aneuploidy thresholds. Likewise, NCV values for chromosomes X and Y were evaluated together by the algorithm to determine fetal gender or sex chromosome aneuploidies. Results were agglomerated and reviewed by a panel of scientists to certify the results of aneuploidy calls. Reviewed results were tabulated and sent to a WCI for unblinding and comparison to both the Illumina results as well as clinical outcomes. Rules for calling the results and compilation of the database are presented in detail in the “Supplementary Information” section provided.
The distribution of aneuploidy results from reference laboratory for the original sample set, the validation dataset based on results from Progenity, and the clinical outcome dataset are shown in Table 1. The original dataset was reduced as explained in the “Methods” section for the equivalency study and clinical outcome study.
Genetic assessment |
Unaffected |
Trisomy |
Suspected Trisomy |
||||||||
Chromosome 13 |
|||||||||||
Illumina Dataset (N=4031) |
3999 |
29 |
3 |
||||||||
Progenity Validation Data set (N1=3334) |
3310 |
21 |
3 |
||||||||
Outcome Dataset (1,11.1681) |
1673 |
8d |
0 |
||||||||
Chromosome 18 |
|||||||||||
Illumina Dataset (N=4031) |
3977 |
50 |
4 |
||||||||
Progenity Validation Dataset (N-3334) |
32a8 |
43 |
3 |
||||||||
Outcome Dataset (N=1681) |
1666 |
15d |
0 |
||||||||
Chromosome 21 |
|||||||||||
Illumina Dataset (14=4,03111 |
3289 |
137 |
5 |
||||||||
Progenity Validation Dataset (N=3334) |
3218 |
111 |
5 |
||||||||
Outcome Dataset (N=1681) |
1635 |
46° |
0 |
||||||||
Sex |
|||||||||||
XX |
XY |
X- |
XXX |
XXY |
XYY |
TwMa |
TwFb |
NAc |
|||
Illumina Dataset (14=4,031) |
MTh |
2013 |
75 |
23 |
20 |
13 |
6 |
2 |
2 |
||
Progenity Validation Data set (N-3334) |
1536 |
1676 |
65 |
19 |
J.6 |
13 |
4 |
3 |
2 |
||
Outcome Dataset (N=1631) |
301 |
MO |
9d |
4d |
4d |
ad |
3 |
1 |
lb |
Table 1 Characteristics of aneuploidy datasets from illumina /progenity and outcomes assessments
aMale twin Present; bNo Ma le Twin Present; cNo information available/unknown; dConfirmed by Cytogenetic means
Comparison of the chromosome 21 results from reference laboratory and Progenity showed agreement for 3209 of the 3218 (99.72%) “Not affected” results (Table 2). One hundred percent (100.0%) agreement was seen between the 111 samples reported by reference laboratory as positive for C21 trisomy and the blinded data from Progenity. Progenity identified 5 (0.16%) of the reference laboratory “not affected” samples as suspected C21 monosomy and 4 (0.12%) samples as suspected C21 trisomy (Table 3). Of the 5 samples classified as suspected C21 monosomy, clinical outcome data indicated three of the children were born normal (inconsistent with the Progenity result of C21 monosomy, a lethal aneuploidy) and no clinical outcomes were available for the other two samples. Clinical outcome data showed all 4 of the samples reported by Progenity as suspected C21 trisomy were from mothers delivering normal children, supporting the reference laboratory result of “not affected.” All samples categorized as suspected C21 trisomy by reference laboratory were reported as “not affected” by Progenity. One of these 5 samples was reported by clinical outcome assessment as “not affected”/normal. The remaining 4 samples had no clinical outcome data available.
Affected |
Not Affected |
|
Trisomy 21 |
>99.9% (111/111) |
99.7% (3209/3218) |
Trisomy 18 |
95.3% (41/43) |
99.9 (3287/3288) |
Trisomy 13 |
>99.9 (21/21) |
99.9 (3307/3310) |
Monosomy X |
95.4%(62/65) |
99.4%(3250/3269) |
Male |
Female |
|
Fetal sex |
99.8% (1672/1676) |
98.9% (1519/1536) |
Table 2 Progenity calls vs. Illu mina calls demonstrating test equivalency
Genetic Assessment |
2D Barcode ID |
Designation |
||
Illumina |
Progenity |
Outcome |
||
Chromosome 13 |
I69504062 |
Not affected |
Suspected C13 monosomy |
Not affected |
178558318 |
Not affected |
Suspected C13 monosomy |
No response |
|
178559057 |
Not affected |
Suspected C13 monosomy |
Unknown |
|
178558622 |
Suspected C13 trisomy |
Not affected |
Unknown |
|
181983551 |
Suspected C13 trisomy |
Not affected |
No response |
|
Chromosome 18 |
178560090 |
Not affected |
Suspected C18 monosomy |
No response |
178559-879 |
C18 trisomy |
Suspected C18 monosomy |
No response |
|
182013079 |
C18 trisomy |
Suspected C18 trisomy |
Not affected |
|
178530-823 |
Suspected C18 trisomy |
Not affected |
Not affected |
|
134445371 |
Suspected C18 trisomy |
Not affected |
Not affected |
|
Chromosome 21 |
169504117 |
Not affected |
Suspected C21 trisomy |
Not affected |
169505461 |
Not affected |
Suspected C21 trisomy |
Not affected |
|
178536336 |
Not affected |
Suspected C21 trisomy |
Not affected |
|
173553347 |
Not affected |
Suspected C21 trisomy |
Not affected |
|
169509266 |
Not affected |
Suspected C21 monosomy |
No response |
|
178557789 |
Not affected |
Suspected C21 monosomy |
Not affected |
|
178558295 |
Not affected |
Suspected C21 monosomy |
No response |
|
178559072 |
Not affected |
Suspected C21 monosomy |
Not affected |
|
178564061 |
Not affected |
Suspected C21 monosomy |
Not affected |
|
181973222 |
Suspected C21 trisomy |
Not affected |
No response |
|
181973311 |
Suspected C21 trisorny |
Not affected |
Not affected |
|
181983133 |
Suspected C21 trisomy |
Not affected |
Unknown |
|
182013519 |
Suspected C21 trisomy |
Not affected |
No response |
|
182018405 |
Suspected C21 trisomy |
Not affected |
No response |
Table 3 Chromosome 13, 18, and 21 Comparisons of Illurnina and Progenity Assessments
The Progenity blinded testing for Chromosome 18 resulted in 3287 of the 3288 (99.96%) samples listed as “not affected” by reference laboratory having the same determination by Progenity (Table 2). Progenity categorized the remaining sample as suspected C18 monosomy. No result was available for this sample in the outcome assessment to define the true result. The reference laboratory results indicated 43 samples as positive for C18 trisomy. Progenity's blinded testing listed 41 (95.3%) samples as positive for C18 trisomy and 2 (4.7%) samples as suspected C18 trisomy (Table 3). Since both these categories would result in a recommendation to seek confirmatory testing, the two results are essentially equivalent from the patient's perspective. The clinical outcome results indicated that one of the two Progenity suspected CI8 trisomy results resulted in a normal birth (karyotype determined during pregnancy) and no response was received for the second sample. Of the 3 samples for which reference laboratory reported a suspected C8 trisomy result. One of these had the same result from the Progenity blinded analysis and the remaining 2 were categorized as “not affected” by Progenity. Both of these not affected results from Progenity were confirmed as normal births during the clinical outcome follow up.
For Chromosome 13, 3307 of the 3310 (99.91%) samples classified as “not affected” by the reference laboratory had the same normal designation when tested by Progenity (Table 2). All 21 of 21 (100.0%) samples listed as positive for trisomy 13 by the reference laboratory were identified by Progenity in blinded testing. Progenity listed the remaining three samples as "suspected C13 monosomy". Only one of these three samples had clinical outcome information and was stated to be normal by the physician practice. As agreed when designing the current validation, "suspected" samples in this screening method would not be considered discordant since the result would result in advising the patient to seek confirmatory testing. Nevertheless, for the three samples suspected to be trisomy 13 by reference laboratory, one was identified as suspected by Progenity and the other two were designated as “not affected”. No results were available from the clinical outcome assessment to define the true results.
Of the 1536 samples indicated by reference laboratory as normal female (XX), Progenity had the same result for 1519 (98.89%o) samples (Table 2). Progenity results for this grouping included 9 (0.59%) as monosomy X, 7 (0.46%) as XXX, and 1 (0.06%) as male (XY) (Table 4). Clinical outcome data for the 9 monosomy X results indicated 5 normal females (XX) and no response for the remaining 4 samples. For the 7 samples indicated as normal female by reference laboratory and XXX by Progenity, clinical outcome results showed 5 normal females and 2 samples with no response. The outcome results for these samples suggest the reference laboratory results to be the correct finding. The reference laboratory designation of normal female also was confirmed by clinical outcome results as female.
2D Barcode ID |
Designation |
||
Illumina |
Progenity |
Outcome |
|
169502660 |
XX |
Monosomy X |
XX |
169505394 |
XX |
Monosomy X |
XX |
169505490 |
XX |
Monosomy X |
No response |
178530377 |
XX |
Monosomy X |
XX |
178536612 |
XX |
Monosomy X |
No response |
178545876 |
XX |
Monosomy X |
No response |
178559815 |
XX |
Monosomy X |
No response |
178559821 |
XX |
Monosomy X |
XX |
178559834 |
XX |
Monosomy X |
XX |
169504458 |
XX |
XXX |
XX |
169504713 |
XX |
XXX |
XX |
169507637 |
XX |
XXX |
XX |
169503935 |
XX |
XXX |
No response |
169509269 |
XX |
XXX |
No response |
178531091 |
XX |
XXX |
XX |
175860438 |
XX |
XXX |
XX |
175876237 |
XX |
XY |
XX |
169500988 |
XX |
Monosomy X |
.XY |
175835363 |
XX |
Monosomy X |
XY |
169500441 |
XX |
XX |
XX |
169509385 |
XX |
XYY |
XY |
169509130 |
Monosomy X |
XX |
XX |
182007529 |
Monosomy X |
XX |
No response |
182016515 |
Monosomy X |
XX |
No response |
178565057 |
XXX |
XX |
XX |
278560515 |
XXY |
XYY |
XXY |
169508668 |
XYY |
XY |
XY |
278530823 |
Not available |
XY |
XY |
178535836 |
Not available |
XX |
Lost to follow up |
Table 4 Sex Chromosome Comparison of Illumina and Progentiy Assessments
Reference laboratory results showed 1676 samples as normal male (XY) and Progenity reported 1672 (99.76%) with this same result (Table 2). Progenity reported 2 of the remaining samples (0.12%) as monosomy X (Table 4). Outcome results indicated both these to be normal male births and in disagreement with the Progenity finding. One sample (0.06%) was reported as normal female (XX), a result confirmed by clinical outcome assessment, and indicating the Progenity designation to be correct. This revises the Progenity accuracy rate to 99.82% (1672/1675). Progenity reported the remaining sample (0.06%) as XYY, but this was indicated as a normal male (XY) by clinical outcome assessment.
The monosomy X category included 65 samples from reference laboratory. Sixty-two (95.4%) of these samples also were reported as monosomy X by the Progenity blinded analysis (Table 3). Progenity indicated three (4.6%) of the remaining samples to be normal females (XX). One of these 3 results was confirmed as a normal female by clinical outcome data, in agreement with the Progenity categorization, and revising the Progenity accuracy rate to 96.9% (62/64). No clinical outcome information was available for the other two samples.
The reference laboratory reported 19 samples as Triple X Syndrome (XXX). The blinded Progenity testing also reported XXX for 18 (94.7%) of these samples. The discordant sample was determined to be XX by Progenity, a result confirmed by the clinical outcome follow up, providing a revised accuracy for the Progenity method of 100.0% (18 of 18 actual XXX samples). Progenity results showed 15 of 16 (93.7%) samples indicated as XXY to be XXY. The remaining sample (6.3%) was reported as XYY by Progenity. Clinical outcome assessment confirmed the reference laboratory result. For the 13 XYY samples from reference laboratory, Progenity results agreed with 12 of these results. Progenity reported the remaining sample as XY, a result confirmed by clinical outcome analysis including amniocentesis testing. The Progenity concurrence for XXY therefore is 100.0% (12/12).
The overall accuracy of the Progenity method for predicting genetic sex characteristics was 99.31% (3307 of 3330 samples) with data confirmed by the clinical outcome assessment and with reference laboratory characterization other than “not available”, indicating the result could not be collected from the ordering provider. Clinical follow-up was not available for a variety of reasons: no response from the provider, provider refused to give info, patient was lost to follow-up (transferred care), or the office couldn’t find the patient in their system for whatever reason.
While there is overwhelming agreement between the reference laboratory method and the Progenity LDT, the tests are not identical. Comparison to the clinical outcome findings is instructive. In the chromosome 21 analysis, 111 cases of trisomy are found by both methods, but nine cases called “not affected” by the reference method were called “suspected C21 trisomy” in 4 Progenity calls and “suspected C21 monosomy” in 5 calls by the Progenity LDT. The clinical outcome data agrees with the reference method in 7 of 9 of these cases and is indeterminate in 2 of 9. The reference method makes 5 “suspected C21 trisomy” calls, all of which are called “not affected” by the Progenity LDT. Only one of these cases is informed by clinical outcome, and that is confirmed as “not affected”.
In the chromosome 18 analysis, there are 5 total discrepant cases. The reference method calls two cases “suspected C18 trisomy.” These are called “not affected” by the Progenity LDT, and confirmed as such by clinical outcome data. In one case, the reference method call is “C18 trisomy,” while the Progenity LDT call is “suspected C18 trisomy,” and the clinical outcome is “not affected.” There is a second similar case which has no clinical outcome data. A final case has a reference method call of “not affected,” a Progenity LDT call of “suspected C18 monosomy,” and no clinical outcome.
In the chromosome 13 analysis, there are also 5 discrepant cases. Only one of these has clinical outcome follow up, which supports a call of “not affected” by the reference method as opposed to a call of “suspected C13 monosomy” by the Progenity LDT. Two additional cases were called “suspected C13 trisomy” by the reference method and “not affected” by the Progenity LDT. Two additional calls of “suspected C13 monosomy” by the Progenity LDT were called “not affected” by the reference method.
The complete lack of confirmation with any Progenity “suspected monosomy” call by either the reference method or the clinical outcome data was used a warrant to eliminate this category from the results reported to medical providers. Those samples, showing no other finding, will be reported as “not affected”.
With regard to sex chromosome anomalies, 29 total discrepancies were seen. Eleven of these were monosomy X called by the Progenity LDT. Two of the eleven were called XY by the reference method and confirmed by clinical outcome data. Three discrepancies were seen in reference method calls of monosomy X, which were called normal female by the Progenity LDT and confirmed by clinical outcome as XX in one case. Seven discrepancies were XXX as called by the Progenity LDT and XX by the reference method, also supported by clinical follow up as normal female in 5 of the cases. A variety of 8 other discrepancies were seen as shown in Table 4.
The NIPT for chromosomal aneuploidies is meant to be practiced as a screen at this time, leading to diagnostic confirmation. As such, the sensitivity of the assay is the paramount feature. In Table 5A & 5B, the sensitivity of the Progenity assay as defined by the outcome data is compared with the reference. In both cases, the sensitivity for C21 is 100% with similar 95% confidence intervals. Sensitivities for the other chromosomal and sex chromosome aneuploidies are also high underscoring the utility of the test as a screen. The 95% confidence intervals of the reference lab results are generally smaller, consistent with the larger N.
Chromosome |
Total |
Sensitivity |
Sensitivity |
Sensitivity |
Specificity |
Specificity |
Specificity |
Accuracy (In/ N) |
Accuracy |
Accuracy |
Chromosome 21 |
1680 |
46/46 |
100 |
90.4 - 100.0 |
1625/1634 |
99.4 |
98.9- 99.7 |
1671/1680 |
99.5 |
99.1 -99.8 |
Chromosome 18 |
1680 |
14/15 |
93.3. |
66.0- 99.7 |
1662/1655 |
99.8 |
99.4 - 99.9 |
1676/1680 |
99.8 |
99.5-100.0 |
Chromosome 13 |
1680 |
7/8 |
87.5 |
46.7- 99.3 |
1669/1672 |
99.8 |
99.4-99.9 |
1676/1680 |
99.8 |
99.5- 100.0 |
XX |
1660 |
769/814 |
94.5 |
92.6- 95.9 |
841/846 |
99.4 |
98.5- 99.8 |
1610/1660 |
97.0 |
96.2 -97.8 |
XY |
1660 |
830/846 |
98.1 |
96.9 - 98.9 |
804/814 |
98.9 |
97.7- 99.4 |
1634/1660 |
98.4 |
97.8 -99.0 |
X- |
1660 |
9/9 |
100 |
62.9. 100 |
153.4/1651 |
99.0 |
98.3 -99.4 |
1643/1660 |
99.0 |
98.5 -99.5 |
XXX |
1660 |
4/4 |
100 |
39.6- 100.0 |
1649/1656 |
99.6 |
99.1- 99.8 |
1653/1660 |
99.6 |
99.3 -99.9 |
XXY |
1660 |
3/4 |
75 |
21.9 - 98.7 |
1555/1655 |
99.9 |
89.6-99.9 |
1058/1660 |
99.9 |
99.7 - 100.0 |
XYY |
1660 |
3/3 |
100 |
31.0- 100.0 |
1655/1657 |
99.9 |
99.5- 99.9 |
1658/1660 |
99.9 |
99.7- 100.0 |
Table 5A Clinical Outcome Data Performance. Progenity vs. Outcome Data
Chromosome |
Total |
Sensitivity |
Sensitivity |
Sensitivity |
Specificity |
Specificity |
Specificity |
Accuracy (In/ N) |
Accuracy |
Accuracy |
Chromosome 21 |
1990 |
52/52 |
100 |
91.4-100.0 |
1935/1938 |
99.8 |
99.5- 99.9 |
1987/1990 |
99.8 |
99.7 - 100.0 |
Chromosome 18 |
1990 |
16/17 |
94.1 |
69.2-99.7 |
1966/1973 |
99.6 |
99.2 - 99.8 |
1982/1990 |
99.6 |
99.3 - 99.9 |
Chromosome 13 |
1990 |
9/10 |
90 |
54.1-99.5 |
1978/1980 |
99.9 |
99.6 - 99.9 |
1987/1990 |
99.8 |
99.7 - 100.0 |
XX |
1966 |
937/978 |
95.8 |
94.3- 96.9 |
983/982 |
99.5 |
98.8 - 99.8 |
1920/1966 |
97.7 |
97.0 - 98.3 |
XY |
1966 |
972/988 |
98.4 |
97.3-99.0 |
968/978 |
99.0 |
98.1 - 99.5 |
1940/1966 |
98.7 |
98.2 - 99.2 |
X- |
1966 |
11/11 |
100 |
67.9 - 100.0 |
1943/1955 |
99.4 |
93.9 - 99.7 |
1954/1966 |
99.4 |
99.0 - 99.7 |
XXX |
1966 |
5/5 |
100 |
46.3- 100.0 |
1958/1961 |
99.8 |
99.5- 99.9 |
1963/1966 |
99.8 |
99.7-100.0 |
XXY |
1966 |
5/5 |
100 |
46.3 - 100.0 |
1960/1961 |
99.9 |
99.7 - 99.9 |
1965/1966 |
99.9 |
99.8 - 100.0 |
XYY |
1966 |
3/3 |
100 |
31.0 - 100.0 |
1962/1963 |
99.9 |
99.7 - 99.9 |
1965/1966 |
99.9 |
99.8 -100.0 |
Table 5B Clinical Outcome Data Performance. Reference vs. Outcome Data
The results of this blinded combined equivalency and clinical validation study highlight a cfDNA screening test for assessing the presence of fetal aneuploidy. However, the study provides far more than simply documenting the accuracy of Progenity’s test for the practicing clinician. The study also provides a new, highly effective and practical model for laboratorians who wish to validate and implement such testing against predicate tests on the market. Although not specifically discussed in this paper, our novel approach is not only more cost effective than traditional prospective patient recruitment efforts but also faster to accomplish given the dramatic decrement in invasive diagnostic procedures used in the original validation protocols employed by the initial NIPT laboratories.
None.
The author declares no conflict of interest.
©2017 Stoerker, et al. This is an open access article distributed under the terms of the, which permits unrestricted use, distribution, and build upon your work non-commercially.