Background: Non-inferiority testing is used to demonstrate that a new treatment is not unacceptably worse than an existing treatment. Such analyses are useful, for example when placebo-controlled studies are unethical, or when there may be other considerations (e.g. convenience, cost) where the new treatment has an advantage. Prospective, non-inferiority trials in orphan diseases are difficult to coordinate because they often require large sample sizes to detect small margins of difference between treatments. In this report, we present a retrospective study of non-inferiority testing in the rare disease, Guillain–Barre syndrome (GBS).

Methods: Meta-analysis results of PE versus a control group (n=623) were used to derive non-inferiority margins for two endpoints:

1. Improvement of ≥1 grade on the GBS disability scale,
2. Mean change from baseline on the GBS disability scale. These were then retrospectively applied to meta-analysis results to demonstrate the non-inferiority of IVIg to PE (n=567).

Results: For endpoint 1, the non-inferiority margin of the risk ratio was 0.865. The risk ratio of IVIg versus PE was 1.08 (95% confidence interval CI: 0.94 to 1.23). Since the lower bound of the CI is above the non-inferiority margin (0.865), IVIg can be considered non-inferior to PE on this endpoint. For endpoint 2 assessing change from baseline on GBS disability scale, the non-inferiority margin was 0.315. The treatment difference (IVIg – PE) was -0.02 (95% CI:-0.25 to 0.20). Since the upper bound of 95% CI (0.20) is less than 0.315 (the non-inferiority margin), IVIg can be considered non-inferior to PE.

Conclusion: The results demonstrate non-inferiority of IVIg to PE in GBS when the non-inferiority margins are retrospectively applied. Retrospective non-inferiority analyses may also be used in evaluation of treatment effects for other rare diseases.

Keywords: non inferiority margin, orphan disease, guillain barre syndrome, plasma exchange, intravenous immunoglobulin

Randomized, placebo-controlled trials are the standard method by which the efficacy of medical treatments is determined. However, in serious diseases where there is a known effective treatment, allocating one patient group to a placebo arm may be unethical. Furthermore, in some situations, a new treatment may not be expected to be more effective than an existing treatment on the primary endpoint, but may have advantages in terms of secondary endpoints, such as safety, convenience, compliance or cost.^1,2 The non-inferiority trial is a vital tool when evaluating the efficacy of a novel therapy compared with an existing therapy. It aims to demonstrate that the test product is not worse than the comparator by more than a pre-specified, small amount. This amount is known as the non-inferiority margin, or delta (Δ).¹ Guidelines developed by the European Medicines Agency and the US Food and Drug Administration³ recommend predefining the non-inferiority margin. This margin can be derived from previous studies using historical data, and the study medication is typically expected to retain at least 50% of the original treatment effect over placebo or the standard of care to be considered non-inferior.

After a non-inferiority margin is established, a prospective non-inferiority trial is usually conducted to confirm the non-inferiority of the new product when compared to the existing product. Non-inferiority trials typically require considerably larger sample sizes than placebo-controlled trials.⁴ This is due to the fact that the margin of equivalence (non-inferiority) is often much smaller than the treatment difference, which a placebo-controlled trial must be powered to detect. It is therefore important for non-inferiority trials to have large sample sizes, and for this reason, trials of orphan drugs in rare diseases face significant challenges in terms of recruiting sufficient sample sizes to formally assess prospectively defined non-inferiority and of completing the trial within a realistic timeframe.

Here, we present a practical method for demonstrating non-inferiority of drugs for rare disease. This method is based on aggregated data from smaller studies that have been analyses in previously published meta-analyses,^5,6 and is illustrated using the example of intravenous immunoglobulin (IVIg) compared with plasma exchange (PE) for the treatment of Guillain–Barre syndrome (GBS).

GBS is a rare inflammatory disease affecting the peripheral nerves and causing weakness, numbness, breathing difficulty and paralysis. The disease affects between 0.5 and 2 per 100,000 persons per year.⁶ Although still under investigation, the cause of GBS is believed to be an autoimmune response.^5,6 In some patients, the condition can have a lasting impact after the end of its acute phase.⁵ Supportive care for GBS can include the administration of heparin, and the use of pressure stockings to prevent the onset of deep vein thrombosis in bed-bound patients, along with the monitoring of pulse, blood pressure, autonomic disturbances and respiration. Rehabilitation focuses on exercise to encourage strengthening, proper limb positioning, posture and orthotics.⁷

There are two effective immune therapies for GBS: PE, which involves separation of plasma from cells and re-infusion of those cells back into the patient, and IVIg, which uses antibodies purified from plasma that has been pooled from at least 1000 donors.^5,6 Administering IVIg is simple compared with PE. PE requires access to two veins, of which one has to permit high flow volumes, and frequently necessitates the insertion of a central venous line, a PE machine and specially trained personnel. IVIg requires access to only a single peripheral vein and no special equipment or specially trained staffs are necessary. Consistent with the difference in ease of administration, a Cochrane Review found that the risk ratio (RR) of treatment being discontinued was 0.14 less in the IVIg than in the PE group (95% confidence interval CI: 0.05 to 0.36). In addition, there is some evidence that adverse events are more frequent with PE than IVIg.^5,7

The clinical benefits of PE in GBS as measured by improvement on the GBS disability scale developed by Hughes et al.,⁸ have been confirmed in a Cochrane Review,⁶ which included six randomized, controlled trials (RCTs).^9–14 Few trials comparing IVIg with placebo have been conducted because PE was the standard of care when IVIg was introduced for GBS. However, a number of studies.15^–21 show that IVIg speeds recovery from GBS to a similar extent as PE, as concluded by a Cochrane Review.⁵

Due to the rarity of GBS, the majority of studies comparing IVIg and PE has used small sample sizes with limited statistical power and were not formally designed as therapeutic equivalence or non-inferiority trials.⁵ This may help to explain some inconsistency in the findings, and it is possible that some studies finding no significant difference between treatments reflect a lack of power to detect a significant difference rather than indicating true non-inferiority. The Cochrane Review by Hughes et al.,⁵ thoroughly reviewed all individual studies and performed a meta-analysis, but did not formally assess therapeutic equivalence or non-inferiority. The conclusion of no treatment difference cannot be automatically translated into either equivalence or non-inferiority.

Given the strong safety profile of IVIg,22 as well as the convenience of its use in the clinic,15 the current analysis was undertaken to formally establish the non-inferiority of IVIg to PE using existing studies from comparisons of PE versus supportive care, where much more data are available. A Cochrane Review of the benefits of PE in GBS⁶ was used to establish the non-inferiority margin, and then this derived non-inferiority margin was retrospectively applied to results from a Cochrane Review of IVIg benefits in GBS⁵ to demonstrate the non-inferiority of IVIg to PE.

In non-inferiority trials, one of the critical steps is to define the non-inferiority margin. This margin can be derived from previous studies using historical data, and the study medication is typically expected to retain at least 50% of the original treatment effect over placebo or the standard of care to be considered non-inferior. In the example in GBS, the non-inferiority margin was derived using results from the meta-analysis of previous trials comparing PE versus supportive care.

Raphael et al.,⁶ conducted a meta-analysis of five studies (623subjects, a summary of the included trials is shown in (Table 1). The RR of PE versus supportive care for the proportion of subjects with improvement of at least one grade on the GBS disability scale was calculated as 1.64 (95% CI: 1.37 to 1.96) (Table 2). For the proportion of subjects with improvement of at least one grade on the GBS disability scale, the non-inferiority margin for the RR can be derived using the fixed-margin method or the two 95% CI approach.^1,3,23^–24 For the purposes of this study, the new treatment is IVIg and the active control is PE. The fixed-margin approach involves determining the treatment effect (M1) of the active control group over the placebo (or no treatment) group by using the lower bound (or upper bound, depending on the direction) of the 95% CI from previous placebo-controlled trials or meta-analyses of trials. i.e., M1 = 1.37 which is the lower limit of 95% CI of the RR. Typically, preserving at least 50% of M1 from active control versus placebo (or no treatment) is recommended.³ i.e., RR of IVIg versus no treatment is greater or equal to $1+(M1-1)\times 50\%=1.185$. The non-inferiority margin (M2) is excluded by ensuring that the lower bound (or upper bound, depending on the direction) of the 95% CI is >M2. i.e., RR of IVIg versus PE is greater than $M2=\frac{1.185}{1,37}=0.865$.

A further meta-analysis was also performed on four studies (585 subjects) (Table 2) to assess change from baseline to week 4 using the GBS disability scale (endpoint 2). The treatment difference (PE–supportive care) was calculated as-0.89 (95% CI:-1.14 to -0.63).⁶ For the mean change from baseline on the GBS disability scale, the non-inferiority margin for the treatment difference can be derived using the fixed-margin method or the two 95% CI approach.^1,3,23^–24 The fixed-margin approach involves determining the treatment effect (M1) of the active control group over the placebo (or no treatment) group by using the upper bound (or lower bound, depending on the direction) of the 95% CI from previous placebo-controlled trials or meta-analyses of trials. i.e., M1 =-0.63 which is the upper limit of 95% CI of the treatment difference. Preserving at least 50% of M1 from active control versus placebo (or no treatment) is recommended.³ i.e., treatment difference of IVIg-no treatment is less or equal to $M1\times 50\%=-0.315$. The non-inferiority margin (M2) is excluded by ensuring that the upper bound (or lower bound, depending on the direction) of the 95% CI is < $M2=-0.315-(-0.63)=0.315$. The detailed derivation is shown below.

Endpoint 1: Improvement of at least one grade on the GBS disability scale

The treatment effect (M1) for PE versus Control (supportive care) is defined as the lower limit of the 95% CI of the RR.

The non-inferiority margin (M2) for IVIg versus PE can be calculated as follows:
$\frac{P(IVIg)}{P(PE)}=\raisebox{1ex}{$\left[\raisebox{1ex}{$P(IVIg)$}\!\left/ \!\raisebox{-1ex}{$P(Control)$}\right.\right]$}\!\left/ \!\raisebox{-1ex}{$\left[\raisebox{1ex}{$P(PE)$}\!\left/ \!\raisebox{-1ex}{$P(Control)$}\right.\right]$}\right.=\frac{1.185}{1.37}=0.865$

Therefore, the non-inferiority margin of the RR is 0.865, and IVIg is non-inferior to PE if the lower bound of the 95% CI of the RR of IVIg versus PE is greater than 0.865.

Hughes et al.⁵ conducted a meta-analysis of six studies (567 subjects).^15–20 An overview of the trials included is given in (Table 3). The RR of IVIg versus PE was 1.08 (95% CI: 0.94 to 1.23) for the proportion of subjects with improvement of at least one grade on the GBS disability scale. Since the lower bound of the 95% CI (0.94) is above the non-inferiority margin (0.865), IVIg can be considered non-inferior to PE on this endpoint (Table 4).

Endpoint 2: Mean change from baseline on the GBS disability scale

The non-inferiority margin can be derived using the two 95% CI approach.^1,.²³ Treatment effect (M1) for PE versus Control (supportive care) is defined as the upper limit of 95% CI of treatment difference.

M1 = PE - Control = - 0.63 (i.e. upper limit of CI) (Table 2)

$IVIg-Control=-0.63\times 50\%=-0.315$

The non-inferiority margin (M2) for IVIg versus PE is calculated as follows:
$IVIg-PE=(IVIg-Control)-(PE-Control)=-0.315-(-0.63)=0.315$ Therefore, 0.315 is the non-inferiority margin for the mean change from baseline on the GBS disability scale. IVIg can be considered non-inferior to PE if the upper bound of the 95% CI of mean difference of IVIg versus PE is less than 0.315.

Hughes et al.,⁵ Conducted a meta-analysis of five studies (536 subjects) (Table 3).^15-17,19,20 The treatment difference (IVIg–PE) was-0.02 (95% CI:-0.25 to 0.20). Since the upper bound of 95% CI (0.20) is less than 0.315 (the non-inferiority margin), IVIg can be considered non-inferior to PE (Table 4).

This analysis provides an illustration of how data collated from a number of small studies may be used to enable retrospective non-inferiority comparisons of treatments for rare diseases, for which it is often impossible to have adequate sample sizes for prospectively designed non-inferiority studies. In the example analysis presented here, the treatment effect of IVIg for GBS was compared with that of an established treatment (PE) for this condition with efficacy proven in RCTs. Based on this evaluation, we can conclude that IVIg is non-inferior to PE for the treatment of GBS.

Post-hoc analyses of non-inferiority have limitations, such as differences in study design, treatment regimens and patient characteristics across trials. Ideally, a prospective clinical trial should be undertaken to assess the non-inferiority of IVIg. However, based on the derived non-inferiority margin in this study, a sample size of more than 462 subjects would be needed without drop out consideration to conduct a prospective clinical trial to assess the non-inferiority of IVIg versus PE with 80% power for endpoint 1 assuming a rate of 60% for IVIg and 56% for PE. Similarly, a sample size of more than 622 subjects would be needed without drop out consideration to conduct a prospective clinical trial to assess the non-inferiority of IVIg versus PE with 80% power for endpoint 2 assuming no treatment difference between IVIg and PE and standard deviation of 1.4 for both treatments, which would be a considerable challenge for a disease that is as rare as GBS. In addition, since the previous studies have showed the benefit of the IVIg in treating GBS, it is quite challenging for a sponsor to perform a large scale, prospective non-inferiority study. Instead, this retrospective assessment made use of previously collected data, permitting non-inferiority of IVIg compared with PE to be demonstrated.

A 1997 study conducted by the Plasma Exchange/Sandoglobulin Guillain–Barre Syndrome (PSGBS) Trial Study Group established that IVIg is therapeutically equivalent to PE. Treatments were considered equivalent if the 95% CI of the difference in mean improvement in GBS disability scale after 4 weeks between the two groups excluded a true mean difference of more than 0.5 of a grade. Although a change of 1.0 of a grade could be reliably measured and was clinically meaningful, a mean change of less than 0.5 of a grade was considered to be insignificant; however this equivalence value is subjective, and is not based on data from randomized clinical trials. In the current study, a non-inferiority margin of 0.315 of a grade was derived using retrospective data, this is therefore more stringent than the equivalence margin of 0.5 of a grade used in the PSGBS Study Group study.¹⁵

This study demonstrates that, in the case of rare diseases where formal prospective non-inferiority design is rendered unfeasible by the large sample sizes required, retrospective data analyses can be undertaken to ascertain whether a new treatment meets criteria for non-inferiority. We recommend that this strategy be considered in other orphan diseases as a practical means to establish non-inferiority of treatment efficacy when prospectively designed non-inferiority studies are not feasible.

The authors are all employees of Grifols Inc., manufacturer of Gamunex®-C and Flebogamma® (both IVIg products).

CD and KH initiated the idea for developing this paper. CD and JC carried out the calculations and performed the statistical analysis. CD drafted the manuscript, the final version of which was reviewed and approved by all authors.

None.

Author declares that there are no conflicts of interest.

1. Guideline on the choice of the non-inferiority margin.
2. Lesaffre E. Superiority, equivalence, and non-inferiority trials. Bull NYU Hosp Jt Dis. 2008;66(2):150–154.
3. Guidance for industry: non-inferiority clinical trials.
4. Snapinn SM. Noninferiority trials. Curr Control Trials Cardiovasc Med. 2000;1(1):19–21.
5. Hughes RA, Swan AV, Van Doorn PA. Intravenous immunoglobulin for Guillain-Barre syndrome. Cochrane Database Syst Rev. 2012;7:CD002063.
6. Raphael JC, Chevret S, Hughes RA, et al. Plasma exchange for Guillain-Barre syndrome. Cochrane Database Syst Rev. 2012;2:CD001798.
7. Hughes RA, Wijdicks EF, Benson E, et al. Supportive care for patients with Guillain-Barre syndrome. Arch Neurol. 2005;62(8):1194–1198.
8. Hughes RA, Newsom-Davis JM, Perkin GD, et al. Controlled trial prednisolone in acute polyneuropathy. Lancet. 1978;2(8093):750–753.
9. Plasmapheresis and acute Guillain-Barre syndrome. The Guillain-Barre syndrome Study Group. 1985;35(8):1096–1104.
10. Farkkila M, Kinnunen E, Haapanen E, et al. Guillain-Barre syndrome: quantitative measurement of plasma exchange therapy. Neurology. 1987;37(5):837–840.
11. Greenwood RJ, Newsom Davis J, Hughes RA, et al. Controlled trial of plasma exchange in acute inflammatory polyradiculoneuropathy. Lancet. 1984;1(8382):877–879.
12. Osterman PO, Fagius J, Lundemo G, et al. Beneficial effects of plasma exchange in acute inflammatory polyradiculoneuropathy. Lancet. 19842(8415):1296–1299.
13. Efficiency of plasma exchange in Guillain-Barre syndrome: role of replacement fluids. French Cooperative Group on Plasma Exchange in Guillain-Barre syndrome. Ann Neurol. 1987;22(6):753–761.
14. Appropriate number of plasma exchanges in Guillain-Barre syndrome. The French Cooperative Group on Plasma Exchange in Guillain-Barre Syndrome. Ann Neurol. 1997;41(3):298–306.
15. Randomised trial of plasma exchange, intravenous immunoglobulin, and combined treatments in Guillain-Barre syndrome. Plasma Exchange/Sandoglobulin Guillain-Barre Syndrome Trial Group. Lancet. 1997;349(9047):225–230.
16. Bril V, Ilse WK, Pearce R, et al. Pilot trial of immunoglobulin versus plasma exchange in patients with Guillain-Barre syndrome. Neurology. 1996;46(1):100–103.
17. Diener HC, Haupt WF, Kloss TM, et al. A preliminary, randomized, multicenter study comparing intravenous immunoglobulin, plasma exchange, and immune adsorption in Guillain-Barre syndrome. Eur Neurol. 2001;46(2):107–109.
18. El-Bayoumi MA, El-Refaey AM, Abdelkader AM, et al. Comparison of intravenous immunoglobulin and plasma exchange in treatment of mechanically ventilated children with Guillain Barre syndrome: a randomized study. Crit Care. 2011;15(4):164.
19. Nomura K, Hamaguchi K, Hattori T, et al. A randomized controlled trial comparing intravenous immunoglobulin and plasmapheresis in Guillain-Barre syndrome. Neurological Therapeutics. 2001;18:69–81.
20. Van der Meche FG, Schmitz PI. A randomized trial comparing intravenous immune globulin and plasma exchange in Guillain-Barre syndrome. Dutch Guillain-Barre Study Group. N Engl J Med. 1992;326(17):1123–1129.
21. Donofrio PD, Berger A, Brannagan T, et al. Consensus statement: the use of intravenous immunoglobulin in the treatment of neuromuscular conditions report of the AANEM ad hoc committee. Muscle Nerve. 2009;40(5):890–900.
22. Elovaara I, Apostolski S, Van DP, et al. EFNS guidelines for the use of intravenous immunoglobulin in treatment of neurological diseases: EFNS task force on the use of intravenous immunoglobulin in treatment of neurological diseases. Eur J Neurol. 2008;15(9):893–908.
23. Rothmann M, Li N, Chen G, et al. Design and analysis of non-inferiority mortality trials in oncology. Stat Med. 20003;22(2):239–264.
24. Rothmann MD, Tsou HH. On non-inferiority analysis based on delta-method confidence intervals. J Biopharm Stat. 2003;13(3):565–583.