Reporting and interpreting the numerical results of a vaccine trial: a practical guide for students and health care professionals

doi:10.15406/japlr.2021.10.00370

Journal of

eISSN: 2473-0831

Analytical & Pharmaceutical Research

Mini Review Volume 10 Issue 2

Reporting and interpreting the numerical results of a vaccine trial: a practical guide for students and health care professionals

Andrea Manfrin,¹

Verify Captcha

Regret for the inconvenience: we are taking measures to prevent fraudulent form submissions by extractors and page crawlers. Please type the correct Captcha word to see email ID.

Miland Joshi²

¹Faculty of Clinical and Biomedical Sciences, School of Pharmacy and Biomedical Sciences, University of Central Lancashire, Preston, UK
²Clinical Trial Unit, Faculty of Health and Care, University of Central Lancashire, Preston, UK

Correspondence: Andrea Manfrin, Faculty of Clinical and Biomedical Sciences, School of Pharmacy and Biomedical Sciences, University of Central Lancashire, Preston, UK, Tel +44 17 72 89 58 15

Received: April 09, 2021 | Published: April 16, 2021

Citation: Manfrin A, Joshi M. Reporting and interpreting the numerical results of a vaccine trial: a practical guide for students and health care professionals. J Anal Pharm Res. 2021;10(2):93-96. DOI: 10.15406/japlr.2021.10.00370

Download PDF

Abstract

In one year, the COVID pandemic has changed the world. Universities, pharmaceutical companies, and many other organisations worldwide worked effortlessly to develop vaccines to ease the socioeconomic burden of the disease and improve global health care. Nowadays, vaccine efficacy is a very popular term; but do we know what it means and how to calculate it? This article provides information on reporting and interpreting the numerical results of a vaccine trial. It aims to be a practical guide for students and health care professionals. It gives a simple definition of the common terminology, the vaccine efficacy, how to calculate it, and how the confidence interval can be found (though formulae for the latter are not given here). Additionally, it provides two simple examples (A, B), including the formulae for calculating the vaccine efficacy explaining the differences of vaccine efficacy between the two examples simply and pragmatically.

Keywords: COVID-19, vaccine, efficacy, calculation, students, health care, professionals

Introduction

The first human coronavirus was discovered in 1960 and was responsible for upper respiratory tract infection. Since then, two major pandemics have afflicted humanity: SARS-CoV Epidemic (2003) and MERS-CoV (2013).¹ At the end of December 2019, a new disease of unknown aetiology appeared inWuhan, China.² The World Health Organization (WHO), on March 11, 2020, declared the novel coronavirus (COVID-19) outbreak a global pandemic.³ Last year, governments, universities, and industries made a huge effort to develop a vaccine. In September 2020, there were 321 candidate vaccines; of these, 33 were tested in clinical trials.⁴By the end of 2020, a few vaccines were available, and at the beginning of 2021 more were forthcoming. The two vaccines produced by Pfizer and Moderna have a special requirement – they need to be stored at a very low temperature.⁵ The University of Oxford and Astra Zeneca created a vaccine of⁶ that can be stored at normal fridge temperatures. Before the end of 2020, the vaccination campaign started in some countries, and by March 18, 2021, more than 400 million people living worldwide received their first COVID jab.

Since the development of these vaccines, their efficacy and comparison have become a frequent subject of discussion in the media and scientific publications; comments and editorials have appeared in many journals such as the British Medical Journal.⁷We felt the need to clarify, with simple examples, how efficacy can be calculated and compared across vaccines. We aimed to be consistent with the existing guidelines for reporting trials results, such as the Consolidated Standards of Reporting Trials (CONSORT) statement.⁸ Hodgson et al. 2021, in their publication in the Lancet, provided several definitions of efficacy according to different study endpoints.⁹In recent news reports, we might hear that a new COVID vaccine is "95% effective" and regard this as very good without having a clear idea of just what it means. This article aims to explain the term and how to calculate it. We should also explain what this article is not. It is not a comprehensive checklist for reporting a vaccine trial, comparable to CONSORT. It deals with numerical results only, not with problems of selecting subjects from a target population, the safety of a vaccine, finding suitable comparators (particularly older subjects during an epidemic, for whom participation would itself involve risk); nor do we deal with adverse outcomes.

Aim: to provide simple and clear guidance for calculating, interpreting, and reporting the numerical results of vaccine trials.Before looking at the examples, we introduce an item checklist summarising the most important information that should be reported in the results section of a clinical trial using a tabular format for clarity and simplicity (Table 1).

Item	Abbreviation	Definition
Control event rate	CER	The proportion of patients in the control group who experience the studied event.
Experimental event rate	EER	The proportion of patients in the experimental treatment group who are observed to experience the outcome of interest.
Relative Risk	RR	The risk of the outcome in the intervention group compared to the risk in the control group.
Absolute risk reduction	ARR	The difference in risk between the control group and the treatment group.
Number needed to treat (adverse outcomes)	NNT	A measure of treatment efficacy/effectiveness. It is the average number of people who need to be treated with a specific intervention for a given period to prevent one additional adverse outcome.
Vaccine efficacy	VE	The percentage reduction in disease risk in a vaccinated group compared to a non-vaccinated group under optimal conditions (RCT).
Vaccine effectiveness	VE	It is the percentage reduction in disease risk in a vaccinated group compared to a non-vaccinated group in real-world conditions.

Table 1 Terminology used for reporting numerical results of a vaccine trial

Statistical meaning of 95% efficacy and confidence interval

Some people might think that 95% efficacy means that 95 out of every 100 people given the vaccine will not develop the infection. But this could happen even without a vaccine or placebo, particularly in younger people. Therefore, it cannot be the real meaning.We can explain this concept using the subjects in a two-arm randomised controlled clinical trial. One arm of the trial receives the vaccine, the other (control) the placebo. Treatments are allocated randomly, and large numbers of patients are recruited to make the arms similar in every way except for the treatments.In a trial with equal-sized arms, our estimate for efficacy would be the fraction of cases in the control arm that the vaccine prevented in the experimental arm.The sizes of trial arms will not usually be exactly equal, however, and a more comprehensive definition is the proportionate reduction in risk in the vaccine arm.We also need to report the uncertainty in our estimate of "efficacy." To calculate the confidence interval, we can use the fact that vaccine efficacy is 1 minus the relative risk (RR) of infection in the vaccinated group.We can easily obtain a confidence interval for log (RR) using standard formulae for its standard error. However, even an apparent result of 100% efficacy needs to be assessed with due caution. Applying the "rule of 3," a confidence interval for an estimate for a proportion of events, given zero occurring out of n has an upper limit of 3/n. Thus, if zero cases occurred out of 100, the upper limit confidence interval for the proportion would be about 3%, not 0%. For our purposes, the complement of this proportion would correspond to the proportion protected, which is the effectiveness. Hence the lower limit of the confidence interval for the efficacy would be about 97%, not 100%.

Guide for reporting numbers on vaccine trials statistics

We consider two examples, presented in Table 2 & 3. In the examples, we used two-arm trials with equal numbers. Table 2 shows the calculations for vaccine A versus placebo, which has an efficacy of 95%. Table 3 shows the estimates for vaccine B versus placebo, which has an efficacy of 90%.In the tables, there are three numbers highlighted in bold:

The number of infected patients in the intervention group;
The number needed to treat (NNT) to obtain one immune patient;
The efficacy expressed as a percentage.

Main equation
	Infected	Not-infected	Total
Vaccinated	a	b	a+b
Not vaccinated	c	d	c+d
Total	a+c	b+d	Total
Application
	Infected	Not-infected	Total
COVID-19 Vaccine	5	145	150
Placebo	100	50	150
Total	105	195	300
Calculation
Control event rate	c/(c+d)	CER	0.67
Experimental event rate	a/(a+b)	EER	0.03
Relative Risk	EER/CER	RR	0.05
Absolute risk reduction	CER-EER	ARR	0.63
Number needed to treat	1/ARR	NNT	1.58
Efficacy	%=(1-RR) x100	Efficacy %	95
(95% CI)			(88.1-97.9)

Table 2 Vaccine A 95% efficacy versus placebo

Main equation
	Infected	Not-infected	Total
Vaccinated	a	b	a+b
Not vaccinated	c	d	c+d
Total	a+c	b+d	Total
Application
	Infected	Not-infected	Total
COVID-19 Vaccine	10	140	150
Placebo	100	50	150
Total	110	190	300
Calculation
Control event rate	c/(c+d)	CER	0.67
Experimental event rate	a/(a+b)	EER	0.07
Relative Risk	EER/CER	RR	0.1
Absolute risk reduction	CER-EER	ARR	0.6
Number needed to treat	1/ARR	NNT	1.67
Efficacy	%=(1-RR) x100	Efficacy%	90
(95% CI)			(81.6-94.6)

Table 3 Vaccine B 90% efficacy versus placebo

What are these numbers telling us?

The number of infected patients is lower in the population vaccinated with A (n=5) (Table 2) than B (n=10) (Table 3). The difference is that 5 (=10-5) more patients were infected in the Vaccine B group. The NNT, however, is more helpful, as it is the number of patients that need to be vaccinated to obtain one who is immune. The closer the NNT is to 1; the more effective is the treatment. The crucial figure is our estimate of the efficacy expressed as a percentage. We have VE = [(1-RR) x100], where the VE is the vaccine efficacy. RR is the relative risk or risk ratio =(probability of an event in the exposed group)/(probability of the event in not exposed group).¹⁰We also need to allow for uncertainty in our findings, expressed by the confidence interval. Vaccine A's efficacy is 95% whereas vaccine B’s is 90%, which indicates that Vaccine A is more effective than vaccine B. The difference in their efficacy is 5%, corresponding to the extra 5 patients infected in the vaccine B group in our example.

What does "efficacy" mean in practice?

Itis important to note that efficacy is not the proportion of the population who avoid infection but rather the reduction in the risk that they would face if vaccinated. Table 2 shows that 100 out of 150 would get the infection without the vaccine. In contrast, this was reduced to 5 out of 150 by vaccine A where the risk of 2/3 (100/150) was reduced to 1/30 (5/150), a proportionate reduction by 95%. Similar reasoning applies to Table 3.

Comparing efficacies

In comparing vaccines A and B, the number of infected patients using vaccine A is half of the number of patients treated with vaccine B, but A's efficacy – the relative reduction in risk that it achieves – is not double that of B, but only 5% higher.

Conclusion

The COVID-19 pandemic is having a substantial impact on health care and the socioeconomic aspects of people lives. Vaccines appear the most promising solution to the pandemic. The vaccine efficacy is largely debated in peer-review journals and by the media. This article provides a practical guide for students and health care professionals, using two examples with formulae to calculate the vaccine efficacy. It explains using facts and figures the confidence interval's relevance and the differences between 95% and 90% efficacy.