COVID-19 vaccines are coming – how will we know they work and are safe?

Vaccine efficacy reflects a proportion of the number of those who became ill in the vaccinated group and in the non-vaccinated group. (File photo: National Cancer Institute/Unsplash)

Pfizer and BioNTech have just released interim results of their COVID-19 vaccine trial. Although it is not the only vaccine in the late stages of testing, the large size and careful design of the trial, not to mention the promising results, have caused understandable enthusiasm around the world. As we get nearer the long-awaited start of a COVID-19 vaccine roll-out, it is worth looking at how statisticians help medics establish the safety of vaccines.

How effective is the vaccine?

It is not easy to find out how effective a vaccine is. First, researchers need to know whether just an act of injecting somebody can help. The trials involve a large number of people, with half of them given a vaccine and the other half a placebo. Then the participants need to be exposed to the infection with the expectation that most of those in the control group become ill, but vaccination protects at least some in the treated group.

In some cases, such as for HIV or Ebola, even giving a placebo can be ethically controversial as they have such a high death rate. For coronavirus, the researchers need to rely on natural infection because no study, at the moment, intentionally exposes participants to the coronavirus. As a result, the efficacy calculation is based on a relatively small number of those who caught COVID-19 by contact with other infected people.

Vaccine efficacy reflects a proportion of the number of those who became ill in the vaccinated group and in the non-vaccinated group. The Pfizer/BioNTech trial involved nearly 44,000 participants, with 21,999 given the vaccine. The researchers use statistical analysis to set up milestones at which they can be increasingly confident that the vaccine works – or it does not – as the cases trickle in. If the numbers are small, it would not be clear whether the difference in the outcomes between the placebo and the treated groups is real or just a result of a random fluke.

Statisticians use the so-called “power analysis” to discover how many cases we need to observe. For the Pfizer and BioNTech vaccine, the target was 164 cases when the final estimation of efficacy can be made, but this was based on the assumption that the vaccine is only 60% effective. This was based on the seasonal flu vaccine efficacy. However, with the numbers exceeding expectations, the company decided to release the results at one of the interim analysis points.

Ninety-four cases were reported and the split of about 86 cases in the placebo group and eight cases among vaccinated yielded 90% efficacy. This level of protection against infection is remarkable. Even though the study is based on a relatively small number of cases, statistical analysis allows the researchers to extrapolate to what might happen when the vaccine is rolled out.

The trial included different ages as well as people from different ethnic minority groups, but more studies would be needed to assess how the most vulnerable groups are protected.

The final efficacy is likely to be lower, as administering the treatment is difficult for many logistical reasons, including the requirement of mRNA-based vaccines, of which the Pfizer vaccine is one, to be stored at very low temperatures. In the real world, the vaccine might not be stored at the correct temperature and hence may spoil.

How safe is the vaccine?

If the vaccine is to be widely applied, the medical community and the public need to be reassured about its safety.

The Pfizer vaccine was administered to 21,999 people. Some people reported a reaction similar to the one after the seasonal flu vaccination, but so far no serious side-effects have been reported. But how can we be sure that this holds if the treatment is rolled out to millions of people?

Statisticians came up with the “rule of three”. The rule tells us that if 21,999 participants were treated with no side-effects, then with 95% confidence, the probability of a side-effect from the vaccine is expected to be less than three (hence the name) divided by 21,999 and so less than one in 10,000. The chance of these side-effects is probably even lower, but the researchers will be keen to extend the trials further to confirm this.

Safety is just as important as efficacy. If you take a probability of one in 10,000 and extrapolate that out to the 300 million population slated for vaccination in the US alone, the number of people with side effects could be as high as 30,000. Clearly, the doctors need to ensure they are not causing harm, but also any serious side-effect attributable to the vaccine would damage the reputation and significantly affect the take-up.

How to use the vaccine so it is effective and safe?

Medical authorities are now designing ways to implement vaccination in nationwide programmes, but the details on how to do this depend on several factors. The UK government has ordered 40 million doses of the Pfizer vaccine which – with two-dose treatment – would vaccinate 20 million people, that is, everyone aged 55 and up. However, the roll-out will not be fast since production and delivery will take time.

The strategy also depends on what the vaccination programme is supposed to achieve. Childhood vaccines, such as measles, are given to newborns to maintain herd immunity. In this case, only a relatively small proportion of the population needs to be vaccinated. With the rapid spread of COVID-19 – and high levels of existing infection – the proportion would need to be much higher.

Predictions for the level of immunity required to reach herd immunity depend on our estimate of the COVID-19 basic reproductive number, R. In absence of any control measures, R is estimated to be around 3 and so at least 67% of the public need to be fully immune just for the epidemic to stop growing. Higher values would need to be achieved if the aim is to eradicate the virus.

This level will hardly be achievable with 60% efficacy, even if the whole population is vaccinated. The value of R=3 assumes the return to the behaviour before the pandemic. If we keep some level of restrictions and use masks, R could be lower and the herd immunity easier to achieve.

On the positive side, our simple models might be too pessimistic about the herd immunity levels. Additionally, if perhaps as many as 20% of the public already have had COVID-19, the required level of vaccination might be much easier to achieve.

Alternatively, vaccination can be applied to these segments of the society who are either at high risk of infection (healthcare and care home workers) or high risk of death (vulnerable, care home residents). This is the recommended strategy in the UK.

Are we there yet?

The results of the Pfizer vaccine trial are highly promising. But the road to eradicating the coronavirus is likely to be long and difficult. Besides establishing the potential for the vaccine to protect against the virus, we also need to know whether it gives a lasting immunity or whether it would need to be applied repeatedly, for example, as with tetanus or seasonal flu vaccines.

But the policymakers and researchers also need to balance the requirement for stopping the pandemic with the fears of side-effects and the resulting vaccine hesitancy. While it is easy to dismiss these concerns, they need to be taken seriously if the vaccination is to be successful.

Adam Kleczkowski, Professor of Mathematics and Statistics, University of Strathclyde

This article is republished from The Conversation under a Creative Commons license. Read the original article.