Safety and Efficacy of COVID-19 Vaccines: A Systematic Review and Meta-Analysis of Different Vaccines at Phase 3

This systematic review and meta-analysis was conducted to compare the safety and efficacy of 2019 novel coronavirus disease (COVID-19) vaccines according to vaccine platform and severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) infection severity. Articles published between 24 January 2020 and 30 May 2021 were retrieved via a PubMed and EMBASE search. A total of 12 reports on phase-3 clinical trials and observational studies of COVID-19 vaccines were included in the review. In terms of vaccine safety, mRNA vaccines showed more relevance to serious adverse events than viral vector and inactivated vaccines, but no solid evidence indicated that COVID-19 vaccines directly caused serious adverse events. Serious metabolic, musculoskeletal, immune-system, and renal disorders were more common among inactivated vaccine recipients, and serious gastrointestinal complications and infections were more common among viral vector and inactivated vaccine recipients. The occurrence of serious vessel disorders was more frequent in mRNA vaccines. In terms of efficacy, two mRNA vaccine doses conferred a lesser risk of SARS-COV-2 infection (odds ratio: 0.05; 95% confidence interval: 0.02–0.13) than did vaccination with viral vector and inactivated vaccines. All vaccines protected more against symptomatic than asymptomatic cases (risk ratio, 0.11 vs. 0.34), but reduced the risk of severe SARS-COV-2 infection. The COVID-19 vaccines assessed in this study are sufficiently safe and effective. The results indicate that two mRNA vaccine doses prevent SARS-COV-2 infection most effectively, but further research is needed due to the high degree of heterogeneity among studies in this sample. Interventions should be implemented continuously to reduce the risks of infection after one vaccine dose and asymptomatic infection.


Introduction
The 2019 coronavirus disease (COVID- 19) was first identified in Wuhan, China, in December of that year. COVID-19 is caused by a novel coronavirus [severe acute respiratory syndrome coronavirus 2 (SARS-COV-2)], similar to the other two coronaviruses-SARS-CoV and MERS-CoV [1]. Common manifestations of COVID-19 include cough, fever or chills, and shortness of breath, making these diseases difficult to distinguish from influenza [2]. Compared with SARS, MERS and influenza, COVID-19 is transmitted more easily from human to human through droplets and has a higher infection rate, but a lower mortality rate [1]. Given these properties, it spread from its origin to other regions and countries at unexpected speed, causing a global pandemic. Currently, more than 100 million people have been infected globally, and about 2.9 million deaths have been registered globally; this disease has posed an enormous challenge to health systems worldwide [3]. Infection control measures, such as wearing masks, the suspension of public gatherings, school closures, and travel restrictions, remain the mainstays of prevention, although they have seriously impacted daily life and the global economy [4]. Because a standard therapy virus antigen, such as the SARS-COV-2 spike (S) protein or receptor-binding domain (RBD), is manufactured via genetic engineering, and expressed in cells in vitro. When injected into the human body, it elicits the production of corresponding antibodies directly and rapidly [13]. Because these vaccines do not contain a live virus, they are considered to be safe and effective. Phase-1 and -2 results have been reported for NVX-CoV2373 (Novavax, USA) and ZF2001 (Longcom, China), COVID-19 vaccines of this type, and phase-3 clinical trials for them are underway [14,15]. Preliminary analyses indicated that NVX-CoV2373 (Novavax, USA) induces high antibody titers against the SARS-COV-2 S protein, blocking attachment of the virus to human angiotensin-converting enzyme 2, and that ZF2001 (Longcom, China), which targets the RBD of the SARS-COV-2 S protein, effectively blocks receptor binding.
Clear evidence to guide the choice of appropriate vaccines among the variety of candidates for population-based vaccination in different locations is lacking. This systematic review and meta-analysis of the safety and efficacy of vaccines developed using different platforms was conducted to provide more evidence to optimize COVID-19 vaccine usage.

Search Strategy
To explore the efficacy and safety of available COVID-19 vaccines for which phase-3 trial data were published as of 30 May 2021, the PubMed database and EMBASE were searched with no restriction on language or year of publication, in accordance with the PRISMA guidelines [16] ( Table 1). The keywords used for the PubMed search were "COVID-19 vaccine*" AND [(Efficacy OR Effectiveness) OR Safety], and the subheadings used for the EMBASE search were (exp SARS-COV-2 vaccine/or COVID-19 vaccine.mp.) AND [(Efficacy.mp. OR Effectiveness.mp.) OR (Safety.mp. or exp Safety/)]. Additionally, a systematic electronic search of ongoing studies and clinical trials was conducted using the COVID-19 section of the ClinicalTrials.gov database. The systematic review included studies reporting on SARS-COV-2 vaccines for which phase-3 clinical trial data were available, regardless of vaccine platforms, route of administration and doses. Only full-text reports on randomized clinical trials and observational studies of the efficacy and safety of the vaccines were included, to ensure that a sufficient amount of evidence was obtained for evaluation. Selected articles provided data on the incidence of serious adverse events and cases of (post-vaccination) SARS-COV-2 infection in vaccine and control groups. No restriction on the region in which the trial was undertaken, population examined, or virus variants included was imposed (Table 1).

Exclusion Criteria
Review articles, editorial papers, letters, and preprints were excluded from the review. Reports on animal models, in vitro studies, study protocols, preclinical trials, and phase-1 and -2 clinical trials were also excluded.

Data Extraction and Publication Quality Assessment
Publication details (including the authors and year of publication) and data on the study design, vaccine name, platform, number of doses, population, route of administration, number of people experiencing serious adverse events, number of (symptomatic and asymptomatic) COVID-19 cases, and the severity of SARS-COV-2 infections in vaccine and control groups were collected from eligible publications. Using the "Quality in prognostic Studies (QUIPS) tool" [17], the quality of each included study was assessed in the following domains: study population, study attrition, prognostic factor measurement, outcome measurement, study confounding, and statistical analysis and reporting. Given that a small number of eligible articles had been published at the time that this systematic review was conducted, publication bias was examined as described in Section 2.4.

Data Analysis
Currently, symptoms such as fever, headache, fatigue, and vomiting are reported very commonly by vaccine recipients. The mild and moderate adverse reactions occurring in most cases are not life threatening; some serious adverse reactions, however, require hospitalization and are potentially fatal [18]. Thus, the occurrence of serious adverse events is a very important indicator in the evaluation of vaccine safety. Pooled estimates of odds ratios (ORs) based on the number of study participants experiencing serious adverse reactions and the proportions of serious adverse events (by system organ class) occurring after the receipt of at least one vaccine dose were used to evaluate vaccine safety. Vaccine efficacy was evaluated using pooled risk ratios (RRs) for the numbers of COVID-19 cases occurring after one and two vaccine doses, the numbers of symptomatic and asymptomatic cases occurring after two vaccine doses, and the severity of SARS-COV-2 infections occurring after full vaccination. Meta-analysis of vaccine safety and efficacy were conducted separately, and analysis of subgroups defined according to the vaccine platform and case type were performed to characterize diversity among studies. Vaccine efficacy was determined using the formula (1 − RR) × 100%.
Heterogeneity among studies was evaluated using the Higgins statistic (I 2 ), with sequential omission of individual studies performed to identify the source of the heterogeneity. When I 2 values were >50%, a random effects model was applied to compute the overall results; otherwise, a fixed effects model was used. Begg's funnel plots were used to examine publication bias. Asymmetrical plots were constructed to reflect the potential existence of such bias and were adjusted using the trim-and-fill method. All data analyses were performed using Review Manager 5.4 and R 4.0.3 software.

Study Selection
In the initial literature search, 3100 potential articles were identified up to 30 May 2021 (1894 in PubMed, 1202 in EMBASE and four in the ClinicalTrials.gov database). After the removal of 586 duplicate articles, the titles and abstracts of the remaining 2514 articles were screened, and 2480 articles were excluded based on the inclusion and exclusion criteria. The assessment of the full texts of the remaining articles led to the inclusion of 12 articles describing eight clinical trials and four observational studies in the literature review ( Table 2). A flowchart of the literature search is provided as Figure 1. In the initial literature search, 3100 potential articles were identified up to 30 May 2021 (1894 in PubMed, 1202 in EMBASE and four in the ClinicalTrials.gov database). After the removal of 586 duplicate articles, the titles and abstracts of the remaining 2514 articles were screened, and 2480 articles were excluded based on the inclusion and exclusion criteria. The assessment of the full texts of the remaining articles led to the inclusion of 12 articles describing eight clinical trials and four observational studies in the literature review (Table 2). A flowchart of the literature search is provided as Figure 1.
The included studies were conducted in different countries and regions with at least 8000 participants each, and dropout rates were very low. In addition, most studies were undertaken with high-risk populations, such as healthcare workers. Generally, vaccination required two doses administered at a 28 or 21 day interval, and all participants were monitored for some time to evaluate vaccine safety and efficacy. Table 2 summarizes information extracted from the selected literature.

Vaccine Safety
Meta-analyses of vaccine safety were conducted in six studies, with eight comparisons of ORs, because Baden et al. [19] evaluated the occurrence of serious adverse events after one and two vaccine doses, and Sinopharm produced two inactivated vaccines developed from the SARS-COV-2 WIV04 and HB02 strains. BBIBP-CorV, which was developed from the HB02 strain, in collaboration with the Beijing Institute of Biological Products, has been listed as an emergency-use COVID-19 vaccine [30]. All adverse events occurring after vaccination in the included studies were reported using Medical Dictionary for Regulatory Activities (MedDRA) terms [18]; and serious adverse events (hospitalization, death, and other life-threatening events) were classified using the US Food and Drug Administration's Toxicity Grading Scale [31]. The risk of serious adverse event occurrence after at least one dose was greater for mRNA vaccines [OR = 1.47; 95% confidence interval (CI), 0.65-3.29] than for non-replicating viral vector (OR = 0.76; 95% CI, 0.62-0.93) and inactivated (OR = 0.79, 95% CI, 0.62-1.00) vaccines (Table 3), but the 95% CIs indicate the lack of clear evidence for a direct relationship between the receipt of any of these vaccines and serious adverse reactions. Although no significant association between serious adverse events and vaccines was observed in the studies, with the exception of that of Baden et al., which reported a significant association between the receipt of the second vaccine dose and the occurrence of serious adverse events (OR = 4.63; 95% CI, 1.33-16.13) [19], obvious heterogeneity among the three vaccine subgroups was observed (I 2 = mRNA vaccines 61% vs. viral vector vaccines 0% vs. inactivated vaccines 0%; Figure S1). Most heterogeneity derived from the mRNA vaccine subgroup. Thus, a sensitivity analysis was conducted using the sequential omission approach. The omission of data from Baden et al. [19] reduced the degree of heterogeneity to 23.8%, which can be regarded as no significant heterogeneity in other studies. Hence, it was indicated that there was no significant association between serious adverse events and COVID-19 vaccines.
Serious adverse events in all system organ classes were rare (Table 4). Serious metabolic (19%; 14%), musculoskeletal (9%; 13%), immune-system (8%; 6%) and renal disorders (17%; 8%) were more common among recipients of the two inactivated vaccines, and serious gastrointestinal tissues and infections were reported more commonly among recipients of the viral vector and inactivated vaccines (9-34%), which were less seen in mRNA vaccines (<5%). Serious vessel or lymphatic disorders, such as thrombosis or lymphadenitis, were more easily observed in mRNA vaccines, although these disorders also occurred in the two other vaccine platforms.

Vaccine Efficacy
Data from six and 10 publications were used to estimate vaccine efficacy after one and two doses, respectively, with random-effects models used to pool RRs, because >50% heterogeneity among studies was detected (Figures S2-S4). The overall RR for SARS-COV-2 infection after two vaccine doses was 0.17 (95% CI, 0.07-0.40; Table 3), indicating that the vaccines examined in the included studies are effective. The RR for SARS-COV-2 infection was lower by about 0.3 for the mRNA vaccines (0.05; 95% CI, 0.02-0.13) than for the nonreplicating viral vector (0.33; 95% CI, 0.22-0.50) and inactivated (0.32; 95% CI, 0.23-0.42) vaccines, with the latter two groups showing similar efficacy. A subgroup analysis of efficacy after single doses of the mRNA and non-replicating viral vector vaccines (no such data were reported for the inactivated vaccine) yielded an overall RR for SARS-COV-2 infection that was 25% higher than the RR for infection after two doses (post single dose: 0.42; 95% CI, 0.20-0.89; post two doses: 0.17; 95% CI, 0.07-0.40), and a significantly higher RR for the mRNA vaccines than for the viral vector vaccines (0.53; 95% CI, 0.23-1.20 vs. 0.29; 95% CI, 0.11-0.76).
A subgroup analysis of symptomatic and asymptomatic events occurring after two vaccine doses was conducted with pooled data from 10 publications. It showed that the vaccines provided about 84% protection against all SARS-COV-2 infection, 89% protection against symptomatic cases, and 66% protection against asymptomatic cases overall ( Figure 2).

Vaccine Efficacy
Data from six and 10 publications were used to estimate vaccine efficacy after one and two doses, respectively, with random-effects models used to pool RRs, because >50% heterogeneity among studies was detected (Figures S2-S4). The overall RR for SARS-COV-2 infection after two vaccine doses was 0.17 (95% CI, 0.07-0.40; Table 3), indicating that the vaccines examined in the included studies are effective. The RR for SARS-COV-2 infection was lower by about 0.3 for the mRNA vaccines (0.05; 95% CI, 0.02-0.13) than for the non-replicating viral vector (0.33; 95% CI, 0.22-0.50) and inactivated (0.32; 95% CI, 0.23-0.42) vaccines, with the latter two groups showing similar efficacy. A subgroup analysis of efficacy after single doses of the mRNA and non-replicating viral vector vaccines (no such data were reported for the inactivated vaccine) yielded an overall RR for SARS-COV-2 infection that was 25% higher than the RR for infection after two doses (post single dose: 0.42; 95% CI, 0.20-0.89; post two doses: 0.17; 95% CI, 0.07-0.40), and a significantly higher RR for the mRNA vaccines than for the viral vector vaccines (0.53; 95% CI, 0.23-1.20 vs. 0.29; 95% CI, 0.11-0.76).
A subgroup analysis of symptomatic and asymptomatic events occurring after two vaccine doses was conducted with pooled data from 10 publications. It showed that the vaccines provided about 84% protection against all SARS-COV-2 infection, 89% protection against symptomatic cases, and 66% protection against asymptomatic cases overall (Figure 2). For the meta-analysis of SARS-COV-2 infection severity data, data on severe cases, those requiring hospitalization, and those resulting in death from seven, three, and two For the meta-analysis of SARS-COV-2 infection severity data, data on severe cases, those requiring hospitalization, and those resulting in death from seven, three, and two publications, respectively, were used. RRs for severe cases, cases requiring hospitalization, and cases resulting in death between the vaccine and control groups were 0.12 (95% CI, 0.05-0.30), 0.08 (95% CI, 0.02-0.24), and 0.14 (95% CI, 0.03-0.69), respectively (Table 5).

Publication Bias and Risk of Bias Assessment
Obvious asymmetry was observed in Begg's funnel plot for safety and efficacy of vaccines, which revealed potential publication bias existing in the selected articles. The trim-and-fill method was used to adjust for funnel plot asymmetry, and the adjusted results are shown in Figure 3, where the results of the pooled analysis are similar to those of the original analysis in terms of safety and efficacy evaluation.

Publication Bias and Risk of Bias Assessment
Obvious asymmetry was observed in Begg's funnel plot for safety and efficacy of vaccines, which revealed potential publication bias existing in the selected articles. The trim-and-fill method was used to adjust for funnel plot asymmetry, and the adjusted results are shown in Figure 3, where the results of the pooled analysis are similar to those of the original analysis in terms of safety and efficacy evaluation.
As described in Table 6, eight articles presented a low risk of bias, whereas four articles were regarded as a moderate risk of bias based on the QUIPS tool assessment.  As described in Table 6, eight articles presented a low risk of bias, whereas four articles were regarded as a moderate risk of bias based on the QUIPS tool assessment.

Discussion
This systematic review and meta-analysis were conducted to explore the associations of COVID-19 vaccines with the occurrence of serious adverse events, and to compare the protection provided by different COVID-19 vaccines and the vaccines' efficacy against different infection types. The analyses provided the best evidence that the current COVID-19 vaccine candidates are not associated directly with serious or life-threatening adverse events. However, the risk ratios for serious adverse events occurring in different organ systems differed among vaccine platforms, suggesting the existence of a relationship between COVID-19 vaccine platforms and serious adverse events, which should prompt further exploration to avoid potential risk for populations with relative diseases. For example, the population with metabolic, musculoskeletal, immune-system, and renal disorders should avoid the receipt of inactivated COVID-19 vaccines, and it is suggested that patients who have vessel diseases should not receive mRNA vaccines. Most reviews and meta-analyses of COVID-19 vaccine safety conducted to date have focused on the risk of adverse events, with less attention paid to the occurrence of serious adverse events related directly to mortality and morbidity. Our analysis of COVID-19 vaccine efficacy was more comprehensive than that of most previous research, which has been limited to the examination of risk ratios for SARS-COV-2 infection after whole-course vaccination. Our findings reveal that all current vaccine platforms provide sufficient protection against SARS-COV-2 infection and substantially decrease the risk of serious infection. mRNA vaccines are most effective against SARS-COV-2 infection. Our findings also suggest that the risk of SARS-COV-2 infection after receipt of the first vaccine dose is almost 2.5-fold greater than that after receipt of the second dose. Given the improved efficacy of the vaccines after the second dose, people should continue to follow protective measures, such as mask wearing and social distancing, after receipt of the first dose. Although the COVID-19 vaccines provide >50% protection against symptomatic and asymptomatic infections, they provide three-fold greater protection against symptomatic than against asymptomatic infections. Thus, measures are required to avoid the risk posed by vaccinated individuals with asymptomatic infections. For example, large-scale screening in high-risk populations may aid the timely detection of asymptomatic cases and reduce the risk of SARS-COV-2 infection spread. Table 6. Risk of bias assessment based on Quality in Prognostic Studies (QUIPS) tool (low = "low risk", middle = "middle risk", high = "high risk"). Another strength of this review is that it synthesized data from phase-3 clinical trials and observational studies based on a strict search strategy, which ensured the acquisition of strong evidence on the topic. However, the review has several limitations. First, the included studies displayed heterogeneity due to differences in study design and sample size. For example, most studies showed that the COVID-19 vaccines protected against SARS-COV-2 infection, with efficacy improving after the second dose. However, the protection provided by viral vector vaccines decreased by 4%. This finding is not consistent with the conclusion drawn by Merryn Voysey et al. [29], who found no difference in vaccine efficacy against asymptomatic cases relative to the control group, possibly due to the inclusion of asymptomatic cases in the analysis and the small sample. Second, the results may be imbalanced, because five of the 12 publications included reported on BNT162b2. Third, visual inspection of the funnel plots revealed asymmetry reflecting publication bias [32]. Thus, the funnel plots were adjusted using the trim-and-fill approach, which did not affect the significance of the outcomes.

Biases
Large-scale vaccination is currently being implemented in many countries and regions to block further spread of SARS-COV-2, and the safety and efficacy of COVID-19 vaccines influence public acceptance and progress toward universal vaccination. With the rapid global mutation of SARS-COV-2, a long-term surveillance system for the safety and efficacy of COVID-19 vaccines is needed, and more research will be required on different vaccine platforms to optimize community vaccination and effectively slow SARS-COV-2 spread.

Conclusions
This study showed that full COVID-19 vaccination protects most people from SARS-COV-2 infection and reduces the severity of COVID-19. Evidence on the ability of COVID-19 vaccines to cause serious adverse events is insufficient; more data are needed. In addition, data on vaccines developed using other platforms, such as protein subunit vaccines, were not published at the time that this review was conducted, and the articles included in this review exhibited a high degree of heterogeneity. Thus, more research is needed to promote universal vaccination against SARS-COV-2.
Supplementary Materials: The following are available online at https://www.mdpi.com/article/10 .3390/vaccines9090989/s1, Figure S1. Forest plot of association between serious adverse events and COVID-19 vaccine, Figure S2. Forest plot of association between COVID-19 cases and vaccination after the 2 dose, Figure S3. Forest plot of association between COVID-19 cases and vaccination after the first dose, Figure S4