E-health literacy and the vaccination dilemma: an Israeli perspective
Noa Aharony and Romina Goldman.
Introduction. Parents' hesitancy to use vaccines is becoming more prevalent in Israel. The aim of the current research is to examine whether there is a relationship between the increase in parents’ level of involvement in the decision to vaccinate their children, and the increase in searching for and filtering information from online electronic sources. The current research is based on Hochbaum and Rosenstock's health belief model from 1952 and the Lily model, which relates to e-health literacy.
Method. The research population consists of 200 parents of children born from 2000 forward. Researchers surveyed five aspects to gather data: demographic characteristics, vaccination intentions, level of online information sources use, knowledge on the subject of vaccination and perceived e-health literacy.
Results. Findings show that most of the parents who hesitated over whether to vaccinate their children decided not to vaccinate them. These parents believe they can search for information on the Internet and they surf different kinds of sites (government, social network sites, forums and alternatives).
Conclusion. This study expanded the theoretical literature that deals with e-health and vaccination and may help directors of medical institutions to find solutions of how to lower the number of individuals who refuse to vaccinate their children.
Introduction
Parents are usually asked to vaccinate their children against chickenpox, measles, diphtheria, hepatitis a and b, polio, tetanus and many other diseases. However, parents' refusal to use vaccines is becoming more prevalent in Israel. Various studies have suggested that vaccination refusers are older, and have higher levels of education than average (Salmon et al., 2005; Smith et al., 2011). Further, Mikulak (2011) suggests that vaccination refusers are characterised by a rational decision resulting from an intensive process of information research. Brunson (2013) maintains that conformer (non-hesitant) parents usually vaccinate without being aware of the diseases against which they are vaccinating and do not question medical authorities. However, active (hesitant) parents tend not to rely on the information supplied to them by others and carry out independent research. After evaluating the information, these parents agree to the vaccination, postpone it, or refuse it. This process may repeat itself several times (Brunson, 2013).
Theoretical background
Vaccination refusers
Individual or organised resistance to vaccination has been known since the end of the 18th century. In 1998, the controversy over vaccination made headlines after the publication of Andrew Wakefield’s paper in the British medical journal, The Lancet. Wakefield pointed to a causal connection between the triple vaccine against measles, mumps and rubella, and autism in children and babies. This paper led to extensive media coverage, a drastic reduction in the number of children who were vaccinated (Middleton and Wolfe, 2013), and a renewed outbreak of these diseases in England (Burgess, Burgess and Leask, 2006; Wolfe and Sharp, 2002).
In February 2004, the Sunday Times newspaper published an investigation that cast doubts on Wakefield’s research, discovering that the research had been based on a conflict of interest. Twelve years after Wakefield's publication, The Lancet retracted the paper (Deer, 2011). Even though the scientific community considered Wakefield’s findings invalid, the anti-vaccination movement still uses them to advance its agenda by maintaining that the researcher was a victim of the scientific community and pharmaceutical companies (Leask, Chapman, Hawe and Burgess, 2006; Middleton and Wolfe, 2013; Poland and Jacobson, 2012).
Unlike resistance to vaccination in the 18th Century, which was based on religious beliefs, the present-day phenomenon of vaccination refusal is associated with a rational decision based on a thorough process of information research (Bazin, 2009; Betsch and Sachse, 2012; Mikulak, 2011). This phenomenon is part of a post-modern medical paradigm, in which patients are actively engaged in searching for information before making any decision on medical matters (Kata, 2011). Kata maintains that parents who struggle with vaccination decisions are more likely to use a wider variety of resources before making a decision on whether to vaccinate their children. A study conducted in Israel (Aharon, Nehama, Shmuel, and Baron-Epel, 2012) examined the demographic data of children who were not fully vaccinated, revealing a difference between those who were not vaccinated as a consequence of their parents’ decision and those who were not vaccinated because of organisational barriers. Organizational barriers refer to technical and financial difficulties that prevent the population from reaching family health centres (Aharon, 2011). Findings reveal that there is a greater likelihood that a child will not be vaccinated because of a parental decision if the mother is Jewish, has an academic degree, and is over 39 years old. However, the likelihood that a child will not be vaccinated because of organisational barriers is higher among Muslims, having parents who do not have an academic degree, and having a young mother (Aharon et al., 2012). In a study that was carried out in the United States, Smith, Chu and Barker (2004) found that children who were vaccinated tended to be black, had a younger mother who was not married and did not have a college degree, and lived in a household near the poverty level. The unvaccinated children tended to be white, had a married mother who had a college degree and with an annual income exceeding $75,000.
The health belief model
The health belief model was developed at the beginning of the 1950s to predict conditions under which people may act to prevent, screen for, or control illness (Champion and Skinner, 2008). More recent studies that were based on the health belief model tried to predict in which conditions the individual will decide if to vaccinate or not (Coe, Gatewood and Moczygemba, 2012; Smith et al., 2011). The model can be applied to parents to demonstrate the process used by non-conformer or hesitant parents who decide to refuse vaccinations. The characteristics of the model, applied to the process of vaccination refusal, are:
Perceived susceptibility
The individual’s subjective perception of his or her chances of becoming ill with a particular disease (Champion and Skinner, 2008; Strecher and Rosenstock, 1997). Extensive vaccination programmes have caused the eradication of many diseases. Young parents have usually not experienced these diseases, thus their subjective perception may be that the chance of their child becoming ill with the disease is very low (Bond and Nolan, 2011). Another study (Brewer et al., 2007) proposed that those who perceived themselves to be more susceptible to an illness were more likely to be vaccinated.
Perceived severity
The individual’s subjective perception of the severity of the disease and its medical consequences (death, disability, or pain) or its social consequences (work, family life and social relations) (Champion and Skinner, 2008; Strecher and Rosenstock, 1997). Vaccination refusers believe that the diseases are not as dangerous as medical authorities describe, and it is possible to heal them through natural processes without external (specifically, normative medical) intervention (Kasmaei et al., 2014; Luthy, Beckstrand, Callister and Cahoon, 2012).
Perceived benefits
The individual’s perceived benefits of the various available actions for reducing the disease threat. Parents who do not believe in the disease’s severity do not see any benefit in taking preventative action (Champion and Skinner, 2008; Strecher and Rosenstock, 1997). Vaccination refusers do not perceive the disease is as dangerous as commonly believed and therefore any preventative measure, such as vaccination, will not be effective, and will even be perceived as harmful (Luthy et al., 2012).
Perceived barriers
These are factors that prevent the individual from taking preventative measures, such as severe side effects of the treatment (Champion and Skinner, 2008; Strecher and Rosenstock, 1997). Side effects that may appear as a consequence of vaccination (such as autism, allergies, paralysis, etc.) are perceived as more dangerous than the disease itself (Betsch and Sachse, 2012).
The Internet as a tool for seeking information on medical topics
A study that examined information-seeking behaviour on medical topics on the Internet in the United States discovered that young people, especially white adults earning more than $75,000 a year, and those with bachelor’s degrees and higher, are more likely to search the Internet to solve medical problems. When the participants in the study were asked where they searched for information, 77% answered that initially they used search engines such as Google, Bing or Yahoo; 13% began their search on professional Websites such as WebMD; 2% used Websites such as Wikipedia, and only 1% used social media such as Facebook (Fox and Duggan, 2013). Moreover, 52% believe that ‘nearly all’ or ‘most’ of the medical information on the Internet is reliable and 70% agreed that the information they find on the Internet influences their choice of medical treatment (Kata, 2010).
The ability to use reliable sources of information and discern between high and low quality health information is very important (Austin, Pinkleton, Austin and Van de Vord, 2012). Studies show that people who search for health information frequently base their impression of the reliability of a Website according to its design and ease of use, not on the quality of the information it contains (Austin et al., 2012). The individual can interpret the information they retrieve in various ways, without any of these interpretations considered invalid. Therefore, Websites that create the most coherent and concise content will have a greater influence on information consumers (Betsch and Sachse, 2012).
In light of studies showing that most people who look for online information do not carry out an in-depth investigation of the sources of that information, it can be assumed that the likelihood of reaching unreliable information by this method (and also receiving messages against vaccination) will increase (Bean, 2011; Kata, 2011). These messages may influence the decision to vaccinate. In addition, the transition to Web 2.0 has made the issue of the quality of health information appearing in different Websites more problematic. The fact that anyone can contribute easily to health discourse, frequently in an anonymous way, is an increasing source of concern regarding the accuracy and quality of the information (Tafuri et al., 2014; Witteman and Zikmund-Fisher, 2012).
Previous studies maintained that the Internet had little influence on the process of deciding whether to vaccinate (Fredrickson et al., 2004; Salmon et al., 2005). However, more recent studies present a change of this trend. A study which explored the associations between parents’ Internet information-seeking and their knowledge, attitudes and beliefs about human HPV vaccination, found that those who searched for information on the Internet had higher knowledge, and positive attitudes and beliefs about HPV vaccination (Mcree, Reiter and Brewer, 2012). Further, Harmsen et al. (2013) propose that although parents perceived health care workers as the most reliable source of information about vaccination, they also searched the Internet as they feel that the information that was given to them did not present the vaccination side effects.
A study showed that though there was no change in the intention to vaccinate within the group that received information in favour of vaccination but those who surfed anti-vaccination Websites increased their perceived risk of vaccination by 28%: according to the findings, viewing typical vaccine-critical Websites for only 5-10 minutes increases the perception of risk regarding vaccinations (Betsch, Renkewitz, Betsch and Ulshöfer, 2010). A recent study supports Betsch et al.’s findings (Dunn, Leask, Zhou, Mandl and Coiera, 2015). In accordance with this, Ruiz and Barnett (2015), who conducted a semantic analysis of the words retrieved on Google when searching for information about the HPV virus, found that there were more negative than positive words concerning vaccinations and their side effects and other aspects.
Betsch and Sachse (2012) identified characteristics that can increase the likelihood that users who conduct Internet research will acquire unreliable information on vaccination. These are: lower socioeconomic status, lower cognitive ability, lower health literacy, and less knowledge about vaccination. These findings contrast with previous ones showing that people who oppose vaccination were parents with a higher socioeconomic status and higher cognitive ability (Smith et al., 2011).
Other studies that examined the characteristics of Websites identified with the anti-vaccination movement (Bean, 2011; Davies, Chapman, and Leask, 2002; ; Zimmerman et al., 2005) suggested that behind these Websites were groups that promoted an anti-vaccination agenda by presenting themselves as legitimate authorities with scientific credibility. Most of the Websites propounded the scientific validity of their claims by referencing literature dominated by self-published works and the alternative medical press (Davies, Chapman, and Leask, 2002). Betsch’s (2011) research found that people with less knowledge on vaccination, who are more likely to conduct searches, will do so using less complex search terms which lead to more anti-vaccination Websites.
Health literacy in the era of the Internet
Health literacy relates to the individual’s abilities to cope with the complex demands necessary in order to deal with the modern health system. The health literate person has to know how to place in the right context their own health, the health of their family, and the health of the community, in order to understand factors which affect them, and know how to handle them. Researchers have shown that health literacy can lead to increased health knowledge, improved individual health, reduced government expenses, fewer hospitalisations and, finally, contributed to reduced use of these services (Sørensen et al., 2012).
The growing influence of the Internet as an initial source of information on medical topics has given rise to a new concept: e-health literacy. According to Norman and Skinner (2006a, p. 1), this type of literacy relates to the ‘ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem’. Norman and Skinner (2006a) translated their definition of e-health literacy to the Lily model, which is based on a combination of six different kinds of literacies, and aims to ensure an optimal health decision-making process. The six kinds of literacies can be divided into two types: analytic, which includes traditional literacy, media literacy and information literacy, and the content type that includes computer literacy, scientific literacy and health literacy.
Schulz and Nakamoto (2011) maintained that the flexibility afforded by the Internet as a tool for making medical decisions can sometimes be dangerous rather than beneficial. Instead of encouraging the information consumer to apply a methodical, thorough approach, search engines are liable to encourage users to take shortcuts, and thus the individual can ignore critical considerations. Further, studies show that searching anti-vaccination sites increases the individual's level of insecurity regarding vaccination (Betsch, et al., 2010; Dunn et al., 2015; Guidry, Carlyle, Messner and Jin, 2015).
Problem statement
The Pew Research Center (2015) reveals that about 83% of the public in the US say that vaccination for diseases such as measles, mumps and rubella (MMR) is safe for healthy children, 9% think that such vaccinations are not safe and 7% report that they do not know. Thus, due to the growth in the number of vaccination refusers, the question arises as to whether those who are non-conformers and hesitant about vaccination make their decision after a sophisticated and comprehensive search. Do they search for information in medically reliable sites? Are they impressed by the ease of use of the site and its design? Are they influenced by videos and pictures presented or the quality of information it contains? Do they know how to distinguish between reliable and invalid information? This question is important because researchers assume that as people are exposed to the enormous amount of information on the Internet, they may be affected by irrelevant, false information before making their medical decisions (Schulz and Nakamoto, 2011).
The current study focuses on hesitant and non-hesitant parents and its aim is to examine whether there is a difference between these two groups concerning the relationship between the increase in parents’ level of involvement in the decision to vaccinate their children, and the increase in searching for and filtering information from online electronic sources. The study will focus on parents’ perceived level of online literacy (e-health), their level of electronic information sources use and types of information sources, and their actual level of knowledge on the subject of vaccination.
The research may expand the theoretical literature that deals with e-health and vaccination (Betsch and Wicker, 2012). In addition, the study results may help directors of medical institutions to find solutions of how to lower the number of individuals who refuse to vaccinate their children.
The objectives of this study are to investigate: (a) What kind of sites do hesitant and non-hesitant parents use before they make their decisions concerning vaccination?; (b) Are there differences between hesitant and non-hesitant parents concerning their perceived e-health literacy?; (c) Are there differences between hesitant and non-hesitant parents concerning their level of knowledge about vaccinations?
Hypotheses
Assuming that parents’ level of searching for and filtering information from different online electronic sources, their level of e-health literacy, and their actual knowledge on the subject of vaccination, may affect their decision whether to vaccinate or not, the following hypotheses can be made:
H1. Hesitant parents will have broader knowledge concerning vaccination than non-hesitant parents.
H2. No significant difference concerning perceived level of e-health literacy will be found between hesitant and non-hesitant parents.
H3. Hesitant parents will have greater likelihood to refuse vaccination than non-hesitant parents.
H4. Hesitant parents’ use of online electronic resources will be greater than non-hesitant parents.
H5. Parents who refuse to vaccinate their children (refuser parents) will have higher perceived level of e-health literacy and more limited knowledge about vaccination than non-refuser and hesitant parents.
H6. Parents who hesitated in the past to vaccinate and, after searching for information, decided to vaccinate, have higher perceived e-health literacy, and higher knowledge concerning vaccination than non-refuser and non-hesitant parents.
H7. Refuser parents will use more information resources than non-refuser parents.
H8. Parents who received information on Web 2.0 sites have lower knowledge about vaccination than parents who received information on other sites
Method
Data collection
The study took place in December 2014 and January 2015. The researchers wanted to reach parents who use electronic information sources, thus, they addressed Facebook groups, and forums whose main concern was parents’ discourse on raising children, focusing on their education and health. In the first stage they asked Facebook groups and forums’ managers for permission to disseminate questionnaires to their groups. After getting their consent, they sent questionnaires to two forums and to ten Facebook groups. The researchers received 210 questionnaires. The questionnaire was distributed to a convenience sample; therefore it does not represent the entire population of parents in Israel.
Data analysis
Quantitative data analysis consisted of Pearson correlations in order to examine the relationships between the research variables. MANOVA and a chi-squared test analyses were performed to compare the various groups.
The research sample consisted of 210 parents of children born between 2000 and 2014. Of these, 174 (82.9%) were females and 36 (17.1%) were males. 125 (59.5%) were hesitant, and 85 (40.4%) were non-hesitant. As for their level of education, 49 (22.3%) were high-school graduates, 91 (43.4%) had a bachelor degree, and 70 (34.3%) had a masters degree.
Measures
Researchers surveyed five aspects to gather data: demographic characteristics, vaccination intentions, level of online information sources use, knowledge on the subject of vaccination and perceived e-health literacy (see Appendix for copy of questionnaire).
The demographic section consists of six questions.
The intentions section consists of three questions that make it possible to determine whether the parent is a refuser, non-hesitant or hesitant. A refuser parent is one who does not vaccinate their children, a non-hesitant is a parent that either vaccinates their children or refuses to vaccinate them, and a hesitant one is a parent that is uncertain as to whether to vaccinate or not.
The section on level of online information sources use has four statements rated on a 5-point Likert scale (1= no knowledge; 5= a great amount of knowledge). Respondents were asked to mark the degree to which they used online information sources in order to receive information on the subject of vaccination. Internal reliability was examined by Cronbach’s Alpha and found to be 0.79 (Dunsky-Baranga, 2007).
The section on knowledge of the subject of vaccination consists of nine statements. The aim was to examine the parents’ level of scientific knowledge on the subject of vaccination. Respondents were asked to mark: ‘True’, ‘False’, or ‘I don’t know’. The tool was developed by Zingg and Siegrist (2012) and has a reliability of ρ = 0.79. The questionnaire’s reliability was tested using the Mokken scale (equivalent to the Cronbach’s Alpha scale) (Zingg and Siegrist, 2012). For the purpose of analysing the data, the participant received one point for each correct answer, and no points for each incorrect or unknown answer.
The perceived e-health literacy section consists of nine questions rated on a five-point Likert scale (1= strongest disagreement; 5= strongest agreement). Its aim was to examine how parents perceive their level of electronic medical literacy. The tool is taken from the model developed by Norman and Skinner (2006b). The internal reliability was examined by Cronbach’s Alpha and found to be 0.88 (Jordan, Osborne, and Buchbinder, 2011; Norman and Skinner, 2006b).
Results
A one-way MANOVA was performed to examine whether there are differences between hesitant and non-hesitant parents concerning knowledge about vaccination and perceived e-health literacy. The analysis revealed a significant difference concerning parents' knowledge, F (2.207) = 20.16, p < 0.001, η2= 1.16. Means, standard deviations and the MANOVA analysis for each group are presented in Table 1.
| Measures | Hesitant parents | Non-hesitant parents | F(2.207) | η2 | ||
|---|---|---|---|---|---|---|
| Mean | StdDev | Mean | StdDev | |||
| Perceived e-health literacy | 3.59 | 0.85 | 3.52 | 0.90 | 0.32 | 0.00 |
| Knowledge | 3.20 | 2.63 | 5.48 | 2.73 | 36.76* | 0.15 |
| * p < 0.001 | ||||||
Table 1 shows that non-hesitant parents have a greater knowledge about vaccination than hesitant parents. This result contrasts with H1. H2 was supported, revealing no significant difference between perceived e-health literacy of hesitant parents and non-hesitant parents. In order to examine differences between hesitant and non-hesitant parents concerning postponing vaccination, a chi square test was conducted. Significant differences between hesitant and non-hesitant were found concerning delaying vaccination, χ2=65.23, p <0.001, and concerning future vaccination, χ2=96.33, p <0.001. Figure 1 presents hesitant and non-hesitant parents' opinions concerning vaccination postponement.
Table 2 presents hesitant and non-hesitant parents' opinions concerning future vaccination.
| Category | Future vaccination | Total | ||||
|---|---|---|---|---|---|---|
| Yes | % | No | % | Number | % | |
| Hesitant | 30 | 24.0 | 95 | 76.0 | 125 | 100 |
| Non-hesitant | 79 | 92.9 | 6 | 7.1 | 85 | 100 |
| Total | 109 | 51.9 | 101 | 48.1 | 210 | 100 |
Results support H3, confirming that hesitant parents have a greater likelihood of refusing vaccination than non-hesitant parents. In order to examine if there are differences concerning Internet searching habits between hesitant and non-hesitant parents, researchers conducted a chi square test. Table 3 presents the analysis.
| Type of Website used and % | Non-hesitant | Hesitant | χ2 | ||
|---|---|---|---|---|---|
| Yes | No | Yes | No | ||
| Government | 58 | 67 | 41 | 43 | 0.18 |
| % | 46.4 | 53.6 | 49.4 | 50.6 | |
| Alternative | 47 | 78 | 10 | 75 | 17.08* |
| % | 37.6 | 62.4 | 11.8 | 88.2 | |
| Forum | 44 | 81 | 11 | 74 | 12.97* |
| % | 35.2 | 64.8 | 12.9 | 87.1 | |
| Social network | 54 | 71 | 21 | 64 | 7.54* |
| % | 43.2 | 56.8 | 24.7 | 75.3 | |
Table 3 shows that hesitant parents search more in alternative sites, forums, and social network sites than non-hesitant parents. This result confirms H4, which proposed that hesitant parents' Internet use is greater than that of non-hesitant parents.
When participants were asked if they intend to vaccinate their children in the future, 51.9% answered that they will vaccinate them, and 48.1% that they will not vaccinate them.
In order to examine if there is a change between parents' intentions before and after their information search concerning future vaccination, researchers conducted a chi square test that showed a significant difference between the two dates, χ2=96.33, p < 0.001. Figure 2 presents hesitant and non-hesitant parents' intentions concerning future vaccination after their information search.
Figure 2 shows that 92.9% of non-hesitant parents reported that after receiving the information, they intend to vaccinate their children. Among hesitant parents, 24.0% reported that after receiving the information, they intend to vaccinate their children, 49.6% decided not to vaccinate their children, and 26.4% continued hesitating.
In order to examine H5, that suggested that refuser parents will have higher perceived e-health literacy and lower knowledge about vaccination, a one-way MANOVA comparing three groups (non-refuser, refuser and hesitant parents) was conducted. A significant difference was found among the three groups, F(4.412)=34.72, p < 0.001, η2=0.25. Means, standard deviations, and the MANOVA analysis for each group are presented in Table 4.
| Measures | Non-refuser parents | Refuser parents | Hesitant parents | F(4.412) | η2 | |||
|---|---|---|---|---|---|---|---|---|
| Mean | StdDev | Mean | StdDev | Mean | StdDev | |||
| Perceived e-health literacy | 3.58 | 0.90 | 3.78 | 0.79 | 3.06 | 0.75 | 8.53* | 0.07 |
| Knowledge about vaccination | 5.78 | 2.49 | 1.96 | 1.96 | 3.00 | 2.27 | 61.29** | 0.37 |
| * p < 0.01; ** p < 0.0001 | ||||||||
Table 4 shows significant differences within the two measures. However, the differences concerning knowledge about vaccination among the three groups were greater than differences concerning perceived e-health literacy. Table 4 shows that non-refuser parents have the greatest knowledge about vaccination, followed by hesitant parents and refuser parents have the least knowledge. Further, table 4 presents that the refuser parents have the highest level of perceived e-health literacy, followed by non-refuser parents, then by hesitant parents. Researchers performed post-hoc tests, using the Bonferroni correction and found significant differences among the three groups. Means show that hesitant parents estimate they have the lowest level of perceived e-health literacy in comparison with two other groups.
H6, that assumed that parents who hesitated in the past to vaccinate and, after searching for information, decided to vaccinate, will have higher level of perceived e-health literacy, and greater knowledge about vaccination, was rejected. In order to see whether there are differences between refuser, and non-refuser parents, and between hesitant and non-hesitant parents a two way MANOVA (refuser / non-refuser X hesitant / non-hesitant) was conducted. No significant difference was found between refuser and non-refuser parents concerning their decision to vaccinate or not, F (2.203)=1.97, p > 0.05, and no significant difference was found between vaccination or non-vaccination and the hesitation process, F (4.408)=1.47, p > 0.05.
H7 and H8 suggested that refuser parents will use more information resources than other parents, and that parents who received information on Web 2.0 sites, have lower knowledge about vaccination than parents who received information on other sites. In order to examine whether there are differences among the three groups (non-refuser, refuser parents, and hesitant parents) concerning their information search in different sites (government, alternative, forums, and social network sites) a one way MANOVA was conducted. It found a significant difference among the three groups, F(8.410)=11.7, p <0.001, η2=0.19. Means, standard deviations, and the MANOVA analysis for each group are presented in Table 5.
| Websites | Non-refuser parents | Refuser parents | Hesitant parents | F(8.410) | η2 | |||
|---|---|---|---|---|---|---|---|---|
| Mean | StdDev | Mean | StdDev | Mean | StdDev | |||
| Government | 2.71 | 1.17 | 2.75 | 1.24 | 2.40 | 0.84 | 1.23 | 0.01 |
| Alternative | 1.77 | 1.05 | 3.28 | 1.24 | 2.65 | 1.25 | 36.74* | 0.26 |
| Forum | 1.87 | 1.13 | 3.21 | 1.10 | 2.94 | 1.39 | 30.20* | 0.22 |
| Social network | 2.21 | 1.32 | 2.65 | 1.25 | 3.16 | 1.23 | 11.38* | 0.09 |
| * p < 0.0001 | ||||||||
Table 5 shows that refuser parents search more than other parents in government sites, alternative sites, and forums, and that hesitant parents search more in social network sites. Table 5 presents significant differences among the three groups concerning searching in alternative sites, forums, and social network sites. The greatest differences were found regarding searching in alternative sites and in forums.
In order to examine the relationship between perceived e-health literacy, knowledge about vaccination and searching in different sites, researchers performed Pearson correlations, which are presented in Table 6.
| Measures | Government sites | Alternative sites | Forums | Social network sites |
|---|---|---|---|---|
| Perceived e-health literacy | 0.30** | 0.07 | 0.12 | 0.21** |
| Knowledge about vaccination | 0.15* | -0.51*** | -9.40*** | -0.15* |
| *p < 0.05 **p < 0.01 ***p < 0.0001 | ||||
Table 6 presents a significant, positive correlation between perceived e-health literacy and government sites and social network sites. Hence, the higher the perceived e-health literacy, the more people search for information in government sites and social network sites. Further, significant negative correlations were found between knowledge about vaccination and alternative sites, forums, and social network sites, and a positive correlation was found between knowledge about vaccination and government sites. It should be noted that the correlations between knowledge about vaccination and government sites and social network sites is lower in comparison with alternative sites and forums. It seems that the more knowledge parents have about vaccination the less they search in alternative sites, forums, social network sites, and the more in government sites.
Demographic variables
In order to examine whether there are differences between hesitant and non-hesitant parents concerning demographic variables (age, child's age, education, and income), a one way MANOVA was performed, that found a significant difference between the two groups, F(4.205)=3.22, p < 0.05, η2=0.06. Means, standard deviations, and the MANOVA analysis for each group are presented in Table 7.
| Measures | Hesitant parents | Non-hesitant parents | F(4.205) | η2 | ||
|---|---|---|---|---|---|---|
| Mean | StdDev | Mean | StdDev | |||
| Age | 33.92 | 5.68 | 36.21 | 5.15 | 8.86* | 0.04 |
| Child's age | 2.86 | 2.69 | 4.12 | 3.16 | 9.66* | 0.04 |
| Education | 2.52 | 1.04 | 2.70 | 1.24 | 1.26 | 0.01 |
| Income | 3.29 | 1.29 | 3.35 | 1.24 | 0.10 | 0.00 |
| *p < 0.01 | ||||||
Table 7 presents significant differences concerning parents' age and child's age. It seems that the non-hesitant parents are older and that the child's age is higher in comparison with hesitant parents.
In order to see whether there are differences between refuser and non-refuser parents concerning demographic variables, (age, child's age, education, and income), a one way MANOVA was performed, that found a significant difference between the two groups, F (4.205)=2.67, p < 0.05, η2. Means, standard deviations, and the MANOVA analysis for each group are presented in Table 8.
| Measures | Non-refuser parents | Refuser parents | F(4.205) | η2 | ||
|---|---|---|---|---|---|---|
| Mean | StdDev | Mean | StdDev | |||
| Age | 35.85 | 5.72 | 33.51 | 5.11 | 9.40* | 0.04 |
| Child's age | 4.05 | 3.43 | 2.46 | 1.80 | 15.99** | 0.07 |
| Education | 2.56 | 1.20 | 2.64 | 1.03 | 24.0 | 0.00 |
| Income | 3.24 | 1.28 | 3.42 | 1.25 | 1.04 | 0.01 |
| *p < 0.01 **p < 0.001 | ||||||
Table 8 presents similar findings to the ones found with hesitant and non-hesitant parents, meaning there are significant differences concerning parents' age and child's age. It seems that the non-refuser parents are older and that the child's age is higher in comparison with refuser parents.
Discussion
The research reported here focuses on hesitant and non-hesitant parents and its aim is to examine whether there is a difference between these two groups concerning the relationship between the increase in parents’ level of involvement in the decision to vaccinate their children, and the increase in searching for and filtering information from online electronic sources. Further, the study explored parents’ perceived level of online literacy (e-health), their level of electronic information sources use and their actual level of knowledge on the subject of vaccination.
H1 suggested that hesitant parents will have broader knowledge concerning vaccination than non-hesitant parents. The Betsch (2011) psychological model suggests that hesitant parents will search for information through various channels, will evaluate the vaccination benefits and risks, and then will decide whether to vaccinate or not. In contrast, non-hesitant parents will not search for information and therefore will not have broad knowledge about the vaccination. However, this hypothesis was rejected, and the findings reveal that hesitant parents had lower knowledge concerning vaccination than non-hesitant parents. We may assume that non-hesitant parents are not a tabula rasa and probably have prior knowledge about vaccination. We should add that the current research investigated only parents' use of online resources and we may speculate that non-hesitant parents received information from other traditional, authorised resources such as doctors, nurses etc. This finding was already presented in Jones et al. (2012), who revealed that 91.7% of non-hesitant parents receive most of their knowledge from the family doctor. The lower knowledge of hesitant parents may be associated with another study (Fox and Duggan, 2013) that showed that around 70% of hesitant parents used electronic resources to retrieve medical information. However, it is well known that a lot of the information found on the Internet is not reliable or valid (Bean, 2011). Thus, we may conclude that perhaps parents who searched for information on the Internet did not get precise information, and non-hesitant parents got information from other, authorised medical resources.
H2 suggested that no significant difference concerning perceived level of e-health literacy will be found between hesitant and non-hesitant parents, and was accepted. We assume that because the questionnaires were delivered via the Internet, only people who know how to use the Internet and have electronic literacy answered it. Therefore there was no difference concerning their perceived level of e-health literacy. Perhaps if the questionnaire was delivered via traditional ways, the results would have been different, as in this case it might involve people who possess and do not possess e-health literacy skills.
H3, that assumed that hesitant parents will have a greater likelihood of refusing vaccination than non-hesitant parents, was accepted, and shows that most parents who hesitated over whether or not to vaccinate, decided against it. This finding can be associated with the health belief model (Champion and Skinner, 2008) and supports the growth of the anti-vaccination movement. Various researchers have claimed that while parents are considering vaccination, they estimate the risks of both the disease and the side effects of the vaccination, and many prefer not to vaccinate (Bond and Nolan, 2011; Luthy et al., 2012). In addition, young parents—who probably have not experienced these diseases—base their decision on the fact that various diseases have been eradicated. Thus, they assume these diseases are not dangerous anymore, and that it is possible to heal them through natural processes, without external intervention (Bond and Nolan, 2011; Luthy et al., 2012). Further, they maintain that the vaccination's side effects may be more dangerous than the disease itself (Betsch and Sachse, 2012).
H4 suggested that hesitant parents’ use of online electronic resources will be greater than non-hesitant parents and was accepted. Findings reveal that hesitant parents surf more alternative and social network sites, and forums than non-hesitant parents. These findings are consistent with Wilson, Barakat, Vohra, Ritvo and Boon (2008), showing that non-hesitant parents tend to rely on authorised information from medical services and do not expand their search to other information sources. In contrast, hesitant parents used a variety of information sources, including the Internet. Hesitant parents also used more social network sites. Various studies suggested that Web 2.0 sites such as YouTube, Myspace, Facebook and others support the anti-vaccination movement and express anti-vaccination opinions (Keelan, Pavri, Balakrishnan, and Wilson, 2010; Seeman, Ing and Rizo, 2010). Thus, we may understand that hesitant parents are more exposed to anti-vaccination messages.
H5 suggested that parents who refuse vaccination will have higher perceived levels of e-health literacy and more limited knowledge about vaccination, and was confirmed. Findings reveal that refuser parents have the highest level of perceived e-health literacy, and that non-refuser parents have the largest knowledge about vaccination. These findings can be associated with Schulz and Nakamoto's (2011) theory that maintained that individuals think they are capable to search and evaluate medical information (because they have a high level of perceived e-health literacy). However, they actually lack crucial information that may help them make the right medical decision.
H6 proposed that parents who hesitated in the past to vaccinate and, after searching for information, decided to do so, have higher perceived e-health literacy and higher knowledge concerning vaccination than non-refuser and non-hesitant parents. This hypothesis is based on Betsch’s model, assuming that when parents face the problem of whether to vaccinate or not, they will look for information, evaluate it and then decide whether to vaccinate or not. However, this assumption was rejected. We can associate these findings with Luthy, Beckstrand, Asay, and Hewett (2013), who explored parents' experience concerning vaccination. The hesitant parents reported that they did not actually seek information, but rather relied on their feelings, explaining that they did not search for information because they were either overloaded with information, lacked information, or did not know where to locate reliable information about vaccination.
H7 suggested that refuser parents will use more information resources than other parents. This hypothesis was confirmed, echoing Kata's (2011) finding that refuser parents, who explored the possibility of whether to vaccinate or not, had a higher likelihood of using more information resources in comparison with non-refuser parents.
H8, that investigated whether parents who received information on Web 2.0 sites have lower knowledge about vaccination than parents who received information on other sites, was also confirmed. Findings reveal that the more knowledge parents have about vaccination the less they search in alternative sites, forums, social network sites, and the more they search in government sites. This can be associated with previous research (Betsch and Sachse, 2012; Kata, 2011; Venkatraman, Garg and Kumar, 2015; Witteman and Zikmund-Fisher, 2012) showing that most of the fallacious information concerning vaccination can be found in Web 2.0 sites. Thus, parents who use Web 2.0 sites have less knowledge about vaccination.
The researchers also examined the relationships between demographic variables and between hesitant and non-hesitant parents, and between refuser and non-refuser parents. The interesting finding was that hesitant and refuser parents tend to be younger. These findings contrast with previous studies (Aharon et al., 2012; Salmon et al., 2005) that showed refuser and hesitant parents to be older. We may assume that because the current study concentrated only on parents who search for information on the Internet, they are probably young people who grew up in the Internet era. Apparently, older parents look for information from other traditional, authorised sources such as books, doctors, nurses, etc.
Concerning the study's limitations, the current study examined only the relationship between parents' Internet searching and their decision on whether to vaccinate their children. It should be mentioned that the researchers did not come tabula rasa to the study and that their opinion is that vaccination is vital and essential. In addition, the study population was recruited via the snowball strategy using online channels. Perhaps using this method has limited the population in advance, as it did not include the population who hesitated over whether to vaccinate but used traditional channels of information.
The researchers recommend that a future study could also use qualitative methods such as open questions or interviews to supplement the quantitative analysis, and thereby enrich the findings by adding other dimensions to the process. Moreover, a further study may use an Internet site supporting vaccination, where the messages are conveyed via social network sites, videos and pictures, and examine its effect on individuals' decisions to vaccinate.
Conclusion
Theoretically, this study focuses on hesitant and non-hesitant parents concerning the vaccination dilemma, and on their perceived level of online literacy (e-health), level of electronic information sources use and their actual level of knowledge on the subject of vaccination. The research findings show that most of the parents who hesitated over whether to vaccinate their children decided not to vaccinate them. These parents believe they can search for information on the Internet and they surf different kinds of sites (government, social network sites, forums and alternatives). Practically, we suggest that in order to increase parents' willingness to vaccinate their children and their e-health literacy, medical institutions should change their strategy. They should adopt the strategy used by the anti-vaccination movement that conveys its messages via different online channels. Researchers assume that information presented in government sites, that usually include statistical data and verbal explanations, do not influence users (Betsch et al., 2010). However, the anti-vaccination movement uses videos, pictures and social network sites. This strategy was found to be more effective. Therefore, it would be beneficial if medical institutions use these channels to illustrate the dangers of non-vaccination in order to increase parents' e-health literacy. Moreover, we recommend that mainstream medical institutions direct parents to recommended health sites, where they can retrieve reliable information, and thus reduce the likelihood they will search sites offered by the anti-vaccination movement, or sites that present misleading, inaccurate information.
About the authors
Noa Aharony received her Ph.D. in 2003 from the School of Education at Bar-Ilan University, Israel. She is the head of the Information Science Department at Bar-Ilan University. Her research interests are in education for library and information science, information literacy, technological innovations and the librarianship and information science community, and Web 2.0. Prof. Aharony is a member of the editorial boards of Journal of Librarianship and Information Science, Online Information Review, and IJELL. Prof. Aharony has published in refereed information science and education journals. She can be contacted at Noa.Aharony@biu.ac.il ORCID No. orcid.org/0000-0003-1440-3305
Romina Goldman is a graduate student at the Department of Information Science Department at Bar-Ilan University.
References
- Aharon, A. (2011). Parents’ decisions not to immunize their child: past and present, characteristics of the phenomenon and its causes. Israeli Journal of Health Education and Promotion, 1(4), 32-40.
- Aharon, A. A., Nehama, H., Shmuel, R. & Baron-Epel, O. (2012). Distribution of reasons for not completing vaccines. [PowerPoint presentation]. Retrieved from http://video.new-app.com/customers/NIHP/parallel/1A2.pdf (Archived by WebCite® at http://www.webcitation.org/6pTmpaHgl)
- Austin, E. W., Pinkleton, B. E., Austin, B. W. & Van de Vord, R. (2012). The relationships of information efficacy and media literacy skills to knowledge and self-efficacy for health-related decision making. Journal of American College Health, 60(8), 548-554.
- Bazin, H. (2009). History of vaccine refusal. Bulletin de l'Academie nationale de medecine, 194(4-5), 705-718.
- Bean, S. J. (2011). Emerging and continuing trends in vaccine opposition website content. Vaccine, 29(10), 1874-1880.
- Betsch, C. (2011). Innovations in communication: the internet and the psychology of vaccination decisions. Euro Surveillance, 16(17), 4.
- Betsch, C., Renkewitz, F., Betsch, T. & Ulshöfer, C. (2010). The influence of vaccine-critical websites on perceiving vaccination risks. Journal of Health Psychology, 15(3), 446-455.
- Betsch, C. & Sachse, K. (2012). Dr. Jekyll or Mr. Hyde? (How) the internet influences vaccination decisions: recent evidence and tentative guidelines for online vaccine communication. Vaccine, 30(25), 3723-3726.
- Betsch, C. and Wicker, S. (2012). E-health use, vaccination knowledge and perception of own risk: drivers of vaccination uptake in medical students. Vaccine, 30(6), 1143-1148.
- Bond, L. & Nolan, T. (2011). Making sense of perceptions of risk of diseases and vaccinations: a qualitative study combining models of health beliefs, decision-making and risk perception. BMC Public Health, 11(1), 943.
- Brewer, N.T., Chapman, G.B., Gibbons, F.X., Gerrard, M., McCaul, K.D. & Weinstein, N. D. (2007). Meta-analysis of the relationship between risk perception and health behavior: the example of vaccination. Health Psychology, 26(2), 136-145.
- Brunson, E.K. (2013). How parents make decisions about their children's vaccinations. Vaccine, 31(46), 5466-5470.
- Burgess, D.C., Burgess, M.A. & Leask, J. (2006). The MMR vaccination and autism controversy in United Kingdom 1998–2005: inevitable community outrage or a failure of risk communication? Vaccine, 24(18), 3921-3928.
- Champion, V.L. & Skinner, C.S. (2008). The health belief model. In K. Glanz, B. K. Rimer & K. Viswanath (Eds.), Health behaviour and health education: theory, research, and practice (4th ed.) (pp. 45-65). San Francisco, CA: Jossey-Bass.
- Coe, A. B., Gatewood, S. B. & Moczygemba, L. R. (2012). The use of the health belief model to assess predictors of intent to receive the novel (2009) H1N1 influenza vaccine. Innovations in Pharmacy, 3(2), 1.
- Davies, P., Chapman, S. & Leask, J. (2002). Antivaccination activists on the world wide web. Archives of Disease in Childhood, 87(1), 22-25.
- Deer, B. (2011). How the case against the MMR vaccine was fixed. British Medicql Journal, 342(7788), 77-82.
- Dunn, A. G., Leask, J., Zhou, X., Mandl, K. D. & Coiera, E. (2015). Associations between exposure to and expression of negative opinions about human papillomavirus vaccines on social media: an observational study. Journal of Medical Internet Research, 17(6), e144.
- Dunsky-Baranga, L. (2007). A progressive model for explaining mothers’ objection to the vaccination of their infants. Unpublished doctoral dissertation, Bar-Ilan University, Ramat Gan, Israel.
- Fox, S. & Duggan, M. (2013). Health online 2013. Washington, DC: Pew Research Center. Retrieved from http://pewinternet.org/Reports/2013/Health-online.aspx (Archived by WebCite® at http://www.webcitation.org/6pMsZcRwt)
- Fredrickson, D. D., Davis, T. C., Arnould, C. L., Kennen, E. M., Humiston, S. G., Cross, J. T. & Bocchini, J. A. (2004). Childhood immunization refusal: provider and parent perceptions. Family Medicine, 36(6), 431-439. Retrieved from http://www.stfm.org/Portals/49/Documents/FMPDF/FamilyMedicineVol36Issue6Fredrickson431.pdf [Unable to archive]
- Guidry, J. P., Carlyle, K., Messner, M. & Jin, Y. (2015). On pins and needles: how vaccines are portrayed on Pinterest. Vaccine, 33(39), 5051-5056.
- Harmsen, I. A., Doorman, G. G., Mollema, L., Ruiter, R. A., Kok, G. & de Melker, H. E. (2013). Parental information-seeking behaviour in childhood vaccinations. BMC Public Health, 13(1), 1219. Retrieved from https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-13-1219 [Unable to archive]
- Jones, A. M., Omer, S. B., Bednarczyk, R. A., Halsey, N. A., Moulton, L. H. & Salmon, D. A. (2012). Parents’ source of vaccine information and impact on vaccine attitudes, beliefs, and nonmedical exemptions. Advances in Preventive Medicine, 2012, article ID 932741. Retrieved from https://www.hindawi.com/journals/apm/2012/932741/ [Unable to archive]
- Jordan, J. E., Osborne, R. H. & Buchbinder, R. (2011). Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. Journal of Clinical Epidemiology, 64(4), 366-379.
- Kasmaei, P., Shokravi, F., Hidarnia, A., Hajizadeh, E., Atrkar-Roushan, Z., Shirazi, K. & Montazeri, A. (2014). Brushing behavior among young adolescents: does perceived severity matter. BMC Public Health, 14(8), 2-6. Retrieved from https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-14-8 [Unable to archive]
- Kata, A. (2010). A postmodern Pandora's box: anti-vaccination misinformation on the Internet. Vaccine, 28(7), 1709-1716.
- Kata, A. (2011). Anti-vaccine activists, Web 2.0, and the postmodern paradigm–an overview of tactics and tropes used online by the anti-vaccination movement. Vaccine, 30(25), 3778-3789.
- Keelan, J., Pavri, V., Balakrishnan, R. & Wilson, K. (2010). An analysis of the Human Papilloma Virus vaccine debate on MySpace blogs. Vaccine, 28(6), 1535-1540.
- Leask, J., Chapman, S., Hawe, P. & Burgess, M. (2006). What maintains parental support for vaccination when challenged by anti-vaccination messages? A qualitative study. Vaccine, 24(49), 7238-7245.
- Luthy, K. E., Beckstrand, R. L., Callister, L. C. & Cahoon, S. (2012). Reasons parents exempt children from receiving immunizations. Journal of School Nursing, 28(2), 153-160
- Luthy, K. E., Beckstrand, R. L., Asay, W. & Hewett, C. (2013). Vaccinating parents experience vaccine anxiety too. Journal of the American Association of Nurse Practitioners, 25(12), 667-673.
- Mcree, A., Reiter, P. & Brewer, N. (2012). Parents’ Internet use for information about HPV vaccine. Vaccine, 30(25), 3757-3762.
- Middleton, D. B. & Wolfe, R. M. (2013). The vaccine misinformation landscape in family medicine. In Archana Chatterjee, (Ed.). Vaccinophobia and vaccine controversies of the 21st century (pp. 147-164). New York, NY: Springer.
- Mikulak, A. K. (2011). Parents’ vaccination concerns are about more than risk and benefit. Human Vaccines, 7(6), 597-599.
- Norman, C. D. & Skinner, H. A. (2006a). E-health literacy: essential skills for consumer health in a networked world. Journal of Medical Internet Research, 8(2), e9. Retrieved from https://www.jmir.org/2006/2/e9/ (Archived by WebCite® at http://www.webcitation.org/6qHLi91SV)
- Norman, C. D. & Skinner, H. A. (2006b). eHEALS: the eHealth literacy scale. Journal of Medical Internet Research, 8(4), e27. Retrieved from https://www.jmir.org/2006/4/e27/ (Archived by WebCite® at http://www.webcitation.org/6qHLUPFBA)
- Pew Research Center. (2015). 83% say measles vaccine is safe for healthy children. Washington, DC: Pew Research Center. Retrieved from http://www.people-press.org/2015/02/09/83-percent-say-measles-vaccine-is-safe-for-healthy-children/ (Archived by WebCite® at http://www.webcitation.org/6pMtiI1u3)
- Poland, G. A. & Jacobson, R. M. (2012). The clinician’s guide to the anti-vaccinationists’ galaxy. Human Immunology, 73(8), 859-866.
- Ruiz, J. B. & Barnett, G. A. (2015). Exploring the presentation of HPV information online: a semantic network analysis of websites. Vaccine, 33(29), 3354-3359.
- Salmon, D. A., Moulton, L. H., Omer, S. B., DeHart, M. P., Stokley, S. & Halsey, N. A. (2005). Factors associated with refusal of childhood vaccines among parents of school-aged children: a case-control study. Archives of Pediatrics & Adolescent Medicine, 159(5), 470-476.
- Schulz, P. J. & Nakamoto, K. (2011). "Bad" literacy, the internet, and the limits of patient empowerment. In Nancy Green, Sara Rubinelli and Donia Scott, (Eds.) AAAI Spring Symposium: AI and Health Communication. Palo Alto, CA: AAAI. Retrieved from http://www.aaai.org/ocs/index.php/SSS/SSS11/paper/download/2430/2853 [Unable to archive]
- Seeman, N., Ing, A. & Rizo, C. (2010). Assessing and responding in real time to online anti-vaccine sentiment during a flu pandemic. Health Q, 13(Spec. No.), 8-15. Retrieved from http://www.longwoods.com/content/21923 [Unable to archive]
- Smith, P. J., Chu, S. Y. & Barker, L. E. (2004). Children who have received no vaccines: who are they and where do they live? Pediatrics, 114(1), 187-195.
- Smith, P. J., Humiston, S. G., Marcuse, E. K., Zhao, Z., Dorell, C. G., Howes, C. & Hibbs, B. (2011). Parental delay or refusal of vaccine doses, childhood vaccination coverage at 24 months of age, and the Health Belief Model. Public Health Reports, 126(Suppl. 2), 135-146.
- Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z. & Brand, H. (2012). Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health, 12(1), 80. Retrieved from https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-12-80 [Unable to archive].
- Strecher, V. J. & Rosenstock, I. M. (1997). The health belief model. In A. Baum, S. Newman, J. Weinman, R. West & C. MacManus (Eds.), Cambridge handbook of psychology, health and medicine (pp. 113-117). Cambridge: Cambridge University Press.
- Tafuri, S., Gallone, M. S., Cappelli, M. G., Martinelli, D., Prato, R. & Germinario, C. (2014). Addressing the anti-vaccination movement and the role of HCWs. Vaccine, 32(38), 4860-4865.
- Venkatraman, A., Garg, N. & Kumar, N. (2015). Greater freedom of speech on web 2.0 correlates with dominance of views linking vaccines to autism. Vaccine, 33(12), 1422-1425.
- Wilson, K., Barakat, M., Vohra, S., Ritvo, P. & Boon, H. (2008). Parental views on pediatric vaccination: the impact of competing advocacy coalitions. Public Understanding of Science, 17(2), 231-243.
- Witteman, H. O. & Zikmund-Fisher, B. J. (2012). The defining characteristics of web 2.0 and their potential influence in the online vaccination debate. Vaccine, 30(25), 3734-3740.
- Wolfe, R. M. & Sharp, L. K. (2002). Anti-vaccinationists past and present. British Medical Journal, 325(7361), 430-432.
- Zimmerman, R. K., Wolfe, R. M., Fox, D. E., Fox, J. R., Nowalk, M. P., Troy, J. A. & Sharp, L. K. (2005). Vaccine criticism on the world wide web. Journal of Medical Internet Research, 7(2), e17. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1550643/ [Unable to archive]
- Zingg, A. & Siegrist, M. (2012). Measuring people's knowledge about vaccination: developing a one-dimensional scale. Vaccine, 30(25), 3771-3777