Trust or do not trust: evaluation strategies used by online health information consumers in South East Asia

Introduction

According to the Pew Internet Research Center, more than half of American adults have searched for health information online (Fox and Duggan, 2013). American adults go online to search for a range of health issues primarily dealing with diseases and treatments (Fox, 2014). Europeans also perform health searching online with six out of ten going online to locate health information. Results of a survey conducted by Health on the Net Foundation indicate that British citizens search for health information at least once a day (Boyer, 2013). In Italy, family obligations and illness are triggers for performing an online search according to Cioni et al. (2018). Older Flemish citizens usually visited the doctor after performing an online search whilst younger respondents became more worried after performing an online search (Van Riel et al., 2017). Online health information consumers from the Middle East and North African regions who are working adults with more years of experience searching tend to use medical-type search engines (Inthiran et al., 2018). For example, participants with postgraduate qualification and working professionals used search engines such as WebMD, The Centre for Disease Control and Prevention Website, the Abu Dhabi Health Authority Website and Al-Noor Hospital site. These studies indicate that online health information searching is popular and performed by most online users.

In addition, online search activities have evolved from merely searching for health information to reading and watching someone else’s health experience and locating other health consumers who share the same health concerns (Fox, 2014). Health information found on the Internet helped improve lifestyle choices and increased knowledge about health-related topics (European Commission, 2014). Whilst much information can be gleaned from studies conducted in the western world, similar information from other parts of the world particularly from South East Asia is limited.

Background

Online health information searching is slowly gaining ground amongst South East Asia Internet consumers. This section sets the scene for what is happening in South East Asia.

Results of a study conducted in South East Asia of developing economies indicate that parents are motivated to perform an online search for a doctor and non-doctor visit purposes (Inthiran and Soyiri, 2015). South East Asia health consumers often shared information found online with female siblings and with their partner (Inthiran, 2017a). Most South East Asia health consumers spoke to the doctor about health information found online (Inthiran, 2017b), but took a cautious approach when doing so. Whilst most South East Asia health consumers were satisfied with their search experience, others used post-search outcomes as a measure of search satisfaction (Inthiran, 2016). Singaporean youths actively searched for information online pertaining to diseases such as diabetes, cancer, information on sexually transmitted diseases, pregnancy, birth control and HIV/AIDS (Rao and Theng, 2012), while Thai undergraduate students use online sources to obtain information on general health, disease and treatment, and nutrition (Kitikannakorn and Sitthiworanan, 2009).

In an urban city in Malaysia, health searchers predominantly use Google, however specific health Websites such as MedlinePlus, Medline, The Mayo Clinic, The National Institutes of Health Website, Johns Hopkins University Website and WebMD are also utilised. In a rural community in East Malaysia, online information sources were also used to obtain health information (Mohd-Nor et al., 2013).

Results of the research studies mentioned above provide some information about the online health searching scene in South East Asia. However, there is a gap in our understanding about how South East Asia searchers evaluate online health information for trustworthiness. In the western world, research studies indicate that online health information consumers use a myriad of methods to perform trustworthiness evaluation. These methods include evaluating the source, locating trust signs, analysing content features and evaluating Website design and usability features (Dutta-Bergman, 2003; Rowley et al., 2015; Sillence et al., 2007; Zulman et al., 2011).

In South East Asia, it is not known if and how online consumers assess information for trustworthiness. Results from a survey conducted by the Aid and International Development Forum (2017) indicate Internet consumers in South East Asia have limited digital skills and understanding of what the Internet is. Thus, it is postulated that online consumers in South East Asia may not evaluate health information for trustworthiness. Results of another research study indicate South East Asia health consumers found it difficult to decide on which links to click on and health consumers ignored information with negative connotations (Inthiran, 2016). In addition, due to low health literacy levels it is hypothesized that online health information consumers in South East Asia may use inappropriate strategies to evaluate online health information for trustworthiness (Chen et al., 2013). Results of Chen et al. 2013 and Inthiran suggest South East Asia health consumers may not evaluate information for trustworthiness. It is further hypothesized that online consumers in South East Asia may have atypical strategies of trust evaluation. This is because the online health searching scene in South East Asia is at the early stages in comparison to the western world.

Given that more South East Asia citizens are becoming online health information searchers it is important to describe the strategies employed when evaluating online health information for trustworthiness. The ability to evaluate online health information for trustworthiness is important. The following examples are used to illustrate the need for this ability: Websites containing information for home management of children with fever did not adhere to primary guidelines (Impicciatore et al., 1997). Cline and Haynes (2002) in their seminal work summarised how online health information was prone to being inaccurate, misleading and dangerous. In another example, only twelve out of sixty Websites containing information on childhood diarrhoea adhered to treatment recommendations of the American Academy of Paediatrics (McClung et al., 1998). Other research studies continue to highlight an increasing number of dangerous and fictitious information about medical treatments on the Internet (Gottlieb, 2002; Kitchens et al., 2014; McKinley et al., 1999; Abbasi et al., 2012). Thus, without evaluating online information for trustworthiness health consumers may end up inaccurate information.

The aim of this research study is to describe trustworthiness evaluation strategies used by South East Asia health consumers. Specifically the following research questions are tackled: i) do South East Asia health consumers evaluate online health information for trustworthiness and ii) how is information evaluated for trustworthiness. The remainder of this research paper is organised as follows. First, literature review is presented. Then, the methodology is described followed by presentation of results. This research paper ends with a discussion of results and a conclusion.

Literature review

This section begins by providing an overview of the term trust. It is acknowledged that medical professionals and lay people may have different meaning to the word trust. In comparison to laypeople, medical professionals are trained and have been taught to evaluate for trustworthiness with the likes of systematic review, meta-analysis and medical-based critical appraisal techniques. Thus, a brief explanation is provided to note these differences. . After this brief explanation, the literature will focus on methods employed by laypeople in four categories: (i) quality of source, (ii) trust signs, (iii) content feature, and (iv) design and usability. These categories represent popular and current discourse of trust evaluation strategies employed by online health information consumers in the western world. This section then concludes with a summary of literature reviewed. As the aspect of trust evaluation strategies has yet to be researched in South East Asia, literature pertaining to trustworthiness assessments of online health information consumers from the western world is used to set the scene.

Health information searching by medical professionals

In a seminal paper, Hersh (2009) explains the intricacies and uniqueness of online health information searching when performed by a medical professional. Hersh states that there are four properties that make health information searching dissimilar to other types of information searching activities. These are: growth, obsolescence, fragmentation, and linkage of scientific information. These properties are briefly discussed to understand the unique properties of performing an online health search.

Health literature grows extremely rapidly. Each year over 300,000 citations are included in the health database. Due to such growth, health information becomes obsolete quickly. Fragmentation indicates that a single article provides only a small part of the overall picture and further details are fragmented across multiple articles. Hence a searcher may have to read more than one article to obtain an understanding of a new finding. The last property, linkage of scientific information, refers to the study of citations of scientific literature. It is noted that a layperson would not have the ability the perform trust evaluations using these properties because they do not have the requisite high level of medical knowledge and expertise.

Results of previous research studies indicate that it is difficult for laypeople to reach a consensus on what is construed as trust in health Websites (Vega et al., 2011). Health activities are highly personal and trust would have different definitions to laypeople compared to medical professionals (Vega et al., 2011). Vega et al. also note that the top three terms used by laypeople to describe trust when performing an online health search are quality, understanding and reliability. Other research studies use the term credibility as a substitute for trust (Rains and Karmikel, 2009), while Mou et al. (2016) indicate laypeople’s trust evaluation methods are dynamic and may evolve over time. The quality of one’s current health, or the risks of becoming ill or exacerbating a condition, maybe a criterion in influencing a layperson’s trustworthiness assessment, according to Rains and Karmikel (2009).

These studies indicate that there is a difference in the way medical professionals and laypeople define the term trust. More importantly, laypeople have varying definitions and these definitions are complex, personal, and likely to change. Furthermore, the discourse in relation to defining trust in the online health information searching realm amongst laypeople is ambiguous and could be interchangeable (Rains and Karmikel, 2009; Johnson et al., 2015). Hence, instead of providing specific definitions trust, participants in this study were provided with two sentences to set the context for the term trust. These sentences are: How do you know that you can believe information you have read?’ and How do you know that what you have read is true?. The sentences were read out to participants to function as a construct or definition. This approach was also used to take into account a holistic view of participants’ perception of trust.

Quality of source

The conceptual definition for quality of source is the name or owner of the Website. Research studies from the early 2000s indicate a personal doctor’s Website, medical university, insurance company, and federal government Websites are considered trustworthy sources or invoke trust (Dutta-Bergman, 2003; Eysenbach and Kohler, 2002). Walther et al. (2004) provide specific examples of how uniform resource locators are used as a trust indicator. Domains ending with .org are trustworthy whilst domains ending with .edu and .com are less trustworthy. Research studies from the mid 2000s indicate that information found on clinician and hospital Websites is perceived to be trustworthy in comparison to Websites returned by search engines (El-Hadad et al., 2014). Furthermore, online health information consumers do not restrict themselves to information on official Websites but also trust information found on the Websites of not-for-profit organisations, private healthcare providers, patient organisations, and in blogs written by physicians (Kwon et al., 2015). This indicates that in the 2000s, online health information consumers trusted information originating from non-governmental sources. However, some online consumers indicate trusting information from official government Websites rather than a private Website (Eysenbach and Kohler, 2002); given the age of this research, this may no longer be the case.

Notably, two research studies conducted in early 2000 and mid 2000 indicate participants trust information found on university research Websites (Alcock, 2016; Dutta-Bergman, 2003), yet results of a study conducted by Walther et al. (2004) indicated the reverse. This possibly indicates that online health consumers do not have a universal method of assessing trustworthiness. This shows that online health information consumers have different trustworthiness evaluation methods. There is a possibility that trust mechanisms from traditional mass media such as government health agencies have transferred to Internet sources (Ye, 2010). For example, if participants trust information provided by government agencies, they would be likely to trust information provided by those agencies’ Websites. This demonstrates the adaptation of offline evaluation strategies into the online realm. This behaviour corresponds with the concept of trust transfer theory where trust may be transferred from different kinds of sources or places and from known to unknown targets (Milliman and Fugate, 1998; Stewart, 2003; Uzzi, 1996). Furthermore, as online consumers’ health literacy levels increase over time, trustworthiness evaluation criteria would also change.

Trust signs

Any features implemented on a Web page are considered trust signs. Research studies from the early 2000s indicate the use of images increased trust in a Website. Specifically, having a picture of the author on the site increases trustworthiness; more so if the image shows the author with a pleasing face, smiling or a likeable appearance (Eysenbach and Kohler, 2002). However, the use of particular advertisements such as those for pornography on health Websites negatively affected trustworthiness (Eysenbach and Kohler, 2002). Other trust signs such as the appearance of third party seals, branding, and self-regulating policies (Luo and Najdawi, 2004) positively influenced trustworthiness evaluation. Other salient features such as the ability to make contact with the author in relation to information available on the site via an email address increases the level of trust. Details of information currency such as when the page was last updated also increases trust value for the page (Eysenbach and Kohler, 2002). However, Morahan-Martin (2004) indicate that trust signs did not encourage the formation of high levels of trust. This presents a conundrum since Morahan-Martin’s results negate results of previous research studies. Rains and Karmikel (2009) state information currency did not influence trust.

In addition, Walther et al. (2004) found the use and appearance of advertisements influenced the perception of trustworthiness on domains ending with .org but not with .com and .edu. More recently Sillence and Briggs (2015) have shown that the use of images did not increase the value of trust in a Website. Alcock (2016) indicates participants with low health literacy use image quality as an assessment of trustworthiness. For participants with low literacy levels, the availability of celebrity endorsements increased the value of trust in information provided on the Website (Alcock, 2016). Thus, there is a difference in opinion on the usage of images over time.

Overall, there are varying perceptions of how trust signs influence trustworthiness evaluation. It is possible that as online consumers become more familiar with online health searching they rely on trust sign available on the domain used to perform trustworthiness evaluation.

Content features

Cline and Haynes (2002) were among the earliest researchers to explore whether content provided on a Web page could be used as a trustworthiness indicator. Their results indicate author credibility and authoritativeness of the content are used as trustworthiness indicator. Research studies continue to indicate that author credibility is a pertinent trustworthiness evaluation criterion (Diviani et al., 2015; Fogg et al., 2003; Gray et al., 2005; Hong, 2006; Lankes, 2008; Stella, 2003).

Whilst research studies indicate information quality and credibility of the author or the Website are not used as a trustworthiness assessment criterion (Corritore et al., 2007; Morahan-Martin, 2004), later research indicates trustworthiness of the contributor, citations and references, objectivity, completeness, and peer evaluation comments are also used as trustworthiness indicators (Moturu et al. (2008). However, Rains and Karmikel (2009) indicate references and author credibility did not influence trust evaluation. Thus, it is assumed that trustworthiness evaluation continue to change over time.

Cunningham and Johnson (2016) and Eysenbach and Kohler (2002) state readability of information provided, length of the text, vocabulary used, and paragraphing structure are used as criteria when performing trustworthiness assessments. Articles that have small print, too much text, and irrelevant or inappropriate content negatively affected trustworthiness assessment (Sillence and Briggs, 2015). Mou et al. (2016) indicate that the manner in which information is written and presented is important because it influences trustworthiness judgements. For example, if information provided seems to be acting in the interest of the advertiser rather than the online health information consumer then a low value of trustworthiness is assigned to the Website’s content. An example of this is when an insurance or hospital Website provides information that is written in a way that favours the organisation rather than the audience. Thus, it is noted that particular criteria withstood the test of time.

Yi et al. (2013) indicate argument quality increased trustworthiness and older adults indicate using information quality as an assessment criterion for trustworthiness (Zulman, 2011) whilst young adults indicate reliability, information quality, and prevalence of factual information as the top three elements influencing trust judgements (Rowley et al., 2015). Thus, it appears different age groups have different evaluation criteria.

Hong (2006) and Divianni et al. (2015) emphasise that information provided on a health Websites needs to be complete, easily understood, clear, and comprehensive. In some cases, participants had to have found the information useful in order to be able to trust that information (Rowley et al., 2015; Song and Zahedi, 2007). This indicates that online health information consumers use post-search evaluation criteria as a trustworthiness evaluation criterion.

Design and usability

Fruhling and Lee (2006), Corritore et al. (2007) and Mou and Cohen (2017) indicate that having an easy to use Website is equated with being able to trust the Website. Among features that negatively affected trustworthiness assessments are Websites that are confusing, time-consuming to use, take a lot of effort, complicated, and require a lot of work to use (Lanseng & Andreassen, 2007).

These research studies indicate online health information consumers rely on usability features as trustworthiness evaluation criteria. In addition to ease of use, health consumers also indicate Website design and structure as an indication of whether they can trust information; this has been consistently found over time (Eysenbach & Kohler, 2002; Corritore et al., 2007; Cunningham and Johnson, 2016; Rains and Karmikel, 2009; Sillence and Briggs, 2015; Zulman, 2011).

The availability of navigation features also contributed to the assessment of trustworthiness (Rains & Karmikel, 2009). Web pages that had a clear menu or list were preferred and lack of navigation aids and search facilities contributed to a low trust indicator (Kim, 2016; Sillence and Briggs, 2015). Poor site organization and design also contributed to a low trust indicator (Kim, 2016; Sillence and Briggs, 2015). It is also important that Websites adhere to usability requirements such as adaptability, functionality, flexibility, and error-free interaction (Bart et al., 2005). Results of these research studies clearly show that aspects of design and usability are pertinent trustworthiness assessment criteria for online health information consumers. In addition, Divianni et al. (2015) indicate the availability of chat and forum features increased trustworthiness of the Website.

Summary of literature review

Table 1 provides a summary of research findings based on the literature review on the four main criteria: quality of source, trust signs, content feature, and design and usability. The design and usability criteria are most stable and research studies do not indicate conflicting information in relation to these criteria being used to evaluate trustworthiness. As for the remaining three criteria, there seems to be conflicting information. For example, for the quality of source, online health information consumers indicate a difference in opinion when using federal government Websites and private Websites as a trustworthiness assessment criterion. The use of third party seals, branding, policies, email and contact information for the trust signs criteria are used by some online health information consumers as assessment characteristics but not by others. Similarly, author credibility, content authoritativeness, citations, references and peer evaluation within the content feature criteria are used as trustworthiness assessment criteria is some instances and not in others.

While source, content feature, and trust signs are perceived to be typical methods of performing trustworthiness assessments, design and usability characteristics are also used as a trustworthiness assessment criterion. Design and usability characteristics presented in Table 1 are longstanding tenets of good interface design principles, and make up visual perceptual elements which facilitates the cognitive process of forming of an impression of the Website (Faraday, 2000). Thus, it is postulated that online health information consumers also use design and usability characteristics as a trustworthiness assessment criterion.

Method

A grounded theory approach using semi-structured interviews and questionnaires was used in this research study. A purposeful homogenous sampling technique was employed. The grounded theory approach was used to allow the adoption of a neutral view of human action in social context (Simmons, 2006). Semi structured interviews were used to allow for deep exploration of participants thoughts and experiences (DeJonckheree and Vaughn, 2019). A qualitative research method was to answer the research questions. As the grounded theory approach was used for the analysis (Strauss and Corbin, 1998), theories can be developed from research grounded in data. This technique is useful when generating understanding of areas of little past research (Charmaz, 2008; Strauss and Corbin, 1998), such as in this study where trustworthiness evaluation strategies used by South East Asia online health information consumers are not known. The grounded theory approach also provided the ability to systematically represent reality and the experience of people being studied (Strauss and Corbin, 1998). Data collection and analysis were done iteratively and data collection was deemed complete when theoretical saturation was reached. In the following sections, the data collection, recruitment of participants, and characteristics of the sample are described.

Recruitment of participants

Participants were recruited through a call for participation notices placed in universities and bulletin boards at community centres. Participants contacted the researcher and an appointment was set at a convenient location and time. Several inclusion and exclusion criteria were adhered to. Participants had to be a citizen of and be residing in a South East Asian country, be 21 years or older, and have performed an online health search. Participants must not be working in or have formal qualification or be in the process towards achieving an academic qualification from the healthcare or allied healthcare sector.

There were eighty participants in this study. The participants’ average age was 38 years (SD=4.8). There were forty-five female and thirty-five male participants. Participants’ qualifications ranged from a diploma to a PhD. Participants were professionals working in fields such as business, engineering and information technology. Occupations included administrators, consultants, business owners, homemakers, and employees working in the service and construction industries. On average participants had a general search experience of 16.9 years (SD=4.3) and an average health search experience of 5 years (SD=1.5). The frequency of performing a health search ranged from as often as once a week to as and when necessary. The time duration of a search session ranged from 5 minutes to 45 minutes. Mostly mobile devices were used to perform a health search. The search was usually performed at home. Participants were Malaysian (twenty-five participants), Indonesian (twelve participants), Thai (fifteen participants), Cambodian (eight participants), Filipinos (ten participants) and Myanmarese (ten participants).

Data collection

A questionnaire and semi-structured interviews were used to collect data for this research study. The questionnaire collected information on socio-demographic details and health search experience. Information from the socio-demographic questionnaire was used to provide contextual information of the participants that took part in the study. Among questions asked in the questionnaire are sex, age, years of health search experiences, education and occupation.

The semi-structured interviews were used to collect information for a larger study (Inthiran, 2015; Inthiran and Soyiri, 2015) but only results for the following questions were presented for analysis for this study:

1) Do you assess information for trustworthiness?

The term trust may take on a different meaning to different people, so in an attempt to provide a generic definition, the following sentences were read to participants to provide context to the term trust:

‘How do you know that you can believe information you have read?’ and
‘How do you know that what you have read is true?’

Depending on answers one of the following two questions were asked:

2a) Why was information not assessed for trustworthiness? or,
2b) How did you assess information for trustworthiness?

Participants were allowed to describe as many techniques to assess information for trustworthiness as they used. All methods of assessment used by participants were accounted for in the analysis stage.

The interview questions were semi-structured and were mostly designed to be open ended. The English language was the medium used to conduct the interview. At minimum all participants took the English language as a subject up to high school level. As such, there was no language barrier for participants in expressing themselves. The author conducted all the interviews. The interviews were audio-recorded and transcribed verbatim. A pilot test of the interview questions was conducted first with five participants and, interview questions were then fine-tuned. The main experiment took place after the pilot test. Participants were informed of the purpose of the research study via an explanatory statement. Consent was obtained from participants via signing a consent form. On average the interview questions pertaining to trustworthiness took 10 minutes.

Data analysis

Results from the questionnaire were analysed using descriptive statistics. The audio recordings of the interviews were transcribed verbatim. This was to allow close links to be created between the data and the researcher (Strauss and Corbin, 1998).

The open coding technique was used and coding categories was derived inductively from the audio recordings according to the grounded theory approach (Halcomb and Davidson, 2006). A master list of codes was first created based on induction. These codes were revisited after every third participant. Codes were then reduced to themes using the constant comparative method. The constant comparative method was used as it goes through four stages of constant comparison where theory is reinforced (Glaser and Strauss, 1967). This technique is beneficial because it begins with raw data and uses constant comparison from which substantive theory will emerge (Glaser and Strauss, 1967).

The author conducted the transcription and derivation of codes. To increase internal validity, an independent researcher crosschecked the transcription, codes and themes. This complied with dependability, reliability and confirmability dimensions of increasing trustworthiness in data and results (Brown et al., 2002; Morrow, 2005). In addition to validity, reflexivity was also incorporated during the data analysis process (Conrad et al., 1993) to reduce researcher bias (Kolb, 2012) In order to comply with reflexivity, the author incorporated continuous awareness of reflecting, examining and exploring their own relationship with the data through all stages of the research process (Conrad et al., 1993). The author adhered to principles suggested by Bickman and Rog (2008) where data analysis began immediately after each interview and interpretive analysis continued until the completion of the study. The author followed steps for qualitative analysis of content as described by Zhang and Wildemuth (2005). It is noted that theoretical saturation was achieved with forty participants. This means there was no more difference in answers after forty participants. However, to increase internal validity, the number of participants was doubled to eighty. Increasing participants after the point of saturation, complies with increasing internal validity of the grounded theory method (Morrow, 2005).

Results

Results are presented in two main categories: i) evaluation strategies used by participants who evaluated for trustworthiness all the time and occasionally and ii) reasons for not evaluating for trustworthiness and occasionally evaluating results for trustworthiness. Trustworthiness evaluations methods are presented based on theme names. Theme names are presented in the order of the most popular assessment strategy used. Table 2 provides information on the number of participants who perform trustworthiness assessment.

Forty (50%) of the eighty participants reported evaluating sources for trustworthiness, and a further eighteen (22.5%) reported doing so occasionally; twenty-two (27.5%) reported that they did no evaluation.

Strategies used to assess information for trustworthiness

This section presents assessment strategies utilized by participants who assessed information for trustworthiness all the time and occasionally.

Source

The most common assessment strategy was to scrutinize the source. Eight-two per cent of participants who assessed information for trustworthiness used this assessment strategy. Participants automatically trusted information originating from specific sources, that they considered authoritative: The Mayo Clinic, the National Institutes of Health Website, the National Health Services Website, government health Websites, hospital Websites and the WebMD Website. Participants did not necessarily commence the search on these sites but if results were from the above-mentioned sites then information presented was automatically trusted.

Participants were asked why they trusted information from these sources. They indicated information from these sources is reputable because it originates from authoritative agencies.

Participant E: I use the National Health Services Website. This is a reputable site, even doctors and nurses use this site.
Participant E1: It is a government Website so I trust information provided here.
Participant M: I use a hospital Website... so I do not need to worry about reading wrong and inaccurate information.
Participant L2: I use professional Websites like a clinic or hospital Website. I know information presented here can be trusted.

Eight three percent of participants who occasionally assessed information for trustworthiness did so by checking the source. In all cases, participants indicate trusting information from authoritative Websites.

Counter-checking with individuals

The second most popular method of assessment is by means of counter-checking information with individuals. A total of seventy five per cent of participant who assessed information for trustworthiness used this technique. These individuals are family members, physicians, friends who are in the medical field, and with people who had experienced a similar health situation (in this order of popularity).

In the family members category, the participants’ partner, mother, mother-in-law, aunts or siblings were consulted to verify information. Participants indicated when verifying information with their siblings the assumption was that the sibling consulted is perceived to be more experienced in performing a health search. Participants who verified information with their mother, mother-in-law or aunts wanted to compare information found online against the family member’s personal experience and knowledge. Participants who verified information with their partner wanted to obtain insights from their partner. When contradictory information was found (i.e., where information found online did not coincide with the opinion and advice of the person consulted) participants would trust information given by individuals rather than information found on the Website.

It is noted that male participants did not employ the family member verification method. Male participants indicated verifying information with their physician and also with their friends who were in the medical field. This method of performing trustworthiness assessment was a considered to be a reliable method because the individuals participants consulted were deemed to be more experienced and have medical expertise.

Participants also consulted with people who had undergone or were undergoing the same or similar health situations. For example, participants indicate they would speak to cancer survivors to verify information found online. If information from both sources (cancer survivor and online information) coincided then information was deemed trustworthy. In the situation where information did not coincide then participants would deem information found online to be less trustworthy.

Participant R: I would check information with a friend. For example, if I am looking for information about cancer cure, I would consult a friend who is a cancer survivor. I would ask if the information I read was correct... and if the remedy worked.
Participant W: I will compare information with friends who had similar health experience. At least I am checking information with someone who went through the same health situation.
Participant E2: I would call my mum and aunties to confirm information I read. I would ask them if what I read was correct based on their experience.
Participant L: I have some good friends who are doctors. So if the need be I will just call them and tell them what I read and ask if they are of the same opinion. So far, my friends agree with the information I found.
Participant R3: I will ask my husband to check the information to see if it is trustworthy. I am not sure what strategies he uses to check information but I get him to do the checking.
Participant M: I check information with my sister, she is a researcher. Sometimes she teaches me some methods to check information for trustworthiness.
Participant M1: I won’t say that everything I read is 100% trustable so I will check with the doctor.

Content cross-check

The third most popular method of assessment is content cross-checking and thirty seven per cent of participants who assess information for trustworthiness used this technique. Participants searched for as much identical information as possible. They indicated that the more they came across identical information on different Websites, the more they trusted the information provided. Participants were then asked what would happen if they found information that was contradictory. They indicated that this had happened to them. However, based on their experience, the possibility of locating contradictory information was minimal thus they would still trust information that was identical in content. Participants also indicated that since the majority of information was identical in content, they ignored information that was contradictory. Participants who used this content cross checking method of assessment did not use any other method of assessment.

Participant S: I compare information on other Websites. I will look through as many sites as possible and find as much information that say the same thing.
Participant S1: I click on a few links and compare information. If what these pages say are the same then I will take it as I can trust the information

Comment scavenging

Comment scavenging was used by twenty five per cent of participants who assess information for trustworthiness. Participants who use blogs and forums to search for health information said they read the comments contained within the blog and forum as an indicator of trust. Similar to the content cross-checking method, the more comments they find saying identical things or agreeing with the author, the higher the value of trust is placed on the blog content.

Participant R: I will read the comments provided by other users to see if they are saying the same thing.

Author credibility and authority

Seventeen per cent of participants who occasionally assessed information for trustworthiness did so by scrutinizing the author of the article. For example, if the author were a doctor, someone working in the health sciences field or is an academic then participant would trust information read. If author information could not be found, then participants would look at other Websites where they would be a clear indication of who the author is.

Participant M3: I usually check for things like, who wrote this page. Who is writing this information?

Reasons for not assessing for trustworthiness and occasionally assessing for trustworthiness

In this section, participants' rationale for not assessing information for trustworthiness and occasionally assessing for trustworthiness is presented.

Indifference

Of the twenty two participants who did not evaluate information for trustworthiness, seventy two per cent were not concerned about the trustworthiness of information they read online. The reason for this was that information searched for was not critical, thus there was no need to evaluate it for trustworthiness.

Participant H: I have never bothered to think about trusting information. Most of the time information I look for is not critical it is just to get some advice or information.
Participant S: ... laughs... trust information? This is necessary? Why?
Participant A: I don’t bother about this.

Cumbersome and complicated

Twenty seven percent of participants who did not assess information for trustworthiness indicate they found this process too difficult or thought that it would complicate the search process.

Participant S1: I never concerned myself about this part... it will make the search more troublesome.
Participant H2: If I have to check it, it will take a too many steps and too difficult.

Health condition

In all cases, it was the participant’s health condition that led to the need to perform trustworthiness assessments occasionally. When their health condition was improving, information was not evaluated for trustworthiness because the health condition did not warrant it.

Participant A1: when I am not getting better or if the medicine is not working then I would recheck information.

Discussion

Whilst the majority of participants interviewed did evaluate information for trustworthiness, there were participants who ignored the need to do so. Participants who did not evaluate information for trustworthiness had three main reasons: i) indifference, ii) it was complicated and cumbersome, and iii) their health condition. It is postulated that participants who were indifferent perhaps simply did not understand the repercussions of not assessing information for trustworthiness. The inability to assess trustworthiness of health information online could lead to health consumers making the wrong health decisions. This could have catastrophic consequences. Participants who thought it was cumbersome and complicated found the process tedious. There is also a possibility that health consumers generally found the process of searching for health information difficult and did not want to add on trustworthiness vetting procedures to this process (Can and Baykal, 2007; Yang, et al., 2011). On the other hand, participants who occasionally evaluated information for trustworthiness used their health condition as a reason to do so. In this case, evaluating for trustworthiness is a reactive action.

Participants who evaluated information for trustworthiness used several strategies for this purpose. The most popular method of assessing online health information for trustworthiness was by checking the source of information. Participants indicated using reputable and trustworthy sources such as government health Websites. It is noted that participants are aware of and are able to recognise authoritative sources of information. Results of this study support results of earlier studies (Alcock, 2016; Dutta-Bergman, 2003; Eysenbach and Kohler, 2002; Kwon, et al., 2015) where information from authoritative Websites is trusted. However, unlike previous research studies, where participants trusted information from the federal government less (Alcock, 2016), the results of this research study do not indicate that participants placed a lower value of trust in federal government Websites.

Whilst participants were able to name and utilise many international health Websites, they did not use local health Websites in South East Asia, although there are many local health Websites (Balea, 2016; Health Information Today; Wee, 2013). One reason for this could be the lack of awareness of local health portals. Local portals could have localised health information, which could be beneficial to the community. It is noted what whilst participants were able to name authoritative Websites, they did not name any Websites that they did not trust. This is a research gap that could be filled in future research.

The second-most popular method of assessing information is by counter checking with individuals, mainly with family members, physicians, partner, friends who are in the medical field, and people with a similar health situation. This method of evaluating trustworthiness is a strategy unique to South East Asia health consumers, as previous research studies did not indicate participants from the western world using this method of trustworthiness assessment.

There are several possible explanations for this. Cultural norms, traditional roots and religious connotations are strong in South East Asia. Thus there is reliance on personal experience, home remedies, and traditional medication rather than on information or modern medication and treatment (Bennet et al., 2015; Cotton et al., 2008). In South East Asia, 80% of the population still use traditional remedies for primary care (Farran, 2010).

Another cultural connotation is that the primary role of womenfolk in South East Asia is to take care of the family whereas men are seen as breadwinners (Carteret, 2010). As such, maternal elders are viewed as knowledge sources of traditional medication, traditional treatment and life experiences based on knowledge passed down for generations. Thus, female participants rely on other females closest to them (mother, mother-in-law, sibling, aunt) when assessing information for trustworthiness. Other factors include strong cultural hierarchical bonds such as respect towards people of higher authority and elders (mother, mother-in-law, sibling, partner) and a desire to maintain harmony with them (Claramita et al., 2011; Inthiran and Soyiri, 2015). Thus, participants take it upon themselves to counter check information with the elders and place higher trust value on information given by these individuals. It is possible to see this as an explicit mechanism used by the younger generation to gain knowledge from the older generation.

It is also noted that participants did not consult with male elders (father, father-in-law, brother) possibly because male elders would not be able to advise them on these issues as child-bearing and rearing activities are the responsibility of the womenfolk (Tobin, 2014). However, this aspect requires further investigation. Results of research studies from the western world offer some explanation of why women rely on women. Women are viewed as health guardians of society (Marton, 2011) and often provide health information to others (Metoyer-Duran, 1993). Women family members received high ratings for information source accessibility (Marton, 2011). Women also prefer interpersonal sources (Chatman, 1992; Johnson and Meischke, 1994), which further strengthens the concept of women relying on women.

Aside from family members, South East Asia online health information consumers counter-check information with the physician and friends in the medical field. Thus, whilst participants consult with family members for personal knowledge and experience, physicians and friends in the medical field are consulted to obtain knowledge on modern medicine. South East Asia health consumers also rely on information provided by people who have gone through a similar health situation, indicating the importance placed on personal experience and knowledge.

Whilst some health consumers in developing countries seem to ignore information from accredited sources, they value information from individuals with a particular disease or condition. These individuals are validated as authoritative sources (Wilson, 1999). It is noted that South East Asia online health information consumers also rely on information provided by people who have gone through a similar health situation. As such, it is postulated that patient-led health Websites such as PatientsLikeMe will be a popular source of obtaining health information. Results indicate South East Asia online health information consumers rely on people to perform trustworthiness evaluation. Thus, an offline mechanism is used to perform trustworthiness evaluation for information found in the online realm.

The content cross-checking and comment scavenging technique are also used by health searchers in the western world however, these is a variation in the way these are implemented.

These methods of assessment seem ineffective. One reason for this could be attributed to low health literacy levels in South East Asia (Chen et al., 2013) and poor understanding of information quality on the Internet (Pigato, 2001). Looking for as much identical information as possible may cover a mass of Websites but there is a tendency to experience confirmation bias (Nickerson, 1998). This situation is compounded by the fact that the searcher and the search engine algorithm are subjected to bias when performing a health search (White, 2013). This means participants seem to seek out positive information rather than negative information . In addition, search engines also strongly favour a particular, usually positive, perspective at the expensive of the truth (White, 2013). Results indicate that even when contradictory information is found, participants would still continue to search for pages that provide non-contradictory information. This indicates a very strong confirmation bias behaviour amongst South East Asia health consumers.

Results of a research study conducted in a western world indicate young parents often perform triangulation of information. Three major methods of information triangulation used are escalating authoritativeness, second opinion, medical/non-medical and inclusive (Greyson, 2015). However, these triangulation methods are performed using different sources (authoritative person, Website, non-authoritative person), whereas in the case of South East Asia health information consumers, the same type of sources (multiple Websites) are used to content cross check. Results of a research study conducted in a western world indicate contributions to comments on a health discussion group commonly fit the behaviour of re-iteration, echoing and reinforcing (Godbold, 2012). Thus, South East Asia health consumers are bound to find similar information whilst content scavenging. In addition, there is a norm in South East Asia where patients avoid bad news (Silbermann, 2016) thus health information consumers in South East Asia may be purposely avoid negative information during the content cross check and comment scavenging techniques.

The least popular method of assessing trustworthiness is by checking author credibility and authority. Previous research studies indicate this method of assessment is used by some western health information consumers (Clines and Haynes, 2002; Diviani, et al., 2015; Gray et al., 2005; Hong, 2006; Stella, 2003) and not by others (Morahan-Martin, 2004; Rains and Karmikel, 2009). Participants who demonstrate this technique seek to find out whether the author of the information is credible by looking up the author credentials. Participants use valid assessment methods by determining if the author is trained health professional or an expert in a particular area.

Participants in western world use source as a criterion and so do participants from South East Asia. However, there are some differences in the manner in which the characteristics of source is used. For example, participants in South East Asia do not indicate using religious Websites, patient organisation Websites or using the uniform resource locator as an indicator of trustworthiness. It is possible to suggest that there are limited religious Websites in South East Asia and that participants were not aware of patient organisation Websites. There is also some similarity in trustworthiness evaluation with the content feature criterion. Participants in the western world indicate using author credibility and authority as trustworthiness evaluation criteria and participants from South East Asia also use author information as a trustworthiness evaluation criterion. However, participants in South East Asia do not use other aspects of content feature such as citations and references, and readability of information as trustworthiness evaluation criteria. It is postulated that participants in South East Asia do not pay attention to citations and referencing due to low health literacy levels (Chen et al., 2013).

Participants in western world use trust signs as a method of trustworthiness evaluation, however participants in South East Asia do not use this criterion when evaluation online health information for trustworthiness. Nor do participants in South East Asia use design and usability aspects of a page. On the other hand, participants in South East Asia utilise techniques such as counter checking with individuals, seeking out as much identical information as possible and scavenging through comments as trustworthiness evaluation criteria. These methods of trustworthiness evaluation are utilised, with some degree of variation, by health information consumers in the western world. Results of this study indicate that there are more differences than similarities in trustworthiness evaluation between participants in South East Asia and participants from the western world. Based on the results of the experiment, an initial theoretical explanatory model is proposed as the basis for future work and presented in Figure 2.

Limitations

This work is not without limitation. Amongst these limitations are that results of this research study cannot be generalised. The study population did not include representative participants from all South East Asia countries thus in future studies it is expected that all South East Asia countries will be represented. Additionally, participants were relatively well educated and this could have influenced results. Obtaining participants from various socio-economic groups would enable robust results. Furthermore, results were not discussed in terms of participants’ nationality. It is also acknowledged that self-bias limitations exist as results were reported based on what participants say they use which may not actual methods being used.

Conclusion

This preliminary research study provides information on trustworthiness evaluations utilised by South East Asia online health information consumers. As aspects of clinical care in the South East Asia region undergo positive transformation with the likes of shared decision making (Ng et al., 2013), taking into account patient values and patient centred healthcare practices (Lee, et al., 2013), it is pertinent to develop campaigns educating online health information consumers on performing correct methods of trustworthiness evaluations. Results of a recent research study indicates most South East Asia consumers trusted medical professionals (Saransomrurtai and Reinhardt, 2020). Thus, from a health practitioner perspective, medical professionals should be in a position to educate patients on appropriate ways to do online searching and assessment. Results of a research study indicate that South East Asia online health information consumers discuss with the doctor medical information found online (Inthiran, 2017b). Thus, doctors could now also convert the consultation session into a knowledge sharing session, educating patients how to critically appraise health information found online. From a policy maker’s perspective, there needs to be information awareness campaigns on the need for trustworthiness assessment and appropriate techniques for performing trustworthiness assessment. This could be spearheaded by each country’s Ministry of Health as collaborative effort within the ASEAN region.

In future work, the aspect of cultural norms and cultural hierarchical influence on online health searching will be explored. The second phase of this experiment will include a think aloud and observational study on interactive trustworthiness evaluation methods employed by South East Asia online health to further expand on the theoretical explanatory model.

Acknowledgments

The author would like to thank all participants for their candour, time and willingness to participate.

About the author

Anushia Inthiran is a Senior Lecturer at the University of Canterbury, New Zealand. She conducts research studies in human computer interaction and online health information searching. She can be contacted at anushia.inthiran@canterbury.ac.nz

How to cite this paper