The perceived value of the academic library: a systematic review

Introduction

Higher education institutions are struggling with the management of scarce resources and with justifying the need to maintaining or increasing the investment made in them. As the academic library serves the needs and priorities of its host institution (Rodger, 2009), libraries increasingly need to be able to demonstrate their contributions to institutional goals (Cox, 2018). However, institutions do not necessarily recognize the broader value of libraries (Delaney and Bates, 2015).

Hinderman (2005) recommends the use of outcome-based evaluation, which compares outcomes to the library's goals and mission. Although highly disputed, data on the impact and value of the library is no longer just nice to have; it is an essential management resource (Showers, 2015), and renders libraries' value visible. Library assessments 'has, over time, progressed from measuring inputs to gauging efficiency, then quality, to outcomes assessment' (Wallace, 2001, p.68). Furthermore, studies of impact are increasingly intended for higher-level administrators and tend to explore the impact of the library on institutional-level goals (e.g., student-centred outcomes) (Connaway, et al., 2017).

Markless and Streatfield (2017) call for theory-based evaluation as this approach uses a causal model between programme activities and desired ends to guide the evaluation. Kaufman and Watstein (2008) argue that libraries need to define outcomes relevant to their institution and measure their values. Town (2011) argues that there is a need for a broader assessment of the meaning of value and recognition that value is dependent on values sets or systems. Consequently, the first step in an evaluation of impact or value is to determine the desired ends. As pointed out by Hernon (2002) and Hernon and Nitecki (2001), many evaluation components exist, e.g., performance and outcomes measures, service quality, customer satisfaction and effectiveness. The relationship between these concepts is complex (Sputore and Fitzgibbons, 2017). ISO 16439 (2014, p.13) defines value as 'the importance that stakeholders (funding institutions, politicians, the public, users, staff) attach to libraries and which is related to the perception of actual or potential benefit'. Consequently, values can only be explored if stakeholders are asked.

Stenstrom et al. (2019) argue that value as a concept applied to libraries is highly complex and addresses both financial and social concerns. The former can be measured, whereas the latter is a product of the perceptions of stakeholders. Tenopir 2012) argues that value of libraries can be measured using implicit, explicit and derived values. Institutional missions vary and of course, academic libraries vary as well (Fraser, et al., 2002). How stakeholders perceive the value of the academic library is important to understand for libraries to be able to communicate their value to their funding institution. It may be that the perception of the value of the library differs among various stakeholder groups.

Kostagiolas and Asonitis (2009) stress that the traditional tools for value measurement will only provide a partial answer to the demand for proof of worth. They argue that the existing frameworks for assessing values fail to include intangible assets or broader definitions of value. Library value is also increased by staff capabilities and capacity, services, and the relationships with both its immediate stakeholders and broader society. Town (2011) argues that the definition of value is contextual and that valuation reflects values, and these are chosen. Because values are tied to people, it is necessary to gain insight into the shared beliefs inherent in values sets. Nitecki, et al. (2015) suggest the following important values: an informed citizenry, freedom of information, open and equitable access to knowledge and intellectual diversity. Mierzecka (2019) suggests the following areas of importance for researchers: general approach of scholars and librarians to cooperation, practices of research support, access to information resources adapted to scholars' needs, data curation support and publication strategies support. Furthermore, Bruff (2019) suggest articulating the value of student work and the library as laboratory, forum and archive. However, these values are not empirically identified in stakeholder groups. Urquhart (2015, p. 86) argues that '[o]nce library and information services have started to think about the various values that users attach to our services, we need to think thoroughly about perceived values.'

The aim of this review is thus to answer the following question: how do the stakeholders find value in the academic library?

Related literature

A number of initiatives exist to support academic libraries in demonstrating their value and impact. An example is the Value of Academic Libraries Initiative by the Association of College & Research Libraries (Oakleaf, 2010, p. 8) that aims to 'help academic librarians participate in the conversation and to identify resources to support them in demonstrating the value of academic libraries in clear, measurable ways'. Oakleaf suggests linking academic library outcomes to institutional outcomes within the following areas: student enrolment, retention and graduation rates, success, achievement, learning, engagement, as well as faculty research productivity, teaching, service, and the overarching institutional quality.

Another example is Lindauer (1998) who suggested key areas for the identification of library performance measures: (1) learning outcomes and enabling instructional outputs; (2) faculty/academic staff teaching effectiveness, scholarly productivity, and professional development; (3) institutional viability and vitality; (4) access, availability, and use of teaching-learning recourses; and (5) infrastructure-human resources, collections, and equipment/facilities. A final example is the perceived value framework of Holbrook's typology including three dimensions of perceived value: (1) extrinsic versus intrinsic value; (2) self-oriented versus other-oriented value; and (3) active versus reactive value (Holbrook, 1994; 1999). The dimensions are used to create a typology of perceived value containing the following values: efficiency, excellence, status, esteem, play, aesthetics, ethics and spirituality.

Schwieder and Hinchliffe (2018) suggest categorizing the studies of academic library value according to the level of analysis: small-scale studies that examine value perceived by a small group of students, single-institution studies that examine values perceived by larger student groups at a single university and multi-institution studies that analyse values perceived by students across several academic institutions. Urquhart (2015) offers an interpretation that enables an application of the typology for evaluation of library and information services. For a recent overview of other approaches to value-based evaluation models, the reader is referred to Urquhart (2018). Finally, Saracevic and Kantor (1997) suggest the following taxonomy of value in using library and information services:

• Reasons for using a library or information service (e.g., for a task, project or personal reasons)
• Interaction with a library service (e.g., availability or accessibility of resources, use)
• Results of using a library service (cognitive, affective, or expectations met).

De Jaeger (2017) argues that the advantages of combining data are noted only briefly in the ISO standard. Attempts include linking evidence of library use with student success data, indicating correlation but not causation (for recent examples see Murray, et al., 2016; Scott, 2017) and assessments of the economic value of libraries including return on investment methods (e.g., Hájek and Stejskal, 2015).

Studies of specific services or programmes are available in vast amounts (recent examples include Perrier, et al., 2018; Gross, et al., 2018). Performance and outcome assessments are plentiful, typically focusing on student retention or success (e.g., Murray, et al., 2016). Several reviews provide an overview of studies on performance and outcome measures (Powell, 1992; Wallace, 2001; Fraser, et al., 2002; Hart and Amos, 2014; Urquhart and Turner, 2016). Likewise, studies of performance and outcome measures fora specific service or programme are also reviewed (Schulte, 2012; Menchaca, 2014; Erlinger, 2018). Furthermore, Blummer and Kenton (2018) is an example of a review of studies addressing specific outcomes. Finally, Urquhart (2015) and Sputore and Fitzgibbons (2017) review the principles, practice and methods for impact assessment for academic libraries.

Connaway, et al. (2017) argue that there are few studies of how stakeholders perceive the value or impact of the academic library and how they prioritize these values or impacts. Unlike research libraries, how stakeholders perceive the values of public libraries has been investigated and reviewed. A recent narrative review of the values of public libraries by Stenstrom, Cole and Hanson (2019) categorizes them into these three types of support: personal advancement, vulnerable populations and community development. Furthermore, Aabø (2009) is the first meta-analytical review of library valuation studies and thirty-two of the thirty-eight explore value of public libraries. Two of the included studies concern academic libraries; however, they only focus on specific library services (Harless and Allen, 1999; Luther, 2008). However, there are no reviews summing up the studies of the stakeholder perceptions of the value of academic libraries.

Methods

Urquhart (2010) explores systematic reviewing, meta-analysis and meta-synthesis for evidence-based library and information science. Urquhart also outlines methods for meta-synthesis in various library and information science research fields. This study aims to identify how the stakeholders find value in the academic library and thus a meta-analysis is not likely to be relevant in this case.

This systematic review was conducted and reported following the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) statement (Moher, et al., 2009) and the AMSTAR measurement tool (Shea et al., 2007).

Search strategy

We searched for eligible studies using Scopus, Library and Information Science Abstracts, and Proquest Dissertations on August 14, 2019. The search strategy was developed and validated using the capture-recapture technique as well as on the basis of retrieval of known items (Booth, 2010). The search strategies are available in Appendix 2. Citation searching using included studies as a starting point was conducted to find further potentially eligible studies (Scopus up to August 27, 2019.

Eligibility criteria

All primary studies were considered eligible for inclusion if they examined the viewpoints of stakeholders in regard to the value of academic libraries, whether it was the main focus of the study or one of several aims. All research designs were considered eligible. Review articles were excluded but reference lists were used to identify further potentially relevant studies. The search was restricted to peer reviewed publications in English.

For this review and its included studies, an academic library is defined as the library associated with a degree-granting institution of higher education. Consequently, studies on specific occupational groups (e.g., cataloguers), specific programmes (e.g., embedded programmes) or specific services (e.g., interlibrary loans) fall outside the scope of this review.

The primary outcome was to understand the facets of values of the academic library as perceived by the stakeholders. For the search strategy a broad definition of value was used to retrieve as many potentially relevant studies as possible (Appendix 2). This review focuses on values and consequently, studies on satisfaction with services, outcome assessments, effectiveness or user satisfaction levels are excluded.

Study selection

Following the AMSTAR recommendation (Shea et al., 2007), two reviewers independently agreed on a selection of eligible studies and achieved consensus on which studies to include. Following recommendations from the Cochrane Handbook (Higgens et al., 2011) study selection took place in two stages. First, titles and abstracts were examined to remove obviously irrelevant items. Second, full-text reports were examined to identify studies that met eligibility criteria. Furthermore, the Cochrane Handbook states that at the first stage numbers of records are sufficient for exclusions, whereas at the second phase a minimum of one explicit reason for their exclusion must be documented. Numbers and reason are provided in the supplemental material. Covidence systematic review software (www.covidence.org) was used to screen and select included studies.

Data extraction and risk of bias assessment

A data extraction form was developed (Appendix 4) and was piloted on one of the included studies. One author extracted the data from the included studies and the other author confirmed the extracted data. Any discrepancies were resolved by consensus. The data extraction form included information relating to the publication: title (title of paper, abstract, report), report ID (if there are multiple reports of this study), reference details, publication type (e.g., full report, abstract, letter), type of study, aim of study, participant description (total number, age, sex, ethnicity), institutions (library, location, quantity), outcomes and results.

Suitable existing risk of bias criteria were not available and consequently, a set of criteria had to be developed using a recent systematic review of the perceived value of public libraries (Sørensen, 2020) as inspiration. The risk of bias was then assessed using this set of criteria: (i) selection bias, (ii), a priori values bias and (iii) outlier values bias (Table 1). Each criterion consisted of a set of signalling questions and was assessed for a low, high, or unclear level of risk.

(i) Selection bias. There are many stakeholders to consider (e.g., politicians, staff, non-users) and they can have very different characteristics. Knowing the characteristics of the participants can help reduce the risk of bias as various stakeholder perspectives can be taken into account. Consequently, not considering the characteristics of participants was considered a high risk of bias.

(ii) A priori determined values. Some studies may ask participants to rank a set of already determined values, which may have an influence on the results (e.g., if the list of values put emphasis on certain aspects of the academic library or by introducing values on the list at various levels of specificity). Applying a list of predetermined values was considered a high risk of bias.

(iii) Outlier values. A stated value may be the opinion of just one individual or a segment of stakeholders. If generalizability for the found values was not analysed, this was considered a high risk of bias.

Quantitative analyses were not performed as the included studies present lists of values that cannot be quantified. Consequently, the sections in the PRISMA 2009 checklist (Moher et al., 2009) that relates to meta-analysis are not followed in this systematic review.

Results

A total of 8,320 unique records were identified in the literature searches and an additional forty records were identified through citation searching (Figure 1). After removing duplicates 6,061 records were screened of which twenty-eight full-text articles for eligibility. Assessing the full text of the twenty-eight articles we found further studies not eligible for inclusion. Following PRISMA, the reasons for excluding at full text stage are available (see Appendix 3). In total, twenty-five were excluded due to the study's wrong outcomes, not being primary, being a duplicate, or having a focus on specific positions, programmes or services. Furthermore, a PhD thesis had to be excluded as it could not be obtained in full text. Therefore, eleven studies were included in the qualitative analysis. See the Appendix for an overview of the studies.

Short descriptions of the studies are provided below.

Balog, Badurina and Lisek (2018) investigates the perceptions of 138 doctoral students at the Electrical Engineering and Computing faculty at the University of Zagreb. This mixed-methods explored the role of the academic library in supporting doctoral students' academic and research work. The results show that the students who uses the library appreciate the following:

• The availability of both library staff and information
• Speed of service
• Staff that is friendly, accommodating and courteous.

Furthermore, the study finds that the participants perceived library staff as a secure and competent resource to retrieve the information they need.

Connaway et al. (2017) analyses how academic libraries can communicate value to higher education stakeholders and how to increase learning and success amongst students. The study applies focus groups and semi-structured interviews of fourteen provosts and fourteen academic library administrators and finds multiple types of values as mentioned below:

• Communication
• Teaching and learning
• Collaboration
• Service
• Space
• Collection
• Student success
• Research support
• Inclusivity and diversity
• Provision of tech
• Teaching support
• Accreditation

Datig (2014) explores themes of international students' conceptualizations and envisions of libraries. The study methods consist of an online survey and individual interviews. Forty-two international students from New York University Abu Dhabi, located in the United Arab Emirates, completed the online survey. Seventeen international students participated in interviews. The survey and interview data were coded by the author using a grounded theory approach to qualitative analysis and the following themes emerged:

• A library is for books
• A library is for academics
• The library as place
• Librarians take care of books
• Culture of libraries
• Higher purposes of the library

Furthermore, the study finds that the international students perceived the library as an aspirational place and a place to preserve knowledge for the future.

Fowler (2016) explores the 'essence' and lived experienced with the library. The study is a qualitative, transcendental phenomenological study based on interviews and focus groups with eight-three participants (administration, researchers, students and management) from twenty-seven samples (convenience samples from public research universities in the Mid-Atlantic region of the United States) and finds the following themes:

• The aspirational library
• The library as servant
• The commons
• Information resources
• Stewardship

Murray and Ireland (2018) analyses the results of a survey study of college and university provosts' and chief academic officers' preferences for communicating library impact. The study seeks to understand what types of data will help prioritise library budgets at an institution. Their results include a list of practices that can be used as evidence for advocating the prioritisation of academic library budgeting:

• Space
• Staffing
• Instructional outreach
• Resources

Nel and Fourie (2016) aims to demonstrate the value of using multiple methods for collecting data and planning services to meet current trends and informational needs of academic library users. Using questionnaires and focus groups, the study investigates the perceptions of 361 researchers at the Faculty of Veterinary Science at the University of Pretoria. In the investigation of the participants' information needs, information seeking and information use, a list of library assets emerged:

• Access to information and information resources (support with literature searches; excellent collections, inter-library lending)
• Affect of services (Good interactions, helpfulness, competent staff)
• Training and workshops

Nitecki and Abels (2013) explores a broad segment of academic library users' perceptions of the value of academic libraries in order to investigate the root causes of library values. The study analyses data from interviews and focus groups with participants from one United States university faculty, comprising researchers, teaching staff and administrative staff. Multiple themes were derived from the focus groups:

• To increase my productivity (e.g., could not do as much or as well without the library)
• To expand student ability to get information
• To do my job for teaching, research, and writing (e.g., prepare online class, write book chapters, prepare grant, save time)
• To save money (e.g., cannot afford to buy publications)
• To indulge intellectual curiosity

The interview data complemented these findings with the following aspects:

• To not feel frustrated (e.g., due to clunky process, hard to use or not intuitive interface, not sure how to search)
• To meet accreditation criteria specifically for faculty performance (e.g., to understand how to do research, publish, and present)
• To change the university (e.g., to change the way we think about learning and learning outcomes)

Furthermore, the study identifies a series of factors that enhance these values: information resources (archives, reserves, stacks, electronic); staff; access (circulation, inter-library lending, online catalogue); assistance (instruction and reference); purpose of use (e.g., reason to use the library); archival or historic value of student work; access to online materials and information; and the physical space for faculty and students.

Thompson (2014) analyses faculty use and perceptions of libraries using a survey sent to a random sample of non-librarian faculty members at colleges and universities in the United States that grant bachelor's degrees or higher, excluding faculty in the health sciences. A total of 1,531 respondents participate in the survey. The survey includes questions regarding five roles, which are:

• Gateway
• Buyer
• Archive
• Teaching Support
• Research Support

The analysis resulted in the development of two additional constructs in this area:

• Negative perceptions of the continuing relevance of libraries
• Perceived dependence on the library

Vaagan and Holm (2004) presents the results of a survey study of value orientation among Norwegian librarians, library staff and information professionals. The survey respondents include eighty-five college or university librarians. They find that the values prioritized are the following:

• Spreading information literacy
• Free access to materials and to information
• Spreading knowledge and literacy

The lowest ranked values are:

• Protection of user confidentiality
• Creation of culture of reading
• Encouragement of cultural diversity

Volentine and Tenopir (2013) explores the role of the academic library collections in teaching as well as research. The data is extracted from an open-ended survey sent to all academic staff members at six UK higher learning institutions. Over 2,000 academic staff members completed the survey, in which 941 comments were made. They analyse the role of the library by presenting anecdotal evidence that is not summarized in specific values.

Webb (2007) draws on personal experiences, unstructured short interviews and a workshop that explored the ways in which libraries support scholarship and enhance research activity. Describing researchers and their information needs informs the principles underlying libraries supporting research activity. The following roles are identified:

• Gatekeeper
• Translator
• Information specialist
• Subject expert
• Safe harbour
• The fount of all knowledge
• Counsel, colleague and critical friend

Qualitative synthesis

Design, choice of stakeholders, institutions and size varied considerably among the included studies (Appendix).

Study design. Four studies analysed the values of the academic libraries as perceived by stakeholders using surveys (Murray and Ireland, 2018; Thompson, 2014; Vaagan and Holm, 2004, Volentine and Tenopir, 2013). The latter survey primarily analysed the comments made in open-ended questions and did not include statistical analyses of the responses. Four studies used qualitative methods (Fowler, 2016; Nitecki and Abels, 2013; Connaway et al., 2017; Webb, 2007). Furthermore, three studies used mixed methods (Datig, 2014; Balog, et al., 2018; Nel and Fourie, 2016).

Choice of stakeholders. The majority of studies (6) analysed the perceptions of academic staff members or faculty members (Thompson, 2014; Volentine and Tenopir, 2013; Webb, 2007; Nel and Fourie, 2016; Nitecki and Abels, 2013; Balog, et al., 2018). Two studies analysed the perception of provosts (Murray and Ireland, 2018; Connaway, et al., 2017). The latter also included library administrators. Finally, one study analysed the perceptions of librarians (Vaagan and Holm, 2004). One study included representatives from a number of stakeholder groups (e.g., library directors, library staff, students and faculty members) (Fowler, 2016). One study included international students (Datig, 2014).

Institutions. Five studies include participants from institutions in the United States (Fowler, 2016; Thompson, 2014; Nitecki and Abels, 2013; Murray and Ireland, 2018; Connaway, et al., 2017). Two studies include participants from UK higher learning institutions (Volentine and Tenopir, 2013; Webb, 2007). The rest of the studies include participants from a number of different countries around the world: the United Arab Emirates (Datig, 2014), South Africa (Nel and Fourie (2016), Croatia (Balog, et al., 2018) and Norway (Vaagan and Holm, 2004).

Size. Six of the included studies have fewer than 100 participants (Datig, 2014; Vaagan and Holm, 2004; Fowler, 2016; Nitecki and Abels, 2013; Connaway, et al., 2017; Webb, 2007). Two survey studies have almost 1,000 and more than 1,500 responses respectively (Thompson, 2014; Volentine and Tenopir, 2013). The remaining three studies have 138 to 361 participants each (Balog, et al., 2018; Murray and Ireland, 2018; Nel and Fourie, 2016).

Risk of bias assessment

Risk of bias was assessed using three criteria: (i) selection bias, (ii), a priori values bias, (iii) outlier values bias. An overview of the assessments made is available in Table 2 and a detailed description of the risk of bias assessments can be found in the supplemental material.

There were several cases of unclear or high risk of bias with respect to all three criteria. The selection bias criterion showed low risk in six studies, unclear in two studies and high risk in three studies. Using the a priori values bias criterion it was revealed that five studies had low risk, whereas three studies had an unclear risk of bias and three studies had a high risk. Finally, using the outlier bias criterion it was revealed that five studies had a low risk, five studies had an unclear risk, and one study had a high risk.

Discussion and conclusion

This study aimed to determine the perceived values of academic libraries by stakeholders using systematic review methodology. Eleven studies were eligible for inclusion in the review.

Although rigorous methodology is used in the review process there are limitations. As there is no validated assessment tool suitable for the purpose, we developed our own set of risk of bias criteria. A validated tool strengthens the risk of bias assessment; however, our tool was not validated. Cochrane's risk of bias tool for assessing randomized trials is an example of a validated set of criteria (Higgins, et al., 2011). Using the developed set of three criteria, all eleven studies have a high risk of bias for one or two criteria. Some of the studies have determined the values a priori and are thus biased towards preliminary stated values, whereas new aspects of values are not explored. Furthermore, some studies include an unbalanced group of participants to represent views of specific stakeholder groups, which leads to overrepresentation of specific subjects. However, we also find studies with little risk of bias for all of the criteria.

Few similar reviews exist that can inform the discussion on which values stakeholders tie to the academic library. A recent review of the value of public libraries by Stenstrom, et al. (2019) categorizes the values in to these three types of support: personal advancement, vulnerable populations and community development. However, as the results are focused on public libraries, they are not easily transferable, which is also the case for Sørensen (2020) who reviews a wide variety of values from nineteen studies. However, the included studies in this review can also enlighten the discussion. Several issues are prominent: development over time, stakeholder foci, diversified terminology and perceptions of value.

The role of the academic library and the academic librarian is undergoing reconceptualization according to Llewellyn (2019). In this review the oldest study (Vaagan and Holm, 2004) seems to put emphasis on collections and information literacy. However, as the publications years range from 2004 to 2018, the time period is too short to make firm conclusions on development over time in the perception of the values of academic libraries.

Perceptions of some stakeholders have been investigated more than others. Generally, most of the studies explore the perception of researchers and faculty members. The perceptions of provosts have also been analysed in several studies, whereas library staff and students have been analysed less.

The terminology on values varies considerably. Although there is little evidence to support the metaphor often used of the library as the 'heart of the university' (Grimes, 1998), many similar metaphors exist which may be because of the difficulties of describing the actual values of the library. The included studies in this review use varying terminology to describe what ISO 16439 (2014) defines as the values of the library: perceptions, role, essence as well as value. Furthermore, the use of metaphors is widespread. Examples of metaphors include the fount of all knowledge, the gateway or the library as servant. The intangible values of the academic library can be difficult to describe and metaphors may describe it better.

These eleven studies show that perceptions of value, not just terminology, vary. However, there are some common denominators. Some of the studies focus on access to information and information resources (the library as archive, buyer) or as a space that is by the stakeholders. Many studies put emphasis on stewardship (spreading knowledge and information literacy). Consequently, collections and spaces are not are not enough when trying to grasp the values of the academic library. Outreach, training, support activities and the like are also important components in the understanding of the values. Furthermore, the library staff are also mentioned in several studies as a value. The participants in the included studies mention the competencies of the staff, their friendliness and the confidence that the library staff can secure access to all necessary information sources. Consequently, the library staff is of great importance to the values perceived by stakeholders.

The library is more than collections, services and staff. Several studies mention the aspirations of the library, the higher purpose and the library as counsel, colleague and critical friend. Fowler (2016) argues that libraries can begin focusing on fulfilling the need of the universities by realigning their purpose from collecting, preserving, and providing access to information to intersubjective communication. Fowler (2016, p. 79) describes intersubjective communication as '[th]e provocation and enabling of users to challenge their current knowledge in a manner that builds greater understanding.' Fowler argues that the library is a phenomenon, not the input (collections, buildings, staff). It is an experience that continues to evolve with every intentional act.

Summing up, these eleven studies on perceived values of academic libraries explore or assess the values of academic libraries. These studies are found when performing a comprehensive search strategy and duplicate study selection. A number of facets of values are identified in these studies and focus on what can be termed input: resources, space and staff. Services are also mentioned as a value of academic libraries. Furthermore, many of the included studies also report values such as the aspirational library, the library as counsel, colleague and critical friend indicating that the values of academic libraries are not solely understood by their input. The academic libraries may be understood as an experience or intersubjective communication.

About the authors

Tove Faber Frandsen is professor WSR at the University of Southern Denmark. She can be contacted at: t.faber@sdu.dk
Kristian Møhler Sørensen is PhD student at the University of Southern Denmark. He can be contacted at: krms@sdu.dk

How to cite this paper

Appendix 1: Additional tables

Appendix 2

Supplementary material – literature search

Appendix 3

Supplementary material – excluded studies

Appendix 4

Supplementary material – Data extraction form and risk of bias assessment