Public library Websites as electronic branches: a multi-country quantitative evaluation

Introduction

Public library Websites are often the first, if not the main, point of contact that people have with their local library. In the future, public library patrons may not go into the bricks and mortar building but just access the Website for the services, materials, and programmes that are needed. Members access a public library Website for a variety of reasons, including locating resources, reserving resources, managing library accounts, bill payments, downloading e-books and audio books, and accessing online databases available free through their library. A library Website can be regarded as an electronic branch of the library. An electronic branch is the Website of the library versus the bricks and mortar version of the library where patrons physically enter the library.

A physical library branch will not be used if it cannot be accessed, if patrons do not know where or how to ask for help, if the information and resources are out of date, and if the building and furniture are dilapidated and unappealing. The same can be said of the electronic branch. An inviting, easily accessible and vibrant Website is an essential part of a successful electronic branch. Library electronic branches also need to be developed, managed and maintained just like any other library branch. In short, the electronic branch needs to be easy to access and use.

Public library Websites provide information that includes catalogues, databases, events, marketing, staff information, programming, and services. In this study, 1,517 public library Websites were assessed and evaluated in a study over four years. The intent of the project was to find out what basic and desirable information was available on the Websites and to make recommendations about how Websites can be improved. The study extends previous research carried out in Pennsylvania (Powers, 2011), from which a checklist of criteria for evaluating Websites was developed. Powers conducted the research in 2010 and used a random sample of 305 points of contact (URLs) of libraries in Pennsylvania. She used ‘yes' and ‘no' responses to code the answers. According to Powers, a basic library Website contains ten of the criteria, while a desirable Website would include eight additional criteria. For this study, these were the criteria used and it was based upon two papers: an article by Persichini, Samuelson, and Zeiter (2008) which set out the ten basic features and another by Mathews (2009), which set out eight desirable criteria. The criteria are described in the Research methods section. This particular checklist was used as it was something that could easily be explained and used for student researchers. While the checklist had aspects that were unique to the USA, these could be changed and adapted for whichever location one chose to use.

Literature review

Evaluation and assessment of Websites

Evaluation of library Websites began as early as 1994, when the Internet and World Wide Web were in their infancy. Empirical studies on Websites in other industries are informative regarding the questions to ask and the frameworks needed to move forward with the study of library Websites (Cao, Zhang and Seydel, 2005; Chua and Goh, 2010; Teo, Oh, Liu and Wei, 2003). The majority of literature available on library Websites relates to the academic library sphere. Although these findings can inform the research on public libraries, they have a different focus. Academic libraries focus on curriculum and supporting academic staff and students with their scholarly research, while public libraries support lifelong learning and leisure. Literature describes the criteria for evaluating Websites as follows below.

Usability

A Website's usability and the quality of its content are important aspects of its viability. According to Poll (2007, p. 1), ‘contents, language, structure, design, navigation, and accessibility' of Websites are key areas of focus for libraries. Determining what usability is can be different for each member of the community. Past research based on users' acceptance of a particular system has shown that usability is based upon the way the site functions or works (Goodwin, 1986; Wang and Senecal, 2007).

Usability has been defined as ‘how well and how easily a visitor, without formal training, can interact with an information system of a website' (Benbunan-Fich, 2001, p. 151). According to Nielsen (2012, para. 3) usability is defined as a ‘quality attribute that assesses how easy user interfaces are to use'. Nielsen goes on to argue that usability refers to the methods for improving the ease of use during the design process. He further defines the five quality components of ‘learnability, efficiency, memorability, errors, and satisfaction' (Nielsen, 2012, para. 4). Implementing Nielsen's usability heuristics will improve any Website.

User-centred design

In order to add value for community members, public libraries should stimulate user involvement and allow users to give feedback (Sørum, Andresen and Vatrapu, 2012).

Nielsen proposed simple user-centred design guidelines that he called usability heuristics. Nielsen's suggestions were to ‘make screens simple and natural, speak the users' language, be consistent, provide feedback, use plain language for error messages, prevent errors, and provide clearly marked exits' (Nielsen, 1996, p. 34). None of Nielsen's suggestions are difficult to envisage when considered within a library framework. These suggestions can make a library Website move from a hard to navigate site, to an easy one that users will want to come back to visit. If a Website is difficult to navigate, users will leave and not return (Nielsen, 2012). For a public library, this has to be avoided, as return visitors indicate that the electronic branch of the library is also a valued branch.

One aspect of good user-centred design is finding out what the community wants and expects on the Web page. It is important to remember that the Website exists for the users, not the library staff (Goodman and Schofield, 2015). Library literature underscored the importance of usability testing and having an iterative process during the design phase (Becker and Yannotta, 2013; Dominguez, Hammill and Brillat, 2015). If a library is redesigning their Website, many different methods can be used. Becker and Yannotta (2015) in their project at Hunter College Libraries used an iterative usability testing process to improve their Website. Dominguez et al. (2015) used a think-aloud protocol during usability testing, focus groups, and card sorting during their many phases of Website design from 2001 to 2012. Becker and Yannotta, as well as Dominguez et al., used many of the processes that Nielsen (1996; 2012) suggested to carry out user-centred design, in order to improve the accessibility of the Website.

Accessibility

Riley-Huff (2012) suggests that every library should seriously consider accessibility for all. When putting the Website together, each piece has to be considered from the standpoint of each potential visitor who will be looking at it. Accessibility can be difficult if there are blind, deaf, or people with other disabilities visiting the library site. A simple colour choice can be something that can make a Website unavailable to a colour blind person. Conway (2011) did a Website audit of public libraries in Western Australia during 2010 to find out if they adhered to the W3C Web Content Accessibility Guidelines and discovered that none of the public libraries adhered to those guidelines. These Guidelines (an international standard) make recommendations for Websites that are ‘perceivable, operable, understandable and robust' (World Wide Web Consortium, 2011).

Other studies focused on public libraries

Metrics are needed to measure the success of Websites for public libraries. Public library Websites can be regarded as the electronic front door of the library. The impression they make may be viewed by some patrons as an indicator of the level of digital services of the library. Bérlanger et al. (2006) suggest that owners' goals for the Websites will differ and therefore the success of the site may need to be specific to the information and audience. Public library Websites have multiple purposes. Community members can use the Website to look for the services and programming that is available at the library, they find information about the physical branch such as the address, telephone number, and operating hours; and gain access to available e-services (Chow, Bridges and Commander, 2014; McMenemy, 2012).

A literature review of studies that mainly focus on public libraries identified only a few worldwide. Beside the study by Powers (2011), which is the basis for the methods used here, there is Conway's (2011) audit, mentioned above, and Maatta Smith (2014), like Conway, looked at specific information for users to find on Websites of urban libraries in the USA. She concentrated on information related directly to disabilities and found that ‘this was one of the greatest areas of disparities among libraries' (Maatta Smith, 2014, p. 199). McMenemy's study researched the emergence of digital libraries in Scotland, which found that there was a ‘consistent set of electronic resource… offering high quality information…' (McMenemy, 2012, p. 525). He also found difficulties with using resources, as there was confusion about terminology across different library services. Chow et al. (2014) studied usability design comparing academic and public libraries, as each user group has different needs and functions. They focused primarily on how the pages were laid out, how the navigation worked, and if the Websites adhered to recommended design guidelines. Overall, Chow et al. determined that more than 70% of the libraries they studied had not conducted usability studies.

Research method

This project partially replicates Powers's study in Pennsylvania, where a random sample of public library Websites were evaluated and assessed against the following eighteen criteria in a checklist (Powers, 2011, p. 8-9):

The research was conducted by postgraduate students who are pursuing a library and information management degree at the University of South Australia, as part of an information management project course. This course was part of the student's capstone project under supervision of the course coordinator and lecturer. A total of forty-six students participated in the project over four years, in groups of four to six students per semester. This research project was ideal for students who needed to do virtual projects, because of either work or family commitments, instead of being placed at a host industry location. The students were located in different states throughout Australia and some were located internationally. The use of forty-six student researchers helped to eliminate bias in the project that would occur if only one person conducted the study (Raward, 2001). Students were required to write a 1,500 to 2,000 word literature review on any aspect of Website assessment, evaluation, or usability.

To assess and evaluate the Websites, nine groups of students worked over nine semesters (four years) from February 2013 to November 2016 to evaluate 1517 Australian, Canadian, and American public library Websites using a pre-determined protocol adapted from Powers' 2011 research. Each student was assigned a minimum of twenty-five public library Websites. The students were given a template (in Excel) on which they had to capture information for each public library. Powers used ten criteria from a study done in Idaho, which determined the elements that a basic public library Website should have (Persichini, Samuelson, and Zeiter, 2008; Powers, 2011). The remaining eight criteria that are considered desirable for library Websites, were based upon an article by Matthews (2009). Powers (2011) combined the two sets of criteria and used it in a spreadsheet protocol to assess which of the items appear on the sample Websites. Two of the criteria that Powers used were excluded for this study: first, the names of library board members were not included in the protocol, as Australian libraries do not have boards; secondly, the description of library services available to patrons was also removed. Instead, a section was introduced for student researchers to make free form comments regarding each public library Website, including what the student researcher thought of the Website, the navigation of the site, programming, or other general comments. The student researchers were encouraged to enter their opinions about the Websites in Item 20 in Table 1. Another deviation from the Powers study was the inclusion of branch libraries in the data, which was time-consuming. Some of the branches share information with the main branch; others have their own Website presence.

The two sets of criteria (basic or desirable) that were used by Powers are shown in Table 1 below.

Many of the students added their own opinions about what constituted a good or bad Website. The disadvantage of having students do the research is that the lead researcher could not evaluate all 1,517 Websites; the advantage is that it helped to mitigate the bias that could occur when only one researcher does the whole investigation (Raward, 2001). The lead researcher did spot checks of some of the Websites to make sure the findings were consistent with what the students recorded on the spreadsheets and to check the quality of the findings.

The binary (yes/no) responses to the questions were converted into numerical data (in Excel) in order to do descriptive statistics.

Research question

The main research question guiding the study was: What kind of online access do public libraries provide to their community members?

Findings

Descriptive statistics

The researchers reviewed 1517 public library Websites between February 2013 and November 2016. The data were gathered in Australia (February 2013 to February 2014), followed by Canada (July 2014 to November 2015) and then the USA (July 2015 to November 2016).The state and provincial data for each country will be shown first. Then all data will be collated, discussed and compared. The data were processed using Excel.

Australian data

Table 2 shows all the states and territories in Australia where the public libraries are located. The Australian Capital Territory (ACT) has one library system with multiple branches throughout the territory. In Tasmania, the State Library of Tasmania is responsible for the public library system and all their Websites are part of the state library system. In South Australia, the public libraries adopted the One Card system during the time the research was conducted. The One Card system requires all public libraries in the state to be part of the one library management system with only one public library catalogue for the state. Users need only one public library card to access any public library in the state. Library users can also reserve an item from any library through the online catalogue, after which the item is delivered to their library of choice (Strempel, 2013). The system went live in South Australia in 2012. The 130 South Australian libraries were incrementally converted to the One Card system over a period of three years from 2012 to 2014 (Strempel, 2014).

Table 3 shows all the provinces and territories represented in the public library Websites in Canada.

The provinces with the largest number of libraries were Alberta, Ontario, Newfoundland and Labrador. Libraries from Quebec were also included, as one of the student researchers had lived in Quebec and could speak French.

Table 4 shows all four the regions of the USA represented in the public library Websites. The Appendix shows how the states were grouped into regions.

Comparison of Australian, Canadian, and American public library Websites

The project was not initially designed as a comparative study. It started as an Australian project, but was expanded because of the success of that project. The project was first expanded to include Canada and then the USA. Such a comprehensive study allows comparison between countries. One cavaet must be mentioned, namely that the study did not include large urban or small rural libraries in the USA, but mostly suburban libraries situated just outside the large metropolitan areas that serve a population between 25,000 and 100,000. In Canada and Australia, libraries of any size and location were included in the study.

Joint-use facilities

Joint-use facilities were notable in the results, with 351 of the 1,517 sites surveyed (23.1%) falling into this category. A joint-use facility is a library building that is shared with another type of library (e.g., school, Technical and Further Education, or university libraries), another type of cultural centre (e.g., gallery or museum), or another type of council or governmental facility (e.g., a community centre or city government council centre). Each country had joint-use Websites, with Canada having the highest number. The top categories for joint-use facility by type were, first schools, then community centres, followed by other types in third place. ‘Other' refers to any type of facility that was not captured in the eight categories listed, such as tourist information centres, family centres, cultural centres (not an art gallery or a museum), post office, welcome centres, women's institute, local history rooms, and gardens. In Australia, joint-use facilities were found in 22% (108) of the public libraries. Within the 108 public libraries, four states noticeably have the majority of the joint-use facilities: New South Wales (26), South Australia (26), Tasmania (21), and Western Australia (26). The other four states or territories with joint-use facilities only had a maximum of four such facilities. The two most common types of joint-use facilities were public schools (30) and ‘other' (32) including family centres, tourist information centres, or a conglomeration of more than one function like a regional gallery, council office, and leisure centre.

As mentioned above, joint-use facilities are also prevalent in Canada. Joint-use facilities were found in 27% (159) of the public libraries that were included in the study. The three most common types of joint-use facilities were public schools (63), community centres (34) and meeting rooms (31) which together represented 80.5% of the joint-use facilities of the library Websites. Within the 159 public libraries, three provinces have the majority of the joint-use facilities, namely Newfoundland and Labrador (63), Ontario (35), and Alberta (33). The other provinces or territories only had minimal numbers of joint-use facilities.

In the USA, joint-use facilities were present in 20% (84) of the public libraries. The two most common types of joint-use facilities were community centres (29) and ‘other' (25). These two categories accounted for 64% of the joint facilities of the libraries in the USA.

Table 5 shows the joint use facilities by type for Australia, Canada and the USA.

Top five and bottom five criteria

The top categories included in the Websites in Australia were the address, phone number, hours of operations, and link to the catalogue. Spelling and grammar used on the Websites were also of a high standard (refer to Table 6). The top categories in Canada were address, phone number, hours of operation, and having text and images on the Website. Similar to Australia, the spelling and grammar on the Websites were of a high standard. Australia and Canada also had the library's street address on the website. Unlike both Canada and Australia, the address was not one of the top five categories found in the USA Websites. Libraries in Australia and the USA had links to their catalogues on the Website. Sites in Canada and the USA included text and images.

The five criteria that were included in the fewest public library Websites for each of the three countries are shown in Table 7:

Three of the bottom five categories were shared by all the countries, namely asking for help, being able to give feedback about the Website, and an indication when the Website was last updated. Australia also had a very few cases where the director's name and the library manager's name appeared on the Website. In Australia, many of the libraries are associated with councils and the councils tend to manage the libraries' marketing, so not many distinct icons or images were present on the libraries' Websites. In Canada and the USA online contact details were not included on many of the Websites. In Canada, the search box was not available in 45% in the libraries. In the USA, there was a lack of an e-mail address in 47% of the libraries.

Basic and desirable data

The research with this particular group of libraries produced some interesting results on the distribution of basic and desriable data. Most of the libraries have basic information about their library on the Website:

The desirable information on the Website depended on what library management deemed important for the patrons and varied across the country, state, county, and municipality. The council's information technology department could also suggest what is included and excluded on the Website. This list of the desirable elements is in accordance with that of Powers (2011). Table 9 shows the summarised results of each country's libraries:

Number of items included

The study identified the number of items that were included on the Websites of the libraries in the three countries. Figure 1 below shows the USA data (solid bar) as a normal distribution (bell curve) that peaks at 13 items. Of the Canadian Websites, 293 out of 599 (49%) contained 13 or 14 items. The USA and Australian numbers were not as high as that of Canada. Australia's data follows a wavy bell curve that peaks at 51 libraries containing 11 and 13 items, followed by 48 libraries with nine and 10 items. Between 9 and 15 items are included in 300 of the 488 libraries (62%). Canada's numbers were higher; 483 libraries (81%) had between 11 and 17 criteria on their Websites. The USA numbers varied from 11 to 17 items for 325 libraries (77%). This number is a little lower than Canada, but the items are spread more evenly. The number of items for Canadian Websites peaks at 14.

When looking at the data by category (Figure 2), it is obvious that, in some areas, one country's public libraries have more effective Websites. Given that there were more Canadian Websites than either Australian or American in most categories or topics, they have a larger impact on the data.

As public library Websites represent the electronic branch, certain data elements need to be included. In many cases, the majority of the libraries' Websites contained the data we searched for, but some included minimal or no data. Either these libraries did not have the ability to provide the services the community needed, or they did not have access to the required funding. Despite operating in the 21st century, some libraries still have not fully committed to including relevant information on their Website. The researchers discovered that static Websites still exist in all the countries. A static Website is one that does not have any links, search boxes or any way to contact the library through the Website. In a static Website there is no way to search for anything and the Website is simply a placeholder Website that has the name of the library, address, telephone number, and the hours the library is open.

Discussion

In the 21st century, public libraries need well designed Websites to encourage community members to access and use their services. Public library Websites can also be referred to as electronic library branches as opposed to the bricks and mortar branch that can be physically entered. This paper reported the findings from research that focused on the accessibility and usability of public library Websites, regarding the inclusion of numerous criteria or items on the Websites. Although some well-designed Websites exist in all three countries included in this research, none of the libraries included all the criteria on the research template. These libraries can improve their Websites by following good Web design principles, especially user-centred design (Goodman and Schofield, 2015; Nielsen, 1996; 2012).

This research contributes to the understanding of public library managers about how the accessibility of their Websites can be improved, to make them effective points of entry to the programmes and services required by community members. The research questions were answered as follows:

What kind of online access do public libraries provide to their community members?

The library Websites in the three countries provided varying degrees of access, first to the Website and secondly, to the resources. Access to a Website can be provided through the city council's site or a separate library Website and examples of both types were included in the research. Some of the Websites were static and included only basic information like the library name, address, telephone number, operating hours, and other branch locations. On some of the sites the user could link to maps and interactive information to find access to databases, catalogues, lists of books, as well as information tailored for a particular patron type (e.g., children, teens and seniors).

The resources that can be accessed also varied. Some Websites do not provide access to any resources, except to a catalogue link. Others made a plethora of information available, but users needed a library card number to obtain access to the databases. For example, in Australia the Websites provided links to the National Library of Australia's Trove database, as well as State Library resources. In South Australia specifically, the streaming services for e-books and movies is done through the State Library of South Australia. In Canada and the USA, the library databases varied and some also included streaming services for movies and e-books.

Were the users able to contact the public library through the website?

Four criteria directly determined whether the library can be contacted through the public library Website. Firstly, the Website can contain an online contact link where the customer can complete a form requesting library staff to contact them. Secondly, an email link with either a form or an email address for the library's general site can be provided. A feedback form can be included to ask questions about sources, programmes, or just about anything. Finally, there could be an ‘ask for help' link. Many libraries have this as a way to go into chat or instant messaging. Interestingly, three of these four items were not on many of the libraries' Websites.

Table 10 illustrates that online contact, feedback forms, and ask for help functions are all areas that are not done well by all of the libraries in our research study.

How can users use the Websites to contact the library?

Contacting the library can be done in many ways, e.g., by telephone or e-mail, in person, through the Website, by providing feedback about the Website, and online contact forms. All of these methods, except in person contact, were captured as part of this research. The majority of the libraries show telephone numbers and addresses to facilitate personal visits to the bricks and mortar site, and e-mail addresses are included on the Websites, but not to the same degree as physical addresses and telephone numbers. E-mail addresses were not often included for all three countries (Australia 59%, Canada 74%, and the USA 53%). In fact, e-mail information was in the bottom five categories for the USA.

Contacting the library through providing feedback appeared in the bottom five categories for all three countries. Search boxes were available on the majority of the Websites for the three counties combined (801/1517, 53%), but Canada and the USA (55% and 63% respectively) were better at providing search capability than Australia (41%). Online contact forms were not popular for providing contact on any of the library Websites. Of the Canadian Websites, 53% provided online contact forms, but these figures were lower in the case of Australia (37%) and the USA (37%). Providing feedback on the Websites was available on only a small minority of Websites for all three countries and it seems that the libraries prefer not to receive feedback about the content and quality of their Websites (Australia 27%, Canada 32%, USA 36%).

Study limitations

A limitation of this study was that inexperienced researchers were used in conducting research. However, the research design has been validated in previous studies and the questions were already developed, which simplified replicating the study with inexperienced researchers.

Conclusions and recommendations

Four major observations emerged from the analysis of the data:

1. From the student comments, some of the Websites that were evaluated were high quality Websites that gave users a positive experience and welcomed them back repeatedly. These public library electronic branches used an intuitive approach and user-centred design to provide the patron with the experience they expect when visiting the site.

2. Based upon our knowledge of how public libraries work with councils, public libraries have to adhere to policies and procedures that are designed and implemented by information technology professionals in their city council offices. This often precludes the development of a custom Website, which is not the image that library management want to portray. When discussing this with public library managers during conferences, it was obvious that they were not always able to do exactly what they felt was necessary with their Websites.

3. From the research, it was obvious that Facebook and Twitter were still the most popular social media sites used by the public libraries. The inclusion of other Web 2.0 tools, such as Instagram and Snapchat, would improve the ability of public libraries to reach younger target audiences in order to improve engagement and perceptions about what libraries actually provide communities.

4. Public libraries do not communicate well to their communities and they do not provide sufficient mechanisms to allow their users to respond to the library staff. Websites lacking information such as e-mail addresses, staff names, feedback mechanisms, contact forms, etc. indicate that library staff are not interested in their users' opinions about their Website content and quality.

The research question investigated the access of libraries and resources through the Websites for the users. In all cases, the electronic branch of the public libraries could be found. However, true access includes satisfaction with the type of information the community members need. Some of the public libraries had databases and catalogues on their Websites and others did not. In the digital world that our communities live in today, there is an expectation that our public libraries need to inhabit this world and provide access to digital sources. The programming and other services that public libraries create for their stakeholders is the other aspect the staff provide.

The study has provided rich binary data on more than 1500 public libraries in three countries. This data shows that some of the public libraries provide a better quality Website experience than others. Overall, some of the staff in the exemplar libraries could guide other library staff towards improving their sites and providing better services to their community.

As part of future research, the researchers will revisit the Australian Websites to determine whether there have been any changes to the public library Websites and whether the original research findings are still valid. Websites evolve and change over a four year period. There have also been amalgamations in councils and changes in how the Australian councils are configured in different states. Changes in political structures will also have influences on how libraries are managed and what will and will not be placed on a library Website. It will investigate similar aspects of Websites, but will expand some of the questions to provide additional information. We also suggest that future research should investigate what types of tools and resources are available on the library Websites.

Acknowledgements

The authors would like to thank the following students who assisted with the research while undertaking Information Management studies at the University of South Australia: Andrew Angus, Sarah Barkla, Andree Brett, William Chan, Michelle Chitts, Natalie Clifford, Seam Connelly, Robin Costelloe, Renee Davy, Lungile Dlamini, Daniel Easterbrook, Jenny Gallas, James Gaunt, Cory Greenwood, Fariha Hamid, Deanna Harrison, Jed Hood, Kerrie Impiombato, Catharine Kelly, Thea Kitchen, Isabelle Laskari, Matthew Leach, Annie Lewis, Zoe Manger, Daniel Mason, Cyprian Maynard, Tyler McCaffery, Claire Morris, Patrick O'Leary, Margaret Parker, Lauren Pillar, Jesse Pollard, Tamsin Reed, Brett Ross, Paula Saegenschnitter, Jessica Smith, Jasmine Taylor, Claire Telfer, Malcolm Thompson, Sarah Thompson, Amelia Trimboli, Amy Vanner, Jana Waldmann, Luke Watsford, Jaq Wheadon, and Fiona Wilkinson.

About the authors

Diane L Velasquez is the Program Director of the Information Management Program at the School of Information Technology and Mathematical Sciences, University of South Australia. She holds a BA in political science from San Jose State University, an MBA in Management from Golden Gate University, a MLS from the University of Arizona, and a Ph.D. from the University of Missouri. Her research interests are in risk management and disaster recovery, administrative and management of information agencies, and evaluation and assessment of public library websites. Dr Velasquez can be reached at diane.velasquez@unisa.edu.au.
Nina Evans is Associate Head, School of Information Technology and Mathematical Science (ITMS) at the University of South Australia. She holds tertiary qualifications in Chemical Engineering, Education and Computer Science, a Masters' degree in Information Technology, an MBA and a PhD. Her research interests relate to business-IT fusion, information and knowledge management, enterprise social networks, innovation and ICT education. She can be contacted at nina.evans@unisa.edu.au

How to cite this paper

Appendix