Issues and challenges in researchers’ adoption of open access and institutional repositories: a contextual study of a university repository

Introduction

Institutional repositories are becoming increasingly commonplace in tertiary education institutions. Research shows that they provide a platform to showcase a university’s research output, allow greater visibility for research, aid in research discovery and the long-term archiving of publications, increase citation counts, are good financial propositions for a university and are a measure of its prestige (Lee, Burnett, Vandegrift, Baeg and Morris, 2015; Norris, Oppenheim and Rowland, 2008; Pinfield, 2015; Stanton and Liew, 2012). Discussion of open access has been increasing in academic discourse, as have funding imperatives that mandate open access repositories across several universities in the UK, Europe, USA, Australia, New Zealand, India, and Japan. These factors have further encouraged scholars to consider ways of making their work open and accessible to the largest possible audience. Universities are also envisioning institutional repositories as a platform to make their research visible by taking advantage of publishers’ green open access policies. Institutional repositories are increasingly being designed with the capacity to support the archiving of research data as well as publications.

Nevertheless, institutional repositories remain an underutilised resource in many universities, for they are often inconsistently and poorly designed (Betz and Hall, 2015; McKay, 2007a), revealing a lack of common understanding between the various stakeholders (St. Jean, Rieh, Yakel and Markey, 2011). There is also a significant gap in the understanding of institutional repositories between those who design and maintain these systems (often library staff) and the university researchers (often academics located within specific disciplines) who deposit their outputs into them (Lagzian and Abrizah, 2015; Rodriguez, 2015).

This paper first surveys the current literature on the adoption of institutional repositories, and then outlines a user experience study conducted at a university library in Australia to understand the specific issues for researchers within the institution. Interviews were conducted with academics and researchers about their awareness of open access, their scholarly communication practices, and their use of the university repository. The usability of the repository was also evaluated. An action research approach was used to address issues on an ongoing basis, based on the findings.

The findings are twofold and relate to two main issues: the design of the institutional repository itself, and individual researchers’ perceptions of institutional repositories. The findings also highlight an unintentional disconnect between university researchers and librarians; researchers and librarians both have increasing demands on their time, but the librarians, who are often the ones curating the institutional repositories, approach the issue from an instructional or governance perspective, whilst the researchers think of it as yet another demand on their time. This calls for greater collaboration between researchers and librarians in order to move towards open access and better uptake of institutional repositories.

Literature review

This literature review focuses on institutional repositories, and situates this research within current discussions on open access and of academic social media platforms that act as repositories, such as ResearchGate, Academia.edu and Google Scholar (Hammarfelt, de Rijcke and Rushforth, 2016).

Scholarly communication and open access

While the term open access has been in use for more than twenty years, changes in researcher behaviour, publisher approaches and research funder policies suggest that it is now a necessary approach to publishing and disseminating research outputs (Pinfield, 2015). Nevertheless, ‘to busy academics, institutional repositories are in danger of being perceived as a “service push” rather than a user-focused solution to improve processes and means of communication of scholarly research, preservation and impact’ (Marsh, 2015).

Subject-specific open access repositories, such as ArXiv for physics and mathematics, and PubMed Central for biomedical sciences, are widely used and have become established as crucial resources in those fields. Other disciplines such as humanities, arts, and social sciences, however, experience low levels of open access adoption, and it would appear that such repositories have not become part of the established workflow for researchers in these fields (Creaser et al., 2010, p. 146).

Although academics resist any institutional mandates, they are intrinsically motivated by an ‘altruistic desire to share research outputs with a specific community’ (Cullen and Chawner, 2010, p. 145), leading them to deposit in subject repositories specific to their disciplines. While institutional repositories make up 83% of the total number of repositories in existence, the majority of research outputs are stored within subject repositories (Pinfield, 2015, p. 617). This imbalance indicates that researchers prefer disseminating within their own disciplines rather than in institutional repositories, since the disciplinary repositories offer more effective dissemination within their own disciplines.

Academic social networks such as Google Scholar, ResearchGate and Academia.edu have also outpaced other forms of online dissemination, including personal Websites and repositories (Laakso, Lindman, Shen, Nyman and Björk, 2017, p. 125). These services have the potential to increase readership and citation counts of research outputs, yet their legitimacy remains a contentious issue, for many are operated by private, commercial companies (Bond, 2017). Nevertheless, the popularity of such platforms, contrasted with the low level of participation in institutional repositories generally, raises an interesting question: why do academics overlook institutional repositories when disseminating their research?

Barriers to open access publishing

Björk (2004) identified barriers to open access a decade ago, and the study by Forrester (2015) confirms that these barriers still remain, especially barriers that Björk categorised as economic, legal, social and psychological, albeit from different perspectives. Whilst most academics agree that the current system of journal publishing is simply not sustainable, many still have a traditional approach to publishing (Waller, Revelle and Shrimplin, 2013). While many academics accept the need for changes to the current scholarly publishing model, the majority still conform to traditional publishing practices (Peekhaus and Proferes, 2015, p. 642). Tenure and promotion practices in many institutions appear to reinforce this system, resulting in many academics prioritising publishing in conventional, high-status journals which are looked upon favourably by committees (Odell, Coates and Palmer, 2016).

Furthermore, academics are often already constrained by administrative work, termed shadow work (Butterwick and Dawson, 2005) that restricts their research and writing time. This is compounded by their increasingly tight teaching schedules, all of which results in an opportunity cost in the form of wasted time and resources within funded research (National Science Foundation, 2014). Academics often raise concerns about institutional policies related to self-archiving, citing a presumed impact on workload, as well as issues such as ‘author choice in the journal of publication, academic freedom, rights retention, and publisher relations’ (Fruin and Sutton, 2016, p. 477). This view suggests that academics do not clearly understand the benefits of open access repositories (Kim, 2011; Marsh, 2015). Bell, Fried Foster and Gibbons (2005) argue that this issue is the most significant barrier to the success of an institutional repository. As pre-prints are often not cited, this means also that work must be formally published for authors to gain professional recognition in the form of impact (McKay, 2007a). These factors make it challenging to increase engagement with the institutional repository.

Tenured professors are more likely to adopt experimental publishing and dissemination models than untenured professors, and are therefore more likely to use an institutional repository (Kim, 2011; Serrano-Vicente, Melero and Abadal, 2016). Another study found that although untenured professors attribute higher levels of importance to the free accessibility of research, tenured professors are more likely to have published in an open access journal (Peekhaus and Proferes, 2015, p. 650).

Some sceptics also consider open access publishing to represent material that is of low quality, not peer-reviewed, not the final version of an article, or vanity publishing (Creaser et al., 2010, p. 152). In countries where open access is not common practice, academics tend to associate open access journals with ‘ephemeral publishing, poor archiving and low prospects for career advancement’ (Peekhaus and Proferes, 2015, p. 643). Singson, Joy, Thiyagarajan and Dkhar (2015) suggest that proactive outreach by librarians to researchers is necessary in overcoming these negative perceptions and attitudes towards open access journals.

Institutional repositories

Institutional repositories hold tremendous potential in breaking down the barriers of access to scholarly communication, yet many remain under-utilised and infrequently accessed (Armstrong, 2014).

There is a significant body of literature on institutional repositories (Pinfield, 2015), including overviews (Bluh and Hepfer, 2013), case studies of particular institutions (Serrano-Vicente, Melero and Abadal, 2016), country-specific studies (Chowdhury, Uddin, Afroz and Sameni, 2011), regional studies (Sajjad Ahmed and Al-Baridi, 2012), and format studies including repositories of theses (Stanton and Liew, 2012) and of data (Antell, Foote, Turner and Shults, 2014), and more recently, the usability of institutional repositories (Luca and Narayan, 2016).

Academics’ participation in institutional repositories

Support from academics is essential in populating a repository, though in many cases developing buy-in is challenging (Betz and Hall, 2015). In Swan and Brown’s study, 51% of 1,296 researchers surveyed had not self-archived a copy of their work in the last three years, and a substantial proportion of those authors (36% of total researchers surveyed) were not even aware that it was possibile to provide open access to their work through self-archiving (Swan and Brown, 2005, p. 43).

Publisher policies are often challenging to interpret, raising concerns around breaching copyright conditions and around embargo periods (Creaser et al., 2010). Antelman (2006, p. 92) found that self-archiving practices vary between disciplines, and that it is the disciplinary norms for scholarly communication which drive author self-archiving behaviour, and not their understanding of publisher policies. Additionally, there is a lack of awareness about what is and what is not an acceptable pre-print copy for self-archiving, which may result in academics choosing to avoid self-archiving in open access repositories altogether (Kim, 2011).

This stands in stark contrast to the ease with which academics can upload their outputs to academic social networks such as Academia.edu. In these academic social network sites metadata is pre-populated and co-authors can share the same document file across profiles (Laakso et al., 2017). It is easy to see and the reasons for low adoption rates of institutional repositories when comparing them with such sites. Repositories can offer long-term preservation of materials in a way that commercial academic social networks do not guarantee, though this preservation aspect does not appear to be in the forefront of academics’ thinking in light of the ease with which they can upload research to commercial academic social networks.

In Australia, where this study was conducted,

the government research funding policies for the university are not tied into open access deposits in the repository, but instead tie into peer-reviewed journals and articles published in journals with high impact factors. Accordingly, the university also overtly rewards traditionally published work, whether or not it is placed in the repository. Consequently mobilization has been haphazard. (Kennan and Cole, 2008).

Librarian-academic dynamics

Often, the access to closed academic materials that a university library provides to their researchers also makes their work invisible in such a way that academics do not experience any barriers to accessing information resources. This can inadvertently result in academics’ low level of motivation to contribute to the institutional repository.

Burns (2014, p. 203) maintains that librarians have a key role to play in this new scholarly landscape, and must respond to these changes with programmes and policies that match opportunities to the needs of users. Librarians might support academics’ adoption of open access by assisting with rights and permissions, maintaining scholarly communication Websites, evaluating open access journals for quality, organising workshops on copyright issues and digital scholarship, shaping discussions of open access policies, and providing training sessions and information to support academics in making their work open access (Emmett et al., 2011, p. 566; Rodriguez, 2015, p. 610).

Librarians often see research outputs as an organisational resource to be managed (Armstrong, 2014), with a view to collecting, cataloguing and sharing all research production of the institution (Marsh, 2015, p. 166), while researchers are more concerned about their research profile, tenure, career progression, and academic freedom (Armstrong, 2014, p. 43).

Xia (2011) argues that librarians who create and manage institutional repositories hold an outsider’s, or etic, perception of scholarly systems, and consequently encounter difficulties developing an insider’s, or emic, understanding of scholars’ needs and attracting their contributions. According to Xia,

OA journal publishing reaffirms the importance of taking a combination of emic and etic approaches for librarians and faculty scholars to work in partnership and reach real understanding of the scholarly communication system. Those who can take the advantage of both insider and outsider views will survive in the transformation of library services. (p. 89)

Xia’s findings also point to the need for greater collaboration between librarians and academics in order for librarians to fully understand the needs of the research community.

A user experience approach to institutional repositories

User experience has become an increasingly valued framework for examining the library technologies we support and create from the user’s perspective. While the definition and scope of user experience is still contested, scholars generally agree that user experience is dynamic, context-dependent, and subjective (Law, Roto, Hassenzahl, Vermeeren, and Kort, 2009). Focusing on user experience allows us to examine more conceptual attributes of the system, including usability, meaning, affect and value.

Usability of digital libraries can be broken down into three categories: content, functionality, and the user interface. While much has been written about the users of information retrieval systems in general, there is little research examining the accessibility and usability of institutional repositories (McKay, 2007b) from the end-user’s perspective. This lack of understanding of repository users is likely a factor in the inaccessibility and limited uptake of these systems.

In short, the literature seems to point to the fact that rather than a discussion of the value of open access publishing, or the value of institutional repositories, current discourse is about academic and researcher adoption of open access publishing and institutional repositories. Our study addressed the latter issue, namely, institutional repositories, within the specific context of an Australian university.

Methodology and context

This study is situated within the context of an Australian university library, namely the UTS Library, which supports the University of Technology Sydney (UTS), and was conducted in 2015 and 2016. The UTS Library uses the open source DSpace repository package which is focused on digital archives and long-term storage, and was launched in 2004. The UTS Library is responsible for creating and maintaining a repository of all research outputs by the university’s scholars and researchers. The university mandates that open access versions of all research outputs need to be deposited in the university repository managed by the library, and yet we found that even after five years of the mandate being in place, the adoption rate was only around 30%.

Action research

Action research is an important source of learning for librarians and information professionals, and has been used extensively in the social sciences and in education.

Action research, devised in 1950s, was spread in information sciences since 1990s, with results that have been interesting in promoting change of involved people and organizations. The method is based on engaging a group coordinated by a researcher. Problem analysis is functional to realizing improvement interventions, in a recursive process of reflection, action, evaluation, and result sharing. (Moroni, 2011, p. 24)

We have adopted this approach using an academic-librarian collaborative action research framework (Bruce, 2001) through an academic-practitioner partnership. A librarian and a library and information science researcher partnered together to conduct this study with the support of both the university library and the researcher’s faculty.

User experience analysis

As part of this process, we first conducted a user experience analysis. We used two complementary approaches to evaluate user experience. The first was an evaluation using the heuristic evaluation method (Nielsen and Molich, 1990), wherein a small set of evaluators, no more than six to eight people (Nielsen and Landauer, 1993), examine the interface and judge its compliance with recognised usability principles (the heuristics), which are:

• visibility of system status,
• match between system and the real world,
• user control and freedom,
• consistency and standards, and
• error prevention

Since the end users of the institutional repository can be anyone from a researcher to a student, we recruited two users from each of three user persona categories for the usability test of the repository: two researchers, two library students, and two librarians.

The second was an evaluation using the task-by-type taxonomy created by Ben Shneiderman (1996), which specifies that a good visualisation of a digital library should:

• provide an overview of the collection,
• allow users to zoom in on items of interest,
• allow users to filter out uninteresting items,
• allow details on demand for selected items,
• allow users to view relationships among items,
• allow users to keep a history of actions, and
• allow users to extract a sub-collection through search parameters.

Qualitative interviews

Finally, we conducted interviews with six academics from humanities, arts, and social sciences. We asked about their current publication practices, self-archiving practices, and also university repository interactions and experience. The questions focused on the following main areas around their scholarly communication practices:

• choice of publishing outlet (e.g., looking up Journal Citation Reports),
• their researcher profile (e.g. using ORCID IDs and Google Scholar Profiles),
• increasing the visibility of their research outputs (e.g. using open access, self-archiving in the institutional repository),
• communicating and promoting their research (e.g. using Twitter, blogs etc.), and
• measuring the impact of their research (e.g. using the h-index and Altmetrics).

Findings

The results of the user experience study revealed issues with the usability of the university repository, and furthermore, a mismatch between user expectations and the design of the system and its interface. The repository used the open source DSpace application, but it was not fully customised to the institution. Some issues identified from the heuristic evaluation and the cognitive walkthrough were as follows.

Limited information discovery due to issues of navigability

Combined findings of the user experience study revealed several issues with the navigability of the repository. The repository software interface was centred on searching rather than browsing, with a search box or basic listing of research areas as the only navigation options. Researchers were often curious about what others in their disciplines had published, without necessarily knowing the names of the researchers or any keywords to search for. Since many faculties were large enough that not everyone knew all their colleagues, this was an issue of discoverability and visibility that one researcher said prevented them from using the repository. This, in turn, affected their own motivation to upload their publications as pre-print copies for self-archiving and open access. This connects with the literature about the need for disciplinary repositories in order to motivate researchers to use them, both for looking up others’ outputs, and for uploading their own (e.g. Pinfield, 2015).

There was also no site-hierarchy in the institutional repository. The organisation of the repository’s content and the way in which it was presented gave no clear direction to the user as to where they may start looking for content. Most users, upon seeing the home page, skipped straight to the search box; however, the lack of contextual information as to what can be searched left users in the dark about how to use the search box.

Issues with information architecture

We found that DSpace’s community > collection structure did not make sense to many of the researchers, who were more used to an organisational hierarchy of faculties and research centres.

This is because of jargon or terminology that was inherited from DSpace, the open source software, which did not make sense within the context of the university. All participants found that the repository interface was cluttered with far too much information, including a lot of metadata and statistics that were not meaningful to them. While all of DSpace’s communities and browsing categories (title, author, etc.) were visible on the homepage of the repository, it was not clear to users what the context of these items was within the site. This was an issue which was part of the navigation and information architecture of the software and the resulting Website on account of a lack of customisation in order to cater to the university’s specific needs.

Issues with user experience

There was no global site hierarchy or logical organisation of content on the repository. It was a confusing system for the users to navigate. Upon clicking on a community, for example, the site showcased a new form of hierarchy, and the community categories that were on the home page were no longer accessible to the user unless they went back to the home page. Many of these user experience issues were easily fixed through library initiatives such as customising the DSpace interface and building a more Web-like interface, which can be seen at https://opus.lib.uts.edu.au/. This redesign process is reported in Luca and Narayan (2016). The site is now more visual and is organised to match the faculties, schools, and research centres at the university (under a more disciplinary structure), and hence is much easier for researchers to navigate and search. The institutional repository now aligns more closely with Shneiderman’s 1996 task-by-type taxonomy. Open access and closed access outputs are clearly marked along with the progression of the item within the repository, once the researcher deposits it.

Usability issues with the deposit interface

Researchers at the university previously had to deposit published versions of research outputs using a university research information management system, and deposit the pre-print manuscript separately in the university’s repository. The first step is essential for academics’ annual workload requirements, and although the second was mandatory, very few actually deposited pre-prints as there were no serious consequences for not doing so. After the university implemented a single deposit system through the implementation of software from Symplectic Elements, the burden of some administrative tasks was eased for researchers and academics, and the system also means that academics do not need to engage with the institutional repository directly. The downside of this is that a significant number of academics were unaware that the system existed, mirroring findings by Kim (2011). Researchers are now asked to upload a version of their work that can be made available open access: in most cases this is the accepted manuscript, i.e., the version after peer-review, but before copy-editing and formatting by the publisher.

Our study found that this distinction is not clear to many researchers and academics, for they are often not aware of their own rights as authors. Consequently, the repository commonly stores the publisher’s version (which it harvests from Symplectic Elements), which in most cases goes into closed access, sometimes even when the publication itself may be an open access journal. The confusion at this point in the system resulted in many items appearing as closed access, which were often fixed on an ad hoc basis based on user requests.

None of the participants had a Google Scholar profile at the time of the interviews, but many had an ORCID ID because the faculty’s research office had helped them to create one. Nevertheless, the profiles were not up-to-date, because although the university’s research management system downloads information from ORCID, ORCID does not automatically download information from the university’s research management system.

Misconceptions about open access

The qualitative semi-structured interviews conducted with six researchers from within the university reinforced several of the insights from the literature, and also pointed to specific issues within our context.

All of the participants considered the prestige of the outlets they published in, and this was often determined by older metrics rather than through newer metrics that include open access publications, social impact, Altmetrics, etc. None of them used Journal Citation Reports, or had an ORCID account. Only one had a Google Scholar profile. There was confusion about open access publications as several of the researchers had heard about predatory publishers in their disciplines, and many of them also receive lot of email solicitations from such publishers every day. This makes it hard for them to differentiate between quality publishers and predatory ones.

In regard to publishers’ copyright and self-archiving of pre-prints and post-prints, none of the academics were aware of the licensing around their own publications and did not know that they could look it up on SHERPA/RoMEO.

All participants had an active Facebook presence where they posted their new publications to their friends (within a closed network), but none of them were active on Twitter although they had an account, and hence did not do any public dissemination of their citations or outputs. A few had blogs but didn’t track their impact. Two of the academics wrote frequently for The Conversation about trending topics, but they did not engage with the comments threads on the site, finding it very upsetting sometimes.

Some other issues identified from the interviews were the inability to export citations the inability to link to the supervisor’s profile (for digital theses), and the inability to download access metrics directly from the repository. All of these are informing the ongoing redesign of the repository interface.

Discussion and conclusion

The findings of this study indicate that several of the issues and challenges described in the literature still remain. Many of the participants did not care about institutional mandates; some still had a stratified notion of publishing outlets and their prestige, and almost all of them did not have the additional time required for the deposit of two versions into the various institutional systems: a post-print version for institutional verification purposes, and a pre-print version for open access purposes.

Through a collaborative approach between librarians and academics, and through two separate methods of data collection – one that was task-based around the university’s institutional repository, followed by qualitative interviews that investigated the academics’ scholarly communication practices – we were able to identify and address some of these issues at a local level, while some more general ones such as attitudes to and awareness of open access still remain. Through adopting a user experience design approach and conducting a thorough usability study of the repository interface, we were able to inform the library technologists in the redesign of the DSpace interface and customise it for the institution. The resulting Web interface is more intuitive and meaningful for academics and researchers at the University of Technology Sydney, and also aids information discovery online for others looking for papers on any given topic through Internet searches. A number of academics are unaware of the institutional repository, and those who do know about it rarely contact the library directly to ensure that their publications are appropriately archived with a pre-print open access version available. Academics are required to record their research outputs for the purpose of their own accountability to their faculty, but many do not understand that this administrative task actually has a tangible impact on the visibility of their research. All of this points to a need for more scholarly information literacy amongst researchers, and librarians have a large role to play in that.

Acknowledgements

The authors wish to thank the UTS Library staff for their support, specifically Mal Booth, the University Librarian, who is a passionate advocate of open access. We also wish to thank Dr Brenda Chawner for organising an excellent Research Applications, Information and Library Studies 2016 conference in Wellington, New Zealand. Finally, we wish to thank the study participants, the reviewers, and the wonderful Amanda Cossham, Editor, Australasia/S.E. Asia for Information Research.

About the authors

Dr Bhuva Narayan is a Senior Lecturer at the School of Communication at the University of Technology Sydney (UTS), Ultimo, NSW 2007, Australia. Her research interests include information behaviour, user experience, and design thinking. She can be contacted at Bhuva.Narayan@uts.edu.au
Edward Luca is an Academic Liaison Librarian at the University of Sydney Library, Camperdown, NSW 2006. His research interests include scholarly communication literacy, embedded librarianship, and user experience design. He can be contacted at Edward.Luca@sydney.edu.au

How to cite this paper