Proceedings of the Tenth International Conference on Conceptions of Library and Information Science, Ljubljana, Slovenia, June 16-19, 2019
The concept of usefulness in library and information science research
Isto Huvila, Heidi Enwald, Noora Hirvonen and Kristina Eriksson-Backa.
Introduction. There is not much doubt that information, information services and systems need to be useful. In this light, the relatively lack of conceptual elaboration of the concept of usefulness in the library and information science literature can be regarded as somewhat surprising.
Method. This paper provides a conceptual overview of the use of the notion of usefulness in library and information science literature, explicates its relation to key parallel concepts, and on the basis of an empirical vignette in the context of health information research, discusses the potential limits and advantages of referring to usefulness instead of and together with other related concepts.
Analysis. A review of literature relating to the concept of usefulness was conducted to examine how it has been used in library and information science.
Results. A close reading of the literature shows an overlap between related concepts but at the same time, diverging foci of interest in and emphasis on what and how information, information services and, for instance, information systems are considered beneficial or suitable for their users and particular uses.
Conclusions. There is a need for better conceptual clarity in the literature regarding usefulness and related concepts. The review shows that usefulness can be literally a useful concept for addressing the user and use (versus e.g. system, content or topic) perspective to engagements with people, services, systems and beyond.
There is little controversy in claiming that information, information services and systems need to be useful. In this light, the relatively lack of conceptual elaboration of the concept of usefulness in the library and information science literature can be regarded as somewhat surprising. Undoubtedly, this can be partly explained by the traction of related and quasi-similar concepts like relevance, acceptance, quality and usability. Of these, especially relevance has been a key concept in the field, although, as the fairly frequent reviews of the notion show, there is no real consensus of what it is. Or, to quote the subtitle of the recent book The Notion of Relevance in Information Science (Morgan & Claypool, 2016) by Saracevic, ‘[e]verybody knows what relevance is. But, what is it really?’. However, considering both the sustained, albeit uneven, interest in the concept of usefulness and its, literally, intuitive usefulness, a better understanding of the notion and its place in the conceptual apparatus of the field would be highly welcome.
The aim of this paper is to provide a conceptual overview of the use of the notion of usefulness in library and information science literature, explicate its relation to key parallel concepts of relevance, acceptance, value and usability, and on the basis of an empirical vignette in the context of health information research, to discuss the potential limits and advantages of referring to usefulness instead of and together with other related concepts. The present work is based on close reading of a selection of library and information science literature related to usefulness and related concepts. By doing this, we aspire to revitalise and bring novel viewpoints into the discussion on various aspects of the ‘goodness’ of information in the contemporary metatheoretical landscape of library and information science.
Usefulness and related concepts in library and information science literature
Usefulness meets value, usability and user acceptance
In general, both in library and information science and beyond, the most typical understanding of usefulness is to see it as the degree to which something, often a system, enhances job performance or task completion (Davis, 1986). As Burns notes in the context of ergonomics research, usefulness 'requires that a design has the functionality required to accomplish work domain objectives (as opposed to just being easy to use)' (Burns, 1997, p. 313). In contrast to generic usefulness, Buchanan and Salako (2009) define system usefulness based on how closely the content and services offered by a system meet user requirements. From a broader perspective, the usefulness of a system can be understood as a degree to which a system supports users’ activity (Burns, 1997), helps them to 'meet their daily needs' (Sin, 2013, p. 108), or otherwise has some significance for them (Kegan, 1970). Conspicuously, in spite of the popularity of the concept, fairly little is known of the factors that make people think that something is useful (Nocera et al., 2007; Gefen & Keil, 1998).
Greenberg and Buxton (2008) discuss the difference between usefulness and usability distilling the typical distinction in the literature by noting that
'technological landscape is littered with unsold products that are highly usable, but totally useless. Conversely – and often to the chagrin of usability professionals – many product innovations succeed because they are very useful and/or fashionable; they sell even though they have quite serious usability problems' (Greenberg and Buxton, 2008, p. 116).
A similar distinction can be made between usefulness and acceptance (cf. Dillon, 2001). Users can be prepared to accept things even if they are not useful or usable. However, in spite of the differences, or a possibility to make a distinction, the concepts are closely related. Both usefulness and usability jointly influence user satisfaction (e.g. Tsakonas and Papatheodorou, 2008), usability and usefulness are antecedents of user acceptance (Davis, 1989), and as an unsurprising consequence, as Tsakonas and Papatheodorou (2006) observed of usability and usefulness, it can be difficult for users to distinguish them when asked.
Whereas (system) usefulness has remained largely an information systems issue even in the context of work related to library and information services in library and information science research, there are studies that focus on the usefulness of information. Taylor (1991) refers to useful(ness) as implying 'ways of resolving a problem through clarification, alteration, or actual solution as a result of information gained’ (Taylor, 1991, p. 221). This particular definition reminds of Wilson’s notion of ‘best textual means’ to one’s end (Wilson, 1968, p. 21) that can be understood as an optimal degree of the usefulness of a text for one’s purposes. Tsakonas and Papatheodorou (2006) write that 'usefulness is the degree to which a specific information item will serve the information needs of the user'. Partly, usefulness can be determined by the inherent attributes of a resource, as a semantic entity and as an object, and partly by the applicability of information to specific contexts of information activities and work tasks (Tsakonas and Papatheodorou, 2006). The conceptualisation of usefulness as an attribute comes close to the idea of perceiving it as a measure of the value of information (Lu & Gustafson, 1994). In a broader sense as a measure of the worth of information, usefulness can also be seen as a factor that influences the emergence of particular regimes of information (Ekbia & Evans, 2009), ‘worlds’, ‘polities’ (cf. Boltanski & Thévenot, 2006) or social contexts where the usefulness or value of information is determined in a particular sense. To exemplify the regimes, a cookbook can be useful as a source of information on healthy diet within a health-related regime of information whereas from the perspective of a gastronomic regime, it can be useful in preparing a meal that tastes delicious.
Even if there has been a general tendency to frame the usefulness of information in relation to rational needs (following the common inclination of information research to focus on rational information needs and behaviours, Huvila, 2012), the reasons why information is considered useful can also be diffuse, emotional and intangible. Kegan (1970) noted that resources (or information items) can be considered useful if they have, in a broad sense, some significance for users (i.e. that they have some type and degree of relevance) even if they do not lead to a specific action, that the usefulness of an information resource increases in pace with its providers knowledgeability of user’s needs, and that a resource could be useful not only because of its information content but also because it 'causes a cognitive restructuring of the researcher's mind or a ''free association.'' (Kegan, 1970, p. 185). Kari (2007) pursues a similar line of thought by distinguishing the use of information from its effects.
There are also examples of how usefulness has been used in common-sensical terms. Alpi et al. (2016) operationalised usefulness in a study of medical images 'as whether the viewer could identify the feature described by the author in the caption accompanying the image', and, for instance, Abdullah and Mal-Allah (2011) and Yoon and Kim (2014) study the usefulness of information without explicitly defining the concept.
Usefulness meets relevance
Apart from its affinities with concepts such as value, usability and user acceptance, usefulness has obvious kinship with one of the most prominent library and information science concepts – relevance. Not surprisingly, as Strassheim (2018, 3) points out, concepts such as usefulness, utility or need have been central in numerous models of relevance proposed by library and information science researchers. Usefulness complements topical relevance with a 'user side' (Saracevic, 2016, p. 23) accounting for users and their interactions with systems (Belkin, 2015) and as its related concept, it is of obvious interest for user and use-centred information research (Tsakonas & Papatheodorou, 2006).
Even if relevance is one of the key concepts in library and information science, the most prominent theories of the notion stem from outside of the discipline. Especially two seminal theories, the one of Schütz (1970) in the interface of philosophy and sociology and Sperber’s and Wilson’s (2011 ) work in the field of linguistics and communication have become influential both in interdisciplinary scholarship and in library and information science research, whereas a third major corpus of relevance research in the context of logic has remained largely uncommented in library and information science (Saracevic, 2007).
The beginnings of the line of research relating to relevance in the context of search and retrieval of information within dedicated information systems can be traced back to the 1930s. In this research tradition, multiple types, levels and dimensions of relevance have been defined (Strassheim 2018, 10). Why relevance became – starting with the early work already in the 1930s (Saracevic, 1975) – and why it still is, a central notion of information science, has been discussed recently by Saracevic (2015) in his keynote speech in the ISI 2015 conference and in the aptly titled book The Notion of Relevance in Information Science: Everybody knows what relevance is. But, what is it really? from the following year (Saracevic, 2016). The problem of unwanted, non-relevant information identified in the 1950s has not disappeared, quite contrary, and has grown from a nuisance of information retrieval specialists to a global concern (Saracevic, 2016).
Several types of relevance have been described in the literature. Strassheim (2018, p. 2-3) states that relevance relates to our access to 'information objects' by three interconnected levels: access to the object itself, the information gained from it and the use, and therefore usefulness, of that information. These three levels echo the fundamental division between user relevance and subject or topical relevance proposed by Vickery in 1959 (Vickery, 1959, 1959a; Saracevic, 2016) and the still prevailing standard procedure to base evaluations of information retrieval systems on the comparisons of user relevance and system relevance (Saracevic, 2016). From the perspective of its users, the same information object can convey different information, 'depending on how it is interpreted by different individuals, in different settings, or at different points of time' (p. 3). Even if the user perspective has been in the focus of a lot of research within information retrieval and library and information science since the 1980s, the majority of the information retrieval systems are still based on the classical model of topical relevance (Saracevic, 2016). Moreover, relevance is measured in a context-independent manner in information retrieval practice (Jiang et al. 2017) even if it has been evident for a long time that it is ineffective for measuring the impact of the systems on their users (Hersh, 1994).
Unsurprisingly, when we think about what could be perceived in colloquial sense as useful, the subjective or user-based type of relevance specified by Saracevic (1996) is more central than system-specific algorithmic relevance. Subjective relevance is concerned 'with the aboutness and appropriateness of a retrieved information object and refers to the various degrees of intellectual interpretations carried out by human observers - whether assessors or users' (Borlund 2003). Saracevic’s category of subjective topical-like relevance sees topic as aboutness, an intellectual assessment of how an information object corresponds to the topical area described by the request for information. A parallel concept, sometimes used for describing aboutness is intellectual topicality (see Borlund 2003, Fig 1 and 2)
Apart from a fit between information, system or service and individuals or groups of users, the sense of what information can potentially be experienced as relevant (i.e. useful) by its users, depends on when and where it is available. Wilson (1973) introduced the idea that relevance judgements vary with the situation or the task already in the early 1970s. Borlund (2003, p. 915) defines situational relevance 'as the utility or usefulness of the viewed and assessed information object(s) by pointing to the relationship between such retrieved object(s) and the work task at hand underlying the information need as perceived by the user'. This type of relevance is highly context dependent and depends, for instance, on tasks and genres of information (Freund, 2013).
Besides situational relevance, Saracevic (1996) defines also motivational or affective relevance, which has obvious links to (perceived) usefulness. However, Borlund (2003) argues that motivational or affective relevance is not an independent type of relevance and can be seen as a characteristic of all of the subjective types of relevance. The drive to want information is an inherent premise of relevance in general (Borlund, 2003). Even if much of the relevance research has accepted the faceted nature of relevance, there seems to be a parallel line of research making a distinction between (topical) relevance and ‘usefulness’. Cool et al. (1993) comment on the inadequacy of topical relevance and propose a faceted concept of usefulness as a complement to topical relevance. Zhang (2014) and Freund (2013) make a similar distinction in their work.
In contrast to attempts to specify the difference between topical and user relevance, Hjørland (2010) argues against the tendency to dichotomise system and user perspectives by underlining that all relevance is human – at least as (long as) systems have no needs. He emphasises subject knowledge as a fundamental aspect of relevance, a perspective emphasised already in the early 1970s by Saracevic. From this essentially domain analytic point of view, relevance depends on the subject knowledge of the person (Hjørland, 2010) who is about to use information. For instance, a healthcare professional is likely to consider different types of scientific and professional evidence-based information – depending on their specialisation – as more relevant, whereas non-professionals prefer information that is accessible, cheap and easy to understand.
In spite of the overlap of relevance and usefulness, it is possible to sense differences in when and how the two concepts are used in the literature. Attfield and Dowell (2003) propose an explicit distinction between perceived usefulness and perceived relevance suggesting that the perceived relevance of information forms a basis of its potential usefulness. Cooper (1971) suggests somewhat more broadly that ‘logical relevance to an information need is an important factor, but by no means the only factor in determining usefulness with respect to the satisfaction of that need’ (Cooper 1971, p. 36). In addition, there are examples where authors have chosen to call topical relevance ‘relevance’ and user relevance ‘usefulness’ (e.g. Mao et al., 2016; Cool et al. 1993). Foskett’s (1972; 1996) surprisingly little commented suggestion to make a difference between relevance and pertinence (defined as utility and usefulness of a document, Foskett 1996, p. 16) is based on a similar distinction. Saracevic (1975) incorporated pertinence in his framework of relevance by distinguishing subject (i.e. subject relevance) and pertinence views of relevance. Saracevic suggests that pertinence, as related to how people know what they know, is a more focussed concept than the broader destination’s (or receiver’s) view of relevance. Kemp’s (1974) elaboration of Foskett’s suggestion that pertinence and relevance can be distinguishes by how relevance relates to public and pertinence to private knowledge is interesting even if, as he acknowledges, making such a clear distinction is problematic.
In contrast to these fairly untypical examples, the distinction between relevance and usefulness has tended to be more subtle. In general, there are fewer studies focusing on user relevance or usefulness of information comparison to studies measuring topical relevance (Freund, 2013). The earlier mentioned study of Tsakonas and Papatheodorou (2006) exemplifies a tentative fault line between usefulness and relevance research. With some reservation, while the focus of relevance tends to be in the content (i.e. information, searching and retrieval – even when it is related to information needs or tasks, e.g. Hjørland & Christensen, 2019; Cooper, 1971), usefulness is used often to emphasise user perspective (e.g. Cool et al. 1993) and systems and task or work performance or completion both in library and information science (e.g. Gerrard et al., 2018; Koohang & Byrd, 1987; Smith, 2003; Tahamtan et al., 2017; Zha, 2013; Zhang, 2014) and in neighbouring disciplines (e.g. Davis, 1989; Holden & Karsh, 2010; MacDonald & Atwood, 2014; Sin & Kim 2013; Terras, 2010).
Usefulness in practice: vignette in the context of health information
In health settings, problems with the usefulness of both information technologies and information are painfully obvious. A part of the problem is a poor fit between technologies and the everyday practices and behaviours of their users (Huvila et al. 2016). Another problem can be that the use of technologies or information in healthcare does not necessarily lead to systematic benefits for patients (Bui et al. 2018).
The practical differences of usefulness and related concepts can be illustrated in the healthcare context using a fictive example. An electronic health record (EHR) system that contains information on the health and healthcare contacts of Lovisa, a 36-year-old electrical engineer working at a company, which develops solar panel technology. The information on Lovisa has been provided by healthcare professionals that Lovisa has met during her visits to several different medical centres and dental care. These professionals include Sally, a general practitioner and John, a nurse, both working at a clinic close to Lovisa’s home, her dentist Therese, and Anders, the midwife who was helping when Lovisa’s son was born. A couple of years ago Lovisa measured her blood pressure using a blood pressure meter she bought at a local general store and reported the measurements in her electronic health record. The measuring device was not of very good quality but reminded Lovisa of her condition and the need to think about her health. The electronic health record does also contain meticulous documentation on the progress of her pregnancy and her complete dental history that has been digitised from earlier paper records. All of this information and the system itself is potentially highly relevant and useful for both Lovisa and her past, current and future healthcare providers, the most of the users of the system have accepted it (i.e. it has a high rate of acceptance) and consider that it is valuable. Depending on situation and individual user, the system and information on Lovisa is not, however, all the time equally well or intuitively described using all the terms. Sometimes it is quite the contrary. In the following section of this article, we several times turn back to Lovisa and her engagements with health information to illustrate how information can be relevant and useful in different senses for different people and what conceptual repercussions these differences could have for library and information science research.
Similarly to relevance (Borlund, 2003), also usefulness is situational. It is not inherent to a specific piece of information, a particular system or service but is constructed in the interaction of a user and that what is being used (Nocera et al., 2007). However, whereas relevance research distinguishes topical relevance and user relevance, usefulness is typically studied as perceived usefulness and is, as such, a concept focused on the use rather than on information. While Borlund writes that subjective relevance is concerned 'with the aboutness and appropriateness of a retrieved information object' (Borlund 2003, emphasis by the authors), Tsakonas and Papatheodorou (2006) note that 'usefulness is the degree to which a specific information item will serve the information needs of the user' (emphasis by the authors). Without claiming that such a distinction would be pertinent to all relevance and usefulness literature, it could serve as a basis for a productive differentiation of the two notions in the spirit of the work of Taylor (1991) who made an explicit distinction between technological, content-driven and a user and uses driven approach to information transfer. These different perspectives could be reflected also on conceptual level by distinguishing technical validity, content-related relevance and user/use-related usefulness. The distinction can be extended even further by considering the differences between users (who) and user-centred paradigm (cf. e.g. Hoffman, 2009) versus the focus on use (how) and non-use following the lead of the discussion in design and science and technology studies (e.g. Woolgar, 1990; Redström, 2008; Baumer et al., 2015). Even if the relevance research has acknowledged the difference between topical and use(r) relevance, relevance can still be considered to be focused on information rather than its use(fulness). In this respect, usefulness could be a useful complement to the conceptual apparatus of library and information science research to highlight, for instance, the momentary, practical and indirect relevance (i.e. usefulness) of information, which is utterly irrelevant by its formal face-value. In the vignette, information on Lovisa’s blood pressure is relevant but because of the cheap blood pressure meter used to measure it, it is not necessaríly useful for making a definite diagnosis. A part of these cases can be undoubtedly explained by the subject knowledge of the person who is using information (cf. Hjørland, 2010) but even a person with considerable subject knowledge can sometimes use irrelevant information for various reasons. In such a situation, the question is when the subject knowledge kicks in and when it does not – when a subject knowledge based judgment is useful or necessary.
The concept of usefulness could also help to bridge the understanding between relevance, quality and diverse values of information as, to paraphrase Zuiderent-Jerak and colleagues (2009) on the societal usefulness of research, a question of acting rather than that of a gold standard. Together with the concept of acceptance (Zhang, 2014) it is closely related to the relevance and value (Taylor, 1991) as well as usefulness of information even if the notions are not synonymous. The issue might not necessarily be how to find information that is useful or relevant as is but rather how to make information acceptable and/or useful in a given situation. In Lovisa’s case, the information on her deteriorating dental health a few years ago could be connected to a period when she was using a manual toothbrush for a year after her previous electric toothbrush had stopped working and before she bought a new one. This information can be made useful in helping to motivate Lovisa to use an electric toothbrush even in the future. Huvila (2015) has described this mode of taking existing information to use (or information taking, Huvila, 2018) as situational appropriation of information and contrasted it with the more frequently addressed seeking of new information.
The critique of the problems with acontextual and static asituational assessments of user relevance (e.g the ephemeral state of relevance of Jian et al. 2017) points also to another argument to consider relevance and usefulness as distinct concepts. Nowadays many information services and systems where individuals seek information are interactive and can also be considered as virtual social worlds, whereas evaluations of relevance and usefulness are commonly done on an individual level leaving aside their complete social and organisational context of utilisation (Franssila, 2017). In the context of health, online support groups are one example of this kind of a virtual social world. Individuals’ judgements relating to the usefulness and trustworthiness of the information are not based solely on individual perspectives but also on social relevance assessments (Costello, 2017; Hirvonen et al., 2019) emerging as cognitive authorities (Wilson, 1983) and, extending the idea, the social usefulness of information in a specific, often fleeting situation and context. Verdicts about the usefulness of information are similarly normative as the judgments of information value (Jaeger & Burnett, 2010) and as Zhang (2014) notes citing Chatman (1996), assume a distinction between insiders and outsiders (communities or individuals), either as opposed to each other or aligned on organisational or community boundaries — for instance, as with boundary objects (Huvila et al., 2017). Haider and Sundin (2019) introduce the concept of friction of relevance to describe the tension between societal and individual interests and assesments of relevance. A similar friction is apparent between the individual and societal assessments of usefulness but also between different individual and social assessments of what is useful and relevant. In this respect, a closer consideration of usefulness and its constituents could also provide new means to approach the conundrum of shifting and overlapping contexts in information research by helping to pinpoint what is useful, relevant, acceptable and, borrowing from Nowotny and colleagues ‘robust’ (Nowotny et al., 2001, p. 168), for whom, when and where. Whereas the general approach of relevance research could probably be described as being oriented towards narrowing down and defining what a specific context is, the concept of usefulness does the opposite by asking what possible contexts there could be. The information on Lovisa’s higher blood pressure can be relevant for Sally and John, that is, the general practitioner and the nurse. Because of the unreliable measurements they might not be useful or of great value for Sally but still relevant, valuable and useful for Lovisa herself in supporting her decision to eat less salt. It is useful also in her immediate social context at home for arguing why the whole family should adapt to her low-sodium diet even if there is an apparent friction between this decision and her husband’s taste for crisps.
Besides explicating the social constituents of why and when information is considered helpful, a closer consideration of usefulness could also provide means to grasp the implications of the breadth of the modalities of information use. These include the material aspects (how the material aspects of information and information use situations influence its relevance or e.g. usefulness), a dimension, which has been criticised of being neglected in the literature (e.g. Zhang, 2014; Lueg & Twidale, 2018), and more broadly, the inter-subjective nature of informing and getting informed (Tuominen et al., 2002; Huvila, 2019). In contrast to the typical content-orientation of relevance and systems-orientation of, for instance, usability and acceptance, referring to usefulness could be a way to put emphasis on the nexus of users, uses and information and their (inter)actions as the shifting locus of the goodness of information.
Finally, a third central insight from the research on usefulness is related to the observation made already by Kegan (1970) who underlined the fact that an information item and information can be useful even if the information itself would not be directly informative or lead to action. This is also an aspect that distinguishes usefulness from pertinence (cf. Saracevic, 1975; Foskett, 1996). Thoburn (2014) and Kwek (2018) make a related point by underlining the occasional significance of useless objects. Similarly to how a useful material object can be important for someone, information can be useful, for instance, because it causes cognitive restructuring or is felt significant. Likewise, there is also plenty of evidence that relevant information is not always useful (Mao et al., 2016; Wilson, 1968) but depends on multiple factors including task and genre characteristics (Freund, 2013) and subject knowledge (Hjørland, 2010). This is also apparent in the healthcare context. There is a lot of professional information that is not useful for non-professionals. In Lovisa’s case, relatively poor quality information that can still be useful, for instance, as a motivation to a behaviour change. In contrast, the professional medical record information she considered as highly relevant (or borrowing from Wilson (1968), ‘best textual means’ to the end of informing about her health) was not that helpful in practice. Even if in most cases a useful piece of information would also be relevant to its users, sometimes relevant information is not useful, and perhaps even more interestingly, it can be useful even if it is not relevant according to any obvious measurable criteria or pertinent to a specific purpose. In order to find out why, it is apparent that more research is needed and as our study shows, much better conceptual rigour is necessary to tease out factors that affect this difference.
When discussing usefulness and relevance, it is still undoubtedly useful to cross the two branches of literature to avoid creating an unproductive gap between the two concepts and conceptual discussions. The proximity of usefulness and subjective relevance means that the two can be operationalised and measured side by side using same and related instruments, and investigated to a degree as two sides of, if not the same, at least closely similar coins. In some contexts, depending on how relevance and usefulness are defined, they may indeed be interchangeable (e.g. Jiang et al. 2017) even if, in general, (judgments of the perceived) usefulness and the relevance of information would not be identical.
The findings of this study have both theoretical and practical implications for future library and information science research. From a conceptual perspective, it contributes to a better understanding of the notion of usefulness and its links to other related concepts, especially to relevance. A more practical implication of the study is that it highlights the importance of a careful consideration of the nuances and consequences of how information and, for example, information services and systems are considered to be advantageous and to whom. As noted already in the introduction, usefulness is and has been a key issue in library and information science research even if the notion itself has often been implied rather than directly referenced. It is apparent that a better conceptual clarity and distinctions between usefulness and relevance would be highly welcome in the literature. It is also important to reconsider the use and definitions of terms, such as usefulness, in the light of the contemporary metatheoretical discussion in library and information science. Instead of using usefulness, relevance and other related concepts as quasi-synonyms, they could be used to mark specific perspectives to information and information (inter)actions. This can be exemplified by referring to Taylor (1991) and his distinction of three different approaches to information and the possibility to think if they all would deserve a specific concept for ‘goodness’, for instance, validity for the technical, relevance for the content-related and usefulness for the user and use centred approach. Further, as noted in this paper, usefulness could open up for addressing the inter-subjectivity and contextual complexity of the goodness of information and the usefulness of non-relevant information. As a whole, as Saracevic (2016) emphasises, usefulness is not a more general concept than relevance but it can be more appropriate, as he suggests, as a criterion for evaluating interactions with information retrieval systems – but also more broadly, as a concept for addressing the user perspective to engagements with people, services, systems and beyond.
About the author
Isto Huvila holds the Chair in Information Studies at the Department of ALM (Archival Studies, Library and Information Science and Museums and Cultural Heritage Studies) at Uppsala University in Sweden, and is Adjunct Professor (docent) in Information Management at Information Studies, Åbo Akademi University in Turku, Finland where he also directs the project Taking Health Information into Account (HIBA). His primary areas of research include information and knowledge management, information work, knowledge organisation, documentation, and social and participatory information practices. He can be contacted at email@example.com.
Heidi Enwald is a University Lecturer in Information Studies, University of Oulu, Finland and a Project Researcher at Information Studies, Åbo Akademi University, Finland. Her research interests focus on issues related to health information literacy, health information behaviour and health communication. She can be contacted at firstname.lastname@example.org.
Noora Hirvonen is a Postdoctoral Researcher in Information Studies, University of Oulu, Finland and a Project Researcher at Information Studies, Åbo Akademi University, Finland. Her research interests focus on health information literacy and related practices. She can be contacted at email@example.com .
Kristina Eriksson-Backa is a University Lecturer, Postdoctoral Researcher and Adjunct Professor (docent) in Information Studies at Åbo Akademi University in Turku, Finland. Her main research interests are information about food and health in traditional and social media, health information behaviour, especially among older adults, understanding of health information, health information literacy, health communication, and e-health. She can be contacted at firstname.lastname@example.org
- Abdullah, A. T., & Mal-Allah, Y. (2011). Nutrition information sources of female athletes at a girls’ sports club in Kuwait: An exploratory study of sources, usefulness, accessibility, and obstacles. International Information & Library Review, 43 (1), 43–52.
- Alpi, K. M., Brown, J. C., Neel, J. A., Grindem, C. B., Linder, K. E., & Harper, J. B. (2016). Scanning technology selection impacts acceptability and usefulness of image-rich content. Journal of the Medical Library Association, 104 (1), 15–23.
- Attfield, S., & Dowell, J. (2003). Information seeking and use by newspaper journalists. Journal of Documentation, 59 (2), 187–204.
- Baumer, E., Ames, M., Burrell, J., Brubaker, J., & Dourish, P. (2015). Why study technology non-use? First Monday, 20 (11).
- Belkin, N. J. (2015). People, Interacting with Information. ACM SIGIR Forum, 49 (2), 13–27.
- Boltanski, L., & Thévenot, L. (2006). On justification. Princeton, NJ: Princeton University Press.
- Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54 (10), 913–925.
- Buchanan, S., & Salako, A. (2009). Evaluating the usability and usefulness of a digital library. Library Review, 58 (9), 638–651.
- Bui, Q. N., Hansen, S., Liu, M., & Tu, Q. (John). (2018). The productivity paradox in health information technology. Communications of the ACM, 61 (10), 78–85.
- Burns, C. M. (1997). Towards viable, useful and usable human factors design guidance. Applied Ergonomics, 28 (5–6), 311–322.
- Chatman, E. A. (1996). The Impoverished life-world of outsiders. Journal of the American Society for Information Science, 47 (3), 193–206.
- Cool, C., Belkin, N. J., Frieder, O., & Kantor, P. (1993). Characteristics of texts affecting relevance judgments. Proceedings of the 14th National Online Meeting, 77–84.
- Cooper, W. S. (1971). A definition of relevance for information retrieval. Information Storage and Retrieval, 7 (1), 19–37.
- Costello, K. L. (2017). Social relevance assessments for virtual worlds: Interpersonal source selection in the context of chronic illness. Journal of Documentation.
- Davis, F. D. (1986). A technology acceptance model for empirically testing new end-user information systems : theory and results (Ph.d. thesis). Sloan School of Management, MIT, Cambridge, MA.
- Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13 (3), 319–340.
- Dillon, A. (2001). User acceptance of information technology. In W. Karwowski (Ed.), Encyclopedia of Human Factors and Ergonomics. London: Taylor and Francis.
- Ekbia, H. R., & Evans, T. P. (2009). Regimes of information: land use, management, and policy. The Information Society, 25 (5), 328–343.
- Foskett, D. J. (1972). A note on the concept of ’relevance’. Information Storage and Retrieval, 8 (2), 77–78.
- Foskett, A. C. (1996). The subject approach to information. London: Library Association Publ.
- Franssila, H. (2017). Coordinative practices and information interaction performance in distributed work (Phdthesis). University of Tampere, Tampere.
- Freund, L. (2013). A cross-domain analysis of task and genre effects on perceptions of usefulness. Information Processing & Management, 49 (5), 1108–1121.
- Gefen, D., & Keil, M. (1998). The Impact of developer responsiveness on perceptions of usefulness and ease of use: An Extension of the Technology Acceptance Model. SIGMIS Database, 29 (2), 35–49.
- Gerrard, D. M., Mooney, J. E., Thompson, D., & Thompson, D. (2018). Digital preservation at big data scales: proposing a step-change in preservation system architectures. Library Hi Tech, 36 (3), 524–538.
- Greenberg, S., & Buxton, B. (2008). Usability evaluation considered harmful (some of the time). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 111–120. ACM.
- Haider, J., & Sundin, O. (2019). Invisible search and online search engines : the ubiquity of search in everyday life. London: Routledge.
- Hersh, W. (1994). Relevance and retrieval evaluation: Perspectives from medicine. Journal of the American Society for Information Science, 45 (3), 201–206.
- Hirvonen, N., Tirroniemi, A., & Kortelainen, T. (2019). The cognitive authority of user-generated health information in an online forum for girls and young women. Journal of Documentation, 75 (1), 78–98.
- Hjørland, B. (2010). The foundation of the concept of relevance. Journal of the American Society for Information Science and Technology, 61 (2), 217–237.
- Hjørland, B. & Christensen, F. S. (2019). Work tasks and socio-cognitive relevance: A specific example. JASIST, 53 (11), 960–965.
- Hoffman, G. L. (2009). Meeting users’ needs in cataloging: What is the right thing to do? Cataloging & Classification Quarterly, 47 (7), 631–641.
- Holden, R. J., & Karsh, B.-T. (2010). The Technology Acceptance Model: Its past and its future in health care. 43 (1), 159–172.
- Huvila, I. (2012). Information services and digital literacy: In search of the boundaries of knowing. Oxford: Chandos.
- Huvila, I. (2015). Situational appropriation of information. Aslib Journal of Information Management, 67 (5), 492–504.
- Huvila, I., Ek, S., Enwald, H., Eriksson-Backa, K., Hirvonen, N., & Känsäkoski, H. (2016). Taking Health Information Behaviour into Account in the design of e-health services. Finnish Journal of eHealth and eWelfare, 8 (4), 153–163.
- Huvila, I. (2018). Ecology of archaeological information work. In I. Huvila (Ed.), Archaeology and archaeological information in the digital society (pp. 121–141). London: Routledge.
- Huvila, I., Anderson, T. D., Jansen, E. H., McKenzie, P., & Worrall, A. (2017). Boundary objects in information science. Journal of the Association for Information Science and Technology, 68 (8), 1807–1822.
- Huvila, I. (2019). Learning to work between information infrastructures. Information Research, 24(2), paper 819. Retrieved from http://www.informationr.net/ir/24-2/paper819.html (Archived by the Internet Archive at http://bit.ly/2MNTP22)
- Izuagbe, R., & Popoola, S. O. (2017). Social influence and cognitive instrumental factors as facilitators of perceived usefulness of electronic resources among library personnel in private universities in South-west, Nigeria. Library Review, 66 (8/9), 679–694. .
- Jaeger, P. T., & Burnett, G. (2010). Information worlds: social context, technology, and information behavior in the age of the Internet. New York: Routledge. .
- Jiang, J., He, D., Kelly, D., & Allan, J. (2017). Understanding ephemeral state of relevance. Proceedings of the 2017 Conference on Human Information Interaction and Retrieval, 137–146. .
- Joo, S., & Choi, N. (2016). Understanding users’ continuance intention to use online library resources based on an extended expectation-confirmation model. The Electronic Library, 34 (4), 554–571. .
- Kari, J. (2007). Conceptualizing the personal outcomes of information. Information Research, 12 (2), paper292. .
- Kegan, D. L. (1970). Measures of the usefulness of written technical information to chemical researchers. JASIS, 21 (3), 179–186. .
- Kemp, D. A. (1974). Relevance, pertinence and information system development. Information Storage and Retrieval, 10 (2), 37–47. .
- Koohang, A. A., & Byrd, D. M. (1987). A study of attitudes toward the usefulness of the library computer system and selected variables: A Further Study. Library and Information Science Research, 9 (2), 105–111. .
- Kwek, D. H. B. (2018). The importance of being useless: A cross-cultural contribution to the new materialisms from Zhuangzi. Theory, Culture & Society. .
- Letchumanan, M., & Muniandy, B. (2013). Migrating to e-book: a study on perceived usefulness and ease of use. Library Hi Tech News, 30 (7), 10–16. .
- Lu, H.-P., & Gustafson, D. H. (1994). An empirical study of perceived usefulness and perceived ease of use on computerized support system use over time. International Journal of Information Management, 14 (5), 317–329. .
- MacDonald, C. M., & Atwood, M. E. (2014). What does it mean for a system to be useful?: An exploratory study of usefulness. Proceedings of the 2014 Conference on Designing Interactive Systems, 885–894. .
- Nasser Al-Suqri, M. (2014). Perceived usefulness, perceived ease-of-use and faculty acceptance of electronic books: An empirical investigation of Sultan Qaboos University, Oman. Library Review, 63 (4/5), 276–294. .
- Nocera, J. A., Dunckley, L., & Sharp, H. (2007). An approach to the evaluation of usefulness as a social construct using technological frames. International Journal of Human-Computer Interaction, 22 (1–2), 153–172. .
- Redström, J. (2008). RE:Definitions of use. Design Studies, 29 (4), 410–423.
- Saracevic, T. (1975). Relevance: A review of and a framework for the thinking on the notion in information science. JASIS, 26 (6), 321–343. .
- Saracevic, T. (1996). Relevance reconsidered. In P. Ingwersen & N. O. Pors (Eds.), Information science: Integration in perspective. Proceedings of the second conference on conceptions of library and information science (CoLIS 2) (pp. 201–218). Copenhagen: Royal School of Librarianship. .
- Saracevic, T. (2007). Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance. Journal of the American Society for Information Science and Technology, 58 (13), 1915–1933.
- Saracevic, T. (2016). The Notion of Relevance in Information Science: Everybody knows what relevance is. But, what is it really? .
- Schütz, A. (1970). Reflections on the problem of relevance. New Haven, CT: Yale University Press. .
- Sin, S.-C. J., & Kim, K.-S. (2013). International students’ everyday life information seeking: The informational value of social networking sites. 35 (2), 107–116. .
- Smith, E. T. (2003). Assessing collection usefulness: An investigation of library ownership of the resources graduate students use. College & Research Libraries, 64 (5), 344–355. .
- Sperber, D., & Wilson, D. (2011). Relevance : communication and cognition (2nd ed.). Malden: Blackwell. .
- Tahamtan, I., Pajouhanfar, S., Sedghi, S., Azad, M., & Roudbari, M. (2017). Factors affecting smartphone adoption for accessing information in medical settings. Health Information and Librarianship Journal, 34 (2), 134–145. .
- Taylor, R. (1991). Information use environments. In B. Dervin & M. Voigt (Eds.), Progress in Communication Sciences (Vol. 10, pp. 217–255). Norwood, NJ: Ablex. .
- Terras, M. (2010). Should we just send a copy? Digitisation, usefulness and users. Art Libraries Journal, 35 (1), 22–27. .
- Thoburn, N. (2014). Commodities, collections and fetishes in the politics of objects. In P. Harvey, C. E. Casella, G. Evans, H. Knox, C. McLean, E. B. Silva, … K. Kath Woodward (Eds.), Objects and materials : a Routledge companion (pp. 208–217). .
- Tsakonas, G., & Papatheodorou, C. (2006). Analysing and evaluating usefulness and usability in electronic information services. Journal of Information Science, 32 (5), 400–419. .
- Tsakonas, Giannis, & Papatheodorou, C. (2008). Exploring usefulness and usability in the evaluation of open access digital libraries. Information Processing & Management, 44 (3), 1234–1250. .
- Tuominen, K., Talja, S., & Savolainen, R. (2002). Discourse, cognition, and reality: toward a social constructionist metatheory for library and information science. In H. Bruce, R. Fidel, P. Ingwersen, & P. Vakkari, H. Bruce, R. Fidel, P. Ingwersen, & P. Vakkari (Eds.), Emerging Frameworks and Methods: CoLIS 4 : Proceedings of the Fourth International Conference on Conceptions of Library and Information Science, Seattle, WA, USA, July 21-25, 2002 (pp. 271–283). Greenwood Village, CD: Libraries Unlimited. .
- Venkatesh, V., Thong, J. Y., & Xu, X. (2012). Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Quarterly, 36 (1), 157–178. .
- Vickery, Brian C. (1959). The structure of information retrieval systems. Proceedings of the International Conference on Scientific Information, 2, 1275–1290. Washington, DC: National Academy of Sciences. .
- Vickery, Brian Campbell. (1959). Subject analysis for information retrieval. Proceedings of the International Conference on Scientific Information, 2, 855–865. Washington, DC: National Academy of Sciences. .
- Williams, M., Pillon, K., Berg, S. A., & Johnston, D. J. (2015). Ease of use and usefulness as measures of student experience in a multi-platform e-textbook pilot. Library Hi Tech, 33 (1), 65–82. .
- Wilson, P. (1968). Two kinds of power : an essay on bibliographical control. Berkeley: University of California Press. .
- Wilson, P. (1983). Second-hand knowledge: An inquiry into cognitive authority. Westport, CN: Greenwood Press. .
- Wilson, T.D. (1981). On user studies and information needs. Journal of Documentation, 37 (1), 3–15. .
- Woolgar, S. (1990). Configuring the user: The case of usability trials. The Sociological Review, 38 (1_suppl), 58–99. .
- Yan, Y., Zha, X., & Xiao, Z. (2013). Exploring users’ perceptions of conventional and unconventional electronic resources. Serials Review, 39 (2), 105–113. .
- Yoon, H.-Y. (2016). User acceptance of mobile library applications in academic libraries: An application of the Technology Acceptance Model. The Journal of Academic Librarianship, 42 (6), 687–693. .
- Yoon, J., & Kim, S. (2014). Internet use by international graduate students in the USA seeking health information. Aslib Journal of Info Management, 66 (2), 117–133. .
- Zha, X., Li, J., & Yan, Y. (2013). Understanding the moderating effect of tie on the transfer of ease of use and usefulness from print resources to electronic resources. Library & Information Science Research, 35 (3), 223–231. .
- Zhang, J., Yan, Y., & Zha, X. (2014). Exploring the effect of individual differences on user perceptions of print and electronic resources. Library Hi Tech, 32(2), 346–367. .
- Zhang, Y. (2014). Beyond quality and accessibility: Source selection in consumer health information searching. Journal of the Association for Information Science and Technology, 65 (5), 911–927. .
- Zuiderent-Jerak, T., Strating, M., Nieboer, A., & Bal, R. (2009). Sociological refigurations of patient safety; ontologies of improvement and ‘acting with’ quality collaboratives in healthcare. Social Science & Medicine, 69(12), 1713–1721.