The problem of probability: an examination and refutation of Hjørland’s relevance equation

Introduction

Relevance is a key concept in information science and Information retrieval research. Many theories have been proposed regarding the nature of relevance, how this concept should be understood, defined, and measured. Some of the latest reviews on these topics include Borlund (2003), Hjørland (2010), and Saracevic (2007). One of these (Hjørland, 2010) also presents a proposal for an equation to determine the relevance of anything. Although this review has been cited about 50 times, no one has so far taken a critical look at the proposed equation. This is unfortunate since the new proposal is followed by a quite bold insistence from its creator (Professor Birger Hjørland) that the new equation is clear cut and thus solves the problems related to the conceptualization and measurement of relevance. To investigate whether this is really the case, the present paper presents a critical examination of Hjørland’s relevance equation.

Hjørland’s relevance equation

Hjørland first presented his relevance equation in a brief communication for JASIS&T co-authored with a former PhD-student:

Something (A) is relevant to a task (T) if it increases the likelihood of accomplishing the goal (G), which is implied by T(Hjørland and Sejer Christensen, 2002, p. 964).

That this equation is developed by Hjørland himself (and not by his former PhD-student) is made clear in a later, and more elaborated presentation in which Hjørland specifically states:

The relevance of knowledge depends on its usefulness to achieve specific goals. Based on this understanding, I provided [this] definition of relevance (Hjørland, 2010, p. 229).

Hjørland (2010, p. 229) maintains that his relevance equation "applies to anything", and that "A should be regarded as a tool in the broadest possible sense of this word, including ideas, texts, and things" (Hjørland and Sejer Christensen, 2002, p. 964). In the abstract of the same paper it is stated that "tool" should be "understood in the widest possible sense, including ideas, meanings, theories and documents" (Hjørland and Sejer Christensen, 2002, p. 960). This is a very wide ranging claim. Hjørland (2010, p. 229) nevertheless states his equation to be "clear cut". According to Hjørland (2010, p. 232) "a document or a piece of information is either relevant or is not relevant", and "may be relevant even if nobody (yet) thinks so (objective relevance). It may thus turn out to be relevant in the future".

Hjørland maintains to have developed the equation on his own, but admits that "by defining relevance in relation to tasks, I am in agreement with a version of what has been termed situational relevance, which was first formulated by Patrick Wilson" (Hjørland, 2010, p. 235). Here it is important to note that Hjørland specifically sees the relation between A, T and G to be a question of logic. Referring to Wilson (1973, p. 464), Hjørland (2010, p. 235) states that he agrees "that an item is situationally relevant is a logical, not a psychological, fact". When Wilson (1973) uses the term "logic" he refers to both deductive and inductive reasoning, including plausible and probabilistic reasoning (Huang and Soergel, 2006).

When Hjørland first presented his relevance equation, he stressed that the relation between A, T and G is a question of scientific consensus:

Theories, paradigms, and epistemologies form the basic socio-cultural environment in which the information seeking takes place, and they imply criteria for what information becomes relevant (Hjørland and Sejer Christensen, 2002, p. 960).

and:

The less consensus about fundamental theories and findings there is in a field, the less clarity and the more difficult is it to establish collectively accepted relevance criteria (Hjørland and Sejer Christensen, 2002, p. 963).

Instead of speculating on whether Hjørland (2010) really changed his mind in favour of "logic" or not, it is sufficient to note that he actually operates with two different warrants for the relevance relation that, for reasons of clarity, need to be taken into account separately when examining them further.

There is, however, an important aspect that needs to be dealt with first. Hjørland’s relevance equation is based on probability logic. It specifically states that A is relevant to T if it increases the likelihood of accomplishing G. In everyday language likelihood and probability are often treated as synonyms, but in statistical usage they are not:

Likelihood is the hypothetical probability that an event that has already occurred would yield a specific outcome. The concept differs from that of a probability in that a probability refers to the occurrence of future events, while a likelihood refers to past events with known outcomes (Weisstein, 2015).

Hjørland refers to the possibility of accomplishing a goal (G). This is a reference to a future event. Not a reference to a previous event with a known outcome. It would therefore have been more precise to use the term probability instead of likelihood (i.e. A is relevant to T if it increases the probability of accomplishing G). Regardless, founding a relevance equation on probability logic, as Hjørland does, has consequences.

Probability logic

Imagine a super-intelligent creature that knows the positions, velocities, and forces on all the particles in the universe at one time. Wouldn’t this knowledge enable that creature to know the universe for all times? The idea about such a creature stems from the French mathematician and astronomer Pierre-Simon Laplace (1749-1827) and is known as Laplace’s Demon:

We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at any given moment knew all of the forces that animate nature and the mutual positions of the beings that compose it, if this intellect were vast enough to submit the data to analysis, could condense into a single formula the movement of the greatest bodies of the universe and that of the lightest atom; for such an intellect nothing could be uncertain and the future just like the past would be present before its eyes (Laplace, [1825] 1995).

Such a creature is of course an illusion. Such foreknowledge is simply not possible. One problem is the storage of data. Another is the lessons learnt from quantum mechanics that micro-particles obey laws that make them indeterministic in character. Yet, playing along a little longer, it means that in a purely deterministic system all objective probabilities would have to have values of zero or one. Either they will occur or they will not occur. Laplace thus thought that probabilistic claims reflect human ignorance about which outcome is determined to occur. Living in a deterministic universe, his demon would not need probability. It would know for sure!

Thus, in cases where we do not possess foreknowledge about the outcome of a particular operation, it makes sense to operate with probability logic.

A dominant position in modern probability theory is that probability logic entirely belongs to deductive logic, and hence should not be concerned with inductive reasoning (e.g., Adams and Levine, 1975). For some, this may seem strange since introducing an element of probability into the standard definition of the principle of induction precisely appears to save it from the standard criticism. This, however, is not the case. The standard definition of the principle of induction goes:

If a large number of As have been observed under a wide variety of conditions, and if all those observed As without exception possessed the property B, then all As have the property B (Chalmers, 1996, p. 5).

The standard criticism against the principle of induction goes:

Inductive arguments are not logically valid arguments. It is not the case that, if the premises of an inductive inference are true, then the conclusion must be true. It is possible for the conclusion of an inductive argument to be false and for the premises to be true and yet for no contradiction to be involved (Chalmers, 1994, p, 14).

Trying to save the principle of induction by incorporation an element of probability could be done like this:

If a large number of As have been observed under a wide variety of conditions, and if all those observed As without exception have possessed the property B, then all As probably possess the property B (Chalmers, 1996, p. 17).

However, this modified version does not overcome the problem of induction. As explained by Chalmers (1996, p. 18), whatever the observational evidence, the probability of any universal statement is zero:

Any observational evidence will consist of a finite number of observation statements, whereas a universal statement makes claims about an infinite number of possible situations. The probability of the universal generalization being true is thus a finite number divided by an infinite number, which remains zero however much the finite number of observation statements constituting the evidence is increased.

Examining Hjørland’s relevance equation

Founding a definition of something on probability logic calls for a warrant for the assumption that the probability of something is determined (or, as in Hjørland’s case, is increased) by something else. Probability theorists have suggested a number of such possible warrants (Gillies, 2000; Mellor, 2005). However, none of these suggestions are without problems. The two main suggestions are the so-called logical theory and the subjective theory. As noted above, Hjørland at one point argues that the relation between A, T and G is a question of logic, and not a question of psychology. We shall therefore leave out the subjective theory of probability when we first examine his relevance equation based on logical warrants. When instead maintaining that scientific consensus is what warrants the assumption between A, T and G, Hjørland is actually (perhaps unknowingly) in agreement with an alternative probability theory known as intersubjective probability (Gillies, 2000). We shall therefore also examine this theory and its consequences for Hjørland’s relevance equation.

A question of logic?

Perhaps the most obvious problem with Hjørland’s relevance equation is his claim that it applies to anything (including documents, ideas, meanings, texts, theories, and things). As his equation operates with a notion of probability for goal accomplishment, it follows from basic probability theory that his equation only concerns non-deterministic conditions. It may seem that this only marginally limits the generalisability of his relevance equation. On the contrary, it limits the generalisability to a quite large extent. Hjørland only provides a few examples of relevance relations. However, none of them apply to his own relevance equation. Take, for instance, this example:

Imagine a patient suffering a well-defined disease such as scurvy caused by lack of vitamin C. The relevant medical treatment for him would be doses of tablets containing vitamin C (ascorbic acid). Other drugs, such as vitamin B, are non-relevant (Hjørland, 2010, p.227).

Hjørland’s relevance equation is based on probability causation, but the scurvy example is based on deterministic causation. Here the element of probability is replaced by certainty. Hjørland is Laplace’s Demon in a deterministic system. He knows for sure that scurvy is caused by lack of vitamin C, that the cure is ascorbic acid, and that all other drugs will fail to cure the patient. See also Hjørland (2007, p. 404): "Causal relation: A semantic relation in which A is the cause of B (e.g., a lack of vitamin C causes scurvy)" and Hjørland (2015, p. 2): "Causal relation, e.g. that scurvy is caused by a lack of vitamin C or that vitamin C cures scurvy". Consequently, when giving the scurvy example, Hjørland ignores his own relevance equation.

Sticking with Hjørland’s relevance equation, the scurvy example should rather look like this:

Ascorbic acid is a relevant drug to give a patient suffering from scurvy if it increases the probability of curing the patient from that disease.

Here we maintain the probability causation in accordance with the original formulation of the equation. But without the full insight of Laplace’s Demon, how do we then warrant the assumption that ascorbic acid increases the probability of curing scurvy? Arguing that we know from experience that many scurvy patients have been cured by this treatment, and thus that this experience raises the probability of treating the next scurvy patient by the same treatment, commits the fallacy of making universal generalisations from a finite number of observations. Actually if we reason like this, according to Hjørland’s relevance equation, ascorbic acid should be seen as a non-relevant treatment. Why? Because, as we learnt from Chalmers in the preceding section, dividing a finite number of observations with an infinite number equals zero. According to Hjørland’s relevance equation, something (A) is relevant to a task (T) if it increases the probability of accomplishing the goal (G), which is implied by T. It follows logically that as the probability of treatment would be zero, probability of treatment is not increased, and the treatment is thus not relevant.

Intuitively, this does not make sense. Yet, it is a consequence of logic, and it is not us, but Hjørland who (at one point) insists that relevance is a question of logic. Admittedly, Hjørland does not exactly propose that probability assumptions should be based on prior knowledge – but he does come close. When discussing how his view of relevance relates to the IR situation he provides a telling example about the difference between experts and ordinary users:

When searching databases, [experts] search for clues (or "signs") that are likely to reveal something about the trustworthiness of the documents, including methodological terms, esteemed authors and publishers, and much more. "The ordinary users", on the other hand, are less aware of such methodological issues and therefore tend to select documents from more superficial characteristics or clues, e.g., the newest ones, the more popular ones, the cheapest ones, etc. (Hjørland, 2010, p. 230).

What distinguishes the two groups is prior knowledge about "methodological issues". According to Hjørland, because the experts possess such knowledge, they are more capable of selecting trustworthy documents when searching databases. Intuitively, this sounds right. However, when applied to Hjørland’s relevance equation, the probability of selecting a trustworthy document from an infinite number of documents based on prior document knowledge is again zero.

But wait! Is it really fair to calculate like this? Searching a database is not the same as searching documents from an infinite pool of documents. Databases only cover a finite number of documents. Thus, dividing any finite number with any finite number provides a result larger than zero. Doesn’t that save Hjørland’s relevance equation? No. It just introduces another insurmountable problem: Everything suddenly becomes relevant. Why? Imagine a lottery with 1,000 tickets. One of the tickets gives a prize, the rest do not. Given that the goal is to win the prize, but you only have one pick. Which ticket should you pick? In other words: Which is the relevant one? Again, for Laplace’s Demon this would be an easy task. The demon would simply pick the winning ticket because it knew which one it was. For the rest of us, our human ignorance necessitates the introduction of probability logic, which tells us that each ticket has the probability of 1/1,000 for winning the prize. Trying to figure out which ticket is relevant using Hjørland’s equation would look like this:

A ticket (A) is relevant to pick (T) if it increases the probability of winning the price (G)

Since all tickets have the probability of 1/1,000 for winning the prize, picking any of them compared to not picking any of them raises the probability of winning the prize from zero to 1/1,000. Thus, because the probability increases, any of the 1,000 tickets are relevant to pick according to Hjørland’s equation.

This compares to searching a database for documents to fulfil an information need. Given the assumption that there is (at least) one relevant document in the database, but we don’t know which one it is, the probability of picking it by a random pick is larger than zero. It consequently follows from Hjørland’s equation that all of the documents in the database are relevant.

Imagine another lottery with 1,000 tickets. 500 are red. 500 are green. Only one of the red tickets gives a prize, but 400 of the green tickets give a prize. Given the goal is again to win a prize, but you only have one pick. Which ticket should you now pick? The rational choice seems to be picking one of the green tickets since the chance of winning is 4/5 compared to picking a red ticket with the implied chance of 1/500. According to Hjørland’s equation, it would be equally relevant to pick either a red or a green ticket. This is due to the fact that Hjørland operates with a binary relevance concept. Either the ticket is relevant or not. What makes it relevant is the fact that any pick increases the probability of winning compared to not picking a ticket. The actual increase is not taken into account.

This compares to Hjørland’s example of the experts using clues or signs when seeking for trustworthy documents in databases. Suppose an expert has learnt that "esteemed publisher" is a highly dependable sign of trustworthiness compared to "costly", which is only marginally dependable as a sign of trustworthiness. Given that the goal is to retrieve trustworthy documents, which of the two search strategies is then relevant to employ? Again, the rational choice seems to be to search for documents by esteemed publishers as it increases the probability of honouring the goal. Though only marginally, seeking for documents that are expensive logically increases the probability of finding trustworthy documents. Thus, according to Hjørland’s equation, both search strategies are equally relevant.

So far we have based our understanding regarding an increase of probability on whether or not the increase exceeds zero. What if we instead understood the increase of probability relative to all other probabilities? Wouldn’t that save Hjørland’s relevance equation? For instance, in the example regarding the 1,000 red and green tickets we could say that picking a green ticket increases the probability of winning compared to picking a red ticket. Thus, it would only be relevant to pick one of the 500 green tickets. Yet, there are at least three practical problems with this interpretation. First, it introduces an element of Laplace’s Demon: We would need to know the probabilities of almost everything in order to do the relative probability calculation. Secondly, it reduces the number of relevant choices to one. Only the most probable choice would be relevant. Thirdly, it actually breaks with a crucial component of Hjørland’s relevance theory, namely his insistence that something is relevant even if it is not (yet) perceived as such. Although one of the red tickets is a winner, using relative probability makes only the 500 green tickets relevant.

Intersubjective probability

In probability theory there is a sharp polarization between the logical theory and the subjective theory (Gillies, 2000). Trying to moderate between the two, Gillies (1991) introduced an alternative theory of probability, which he termed intersubjective probability. Instead of seeking warrant for the probability assumption by logic or subjective belief, he proposes that warrant should be sought in the social group of shared scientific beliefs. His idea is that when general consensus is established, warrants for probability assumptions follow:

The intersubjective probabilities in confirmation theory should then apparently be the consensus probabilities of all the scientists working in the branch of science in question (Gillies, 1991, p. 529).

However, the warrant only applies if there is complete consensus in the social group. This is stressed by Gillies (1991) who points to rebels, dissidents, and conflicting schools of thoughts as entities challenging his theory. The first two entities are quickly written off. Although he acknowledges them as valuable for the scientific community, he sees them as isolated, and argues that "the appropriate interpretation of probability is clearly the subjective one" (Gillies, 1991, p. 529). Referring to Kuhn (1962) and Lakatos (1970) he then proceeds with the third entity claiming that:

As far as intersubjective probabilities are concerned, these can sometimes be taken as the consensus probabilities of the whole relevant scientific collective [P(e,h&k), P(e,k)], while in other contexts, they are only the consensus probabilities of one of the schools of thought within the collective [P(h,k)]. (Gillies, 1991, p. 528).

He then repeats that his theory of intersubjective probability only applies when consensus is complete, and doubts whether his theory can be applied when different schools of thoughts exist:

There are some doubts as to whether the probabilities P(h,k) can be introduced in any sensible fashion, since there are arguments which suggests the P(h,k) must always be zero for any universal hypothesis
We want, as far as possible, to ensure that our confirmation function is based on intersubjective probabilities which are consensus probabilities of the whole relevant scientific thought collective. This means that we should try to confine ourselves to probabilities like P(e,h&k) and P(e,k), and try to avoid the prior probabilities P(h,k) of the Baysians (Gillies, 1991, p. 529).

What are the consequences for Hjørland’s relevance equation? Returning to the scurvy example, and granting that there apparently is complete consensus among contemporary scientists that vitamin C cures scurvy, seems to save his relevance equation when basing the probability assumption on intersubjective probability. This is, however, too hasty a conclusion. Recall that Hjørland claims that his relevance equation applies to anything (including documents, ideas, meanings, texts, theories, and things). Based on intersubjective probability, his relevance equation is only valid if there is complete consensus among scientists on such matters. But how frequent is complete consensus among scientists? As noted above, Gillies (1991) relies on Kuhn (1962) when claiming that intersubjective probabilities can sometimes be taken as consensus probabilities of the whole scientific community. This ignores the widespread critique that has been raised against Kuhn’s paradigm theory by many historians and philosophers of science. A number of critics have praised Kuhn’s book as a very important contribution to the philosophy of science, but have pointed out that Kuhn’s description of science suffers from some acute conceptual and empirical difficulties (e.g. Shapere, 1964). Feyerabend (1970), Lakatos (1970, 1978) and Laudan (1977) have furthermore stressed the historical incorrectness of Kuhn’s notion that normal science is characterized by periods of the sole existence of one dominating paradigm. Instead they maintain that every major period in the history of science is characterized by the co-existence of competing research programmes or research traditions. Moreover, some sociologists of science have questioned whether everyday-science is characterized by consensus. Cole (1992, p. 15) makes a distinction between the research frontier and the core:

The core consists of a small set of theories, analytic techniques, and facts which represent the given at any particular point in time.
The research frontier is where all new knowledge is produced".

Cole argues that it is necessary to make this distinction because the social character of knowledge differs dramatically between the core and the frontier. "By definition there is a high level of agreement on core knowledge" (Cole, 1992, p. 19). Contrary, "the level of cognitive consensus at the frontier is relatively low in all scientific fields" (Cole, 1992, p. 19). As noted by Cole (1992) himself, this position is at odds with positivists, Kuhn, and most traditional sociologists of science. Hargens is one of the sociologists of science who have gone against Cole on this topic. Over the years, he has published a number of empirical studies dealing with the question about consensus levels at the research frontier (e.g., Hargens and Hagstrom, 1982; Hargens, 1988; 2000). Hargens distinguishes between foundational and current scholarship, and believes contrary to Cole that the level of consensus in current scholarship differs from discipline to discipline. In some disciplines the level of consensus in current scholarship is relatively high. In other disciplines the level is relatively low. The results of his reference network analysis (Hargens, 2000) seem to confirm this (see also Nicolaisen and Frandsen, 2012). Although Cole and Hargens disagree on the exact level of consensus at the research frontier/current scholarship, none of them accept the Kuhnian inspired notion of complete consensus.

Taking these objections into account, if not exactly falsifying Gillies’ (1991) theory on intersubjective probability, they at least attest that his theory has only very limited generalisability. When Hjørland maintains that scientific consensus is what warrants the probability assumption between A, T and G, his relevance equation suffers the same limited generalisability.

Does this mean that if one maintains, for instance, that relevance criteria are in fact formed in discourse communities, then ones’ argument has limited generalizability? No, not at all. I myself take this standpoint and have defended it several times (e.g., Nicolaisen, 2003; 2004; 2007). What limits the generalizability of Hjørland’s relevance equation is the introduction of probability logic when arguing that something is relevant if it increases the probability for something else. It calls for a warrant for the probability assumption, and if scientific consensus is used as warrant, then the relevance equation only applies when there is complete consensus, and this is rarely the case. Hjørland is of course quite aware of the varying consensus levels in science. The problem is just that when claiming that his relevance equation applies to anything (including documents, ideas, meanings, texts, theories, and things) he is making a universal statement that neglects this, as it calls for complete consensus (Gillies, 1991) in order to work as a warrant for the probability assumption.

Conclusion

When Wilson (1973) based his relevance theory on deductive and inductive reasoning, including plausible and probabilistic reasoning, he made a very important reservation:

Whether or not plausible inference ever comes to be as well understood as deductive inference, and whether or not it is generally accepted that the conclusions of non-demonstrative arguments hold with degrees of probability, we do now operate, and shall undoubtedly continue to operate with a concept of the evidential or "support" relationship to which one sense of relevance is firmly tied. The notion of evidential relevance is no clearer than the notion of the degree of confirmation of conclusions on given premises, but vague as it is [my highlight], we have it and use it (Wilson, 1973, p. 460).

Hjørland does not make the same reservation. On the contrary, he maintains that his relevance equation is "clear cut" (Hjørland, 2010, p. 229). Hjørland’s relevance equation states that something (A) is relevant to a task (T) if it increases the probability of accomplishing the goal (G), which is implied by T. Yet, taking the word of Hjørland, and thus applying his "clear cut" equation on various examples (including some of his own), we have seen that no matter what kind of logic (inductive or deductive) we applied to warrant the probability assumption of Hjørland’s equation, the outcome of using it to determine the relevance of any A to any T produced quite bizarre consequences: either nothing became relevant or everything became relevant. Using scientific consensus (the intersubjective theory) to warrant the probability assumption was shown to severely limit the generalisability of his equation. Thus, contrary to Hjørland’s claim that his relevance equation applies to anything (including documents, ideas, meanings, texts, theories, and things), it is found at best to have very limited generalisability.

Admittedly, this is a rather harsh conclusion, and one that some might accuse of being based on a straw man. After all, when Hjørland provided his relevance equation, and stated that it applies to anything, and that it is clear cut, he later continued:

What in practice causes problems are theoretical disagreements on what the goals are, on what criteria for good solutions are, and on what methods are available. Goals and problems may be differently conceptualized and connected to different world views and epistemologies (Hjørland, 2010, p. 229).

Doesn’t that quote eradicate all the critique raised in this paper? Not quite. First of all, it does not address the problem of establishing a logical warrant for the probability assumption. Secondly, it is a logical fallacy to maintain that a universal generalisation has limited generalisability, or that it only applies under very special conditions. In the words of Francis Lieber in his Notes on Fallacies of American Protectionists:

That which is not true in practice is not true at all. Let us hear no more about being true in theory but not in practice (Lieber, 1870, p. 15).

Some might think that I am reading Hjørland too literally. This was actually what one of the reviewers commented:

It is surely probable that (notwithstanding the "equation's" pseudo-algebraic form) it was never Hjørland's intention for his equation to be taken seriously as a logically watertight formula that could be used to calculate or measure precise degrees of relevance, but rather as an informal, operational definition of a much-discussed concept that would allow for binary judgments (relevant vs. non-relevant). In particular, I find it difficult to imagine that Hjørland ever would have wanted his formula to be interpreted as an "if ... then ..." statement, but rather as a simple "x = ..." statement of equivalence, where the goal is to specify the meaning of the term x. That he says that statements of the form "this item is situationally relevant" are logical rather than psychological has no bearing on this matter: he could just as well have used the terms "objective" and "subjective" instead of "logical" and "psychological.

This, however, is also a logical fallacy. It is known as wishful thinking. It occurs when an appeal is made to whatever is pleasing to imagine rather than dealing with reality. Despite what the reviewer finds pleasing to imagine, reality is that Hjørland has proposed an equation for the assessment of the relevance of anything, that this equation is not a simple x = … statement, but an if … then statement, and that Hjørland (at one point) makes perfectly clear that the relation between A, T and G is a question of logic. Moreover, if the reviewer knew Hjørland better, s/he would know that Hjørland definitely wants to be taken literally on this. Hjørland knows the importance of formulating theoretical arguments with great precision, and takes pride in doing so himself. See for example his heated discussion with Professor Marcia Bates about theoretical clarity where Hjørland (2011, p. 546-547) clearly states his position on this:

A theory should be formulated in a strict way that allows it to be falsified. Unclear theories are bad science because there are no defined criteria by which they can be tested.
We should try to formulate our scientific views in ways that specify which established theories they are in opposition to.
It is correct that I try to formulate theoretical positions in a clear way.

Finally, some might think that rather than criticising Hjørland’s relevance equation, the critic should instead develop a better relevance equation. This was actually the suggestion of the same reviewer who commented:

My suggestion would be, at the very least, to identify the respect(s) in which Hjørland's formula fails to capture the particular sense of relevance that the author has in mind, and to offer an alternative formula that, in contrast, succeeds.

This, however, is yet another logical fallacy. This one even has a Latin name: Onus probandi incumbit ei qui dicit, non ei qui negat (the burden of proof is on the person who makes the claim, not on the person who denies (or questions) the claim). Recall that it is Hjørland who maintains that it is possible to construct an equation that can be used for assessing the relevance of anything. Thus, the burden of proof is on Hjørland, not on those who may criticise and question his equation. I have criticised and questioned Hjørland’s relevance equation. My burden of proof is thus not to offer an alternative and successful equation, but instead to offer solid arguments and well-founded evidence for my own critique.

About the author

Jeppe Nicolaisen is Associate Professor at the Royal School of Library and Information Science at University of Copenhagen. He defended his PhD on citation theory in 2004 (Professor Birger Hjørland served as his supervisor). His main research interests lie within the fields of bibliometrics and science studies.