Appendix 7:
Paper presented at ISIC3, 2000 —
Uncertainty and its correlates

T.D. Wilson, N.J. Ford, D. Ellis*, A.E. Foster
Department of Information Studies, University of Sheffield, UK

A. Spink
School of Information Science and Technology,
Pennsylvania State University, USA

ABSTRACT

The Uncertainty in Information Seeking and Searching project was carried out in 1998 and 1999 at the Universities of Sheffield and North Texas. Its aim was to test the concept of uncertainty and to explore the relationship between uncertainty and other variables in the information seeking and searching context. Reference interviews were held with 198 clients for whom on-line searches were then performed, data being collected at that stage and also, for the Sheffield clients, in a follow-up stage two months later. This paper reports on the relationships between uncertainty and a variety of other variables, including the problem-stages at which the clients found themselves, Kuhlthau's information seeking stages, and Ellis's information seeking characteristics.

INTRODUCTION

The idea of uncertainty underlies all aspects of information seeking and searching, from notions of fuzzy decision theory to Belkin's (1980) idea of the Anomalous State of Knowledge of the information seeker. Relevance, for example, can be perceived as a surrogate for uncertainty, in that an item of information can be perceived as relevant if there is some probability that its use will remove some of the information seeker's uncertainty. How the idea of uncertainty relates to this and to other aspects of information seeking and searching, however, is not well-researched.

One of the main aims of the 'Uncertainty Project', which has involved collaboration between teams at the University of Sheffield and (originally) the University of North Texas, has been to operationalise the concept of uncertainty and to relate it to other aspects of the information seeking and searching process.

The project from which the data are derived was undertaken at the University of Sheffield between October 1997 and September 1999, and at the University of North Texas over a period of eighteen months in the same period. Common research instruments were used, although there were some differences in the way the data were collected (e.g., in Sheffield the client attended the search sessions) and, for that reason, the description that follows refers to the procedures adopted in Sheffield.

THEORETICAL FRAMEWORK

Wilson's (1999) problem-solving model is used as the top-level concept to explain why people engage in information seeking. This model suggests that information-seeking behaviour (ISB) is goal-directed behaviour with the resolution of the problem and, possibly, the presentation of the solution as the goal. In moving through each of the stages of problem identification, problem definition, problem resolution and solution presentation, uncertainty must be reduced and individuals are seen as engaging in interaction episodes with information sources (including people and other sources, as well as information retrieval systems) to resolve their uncertainty. Of course, attempts to resolve uncertainty may actually increase it and, therefore, the model provides for feedback at each stage. The idea of stages, together with feedback, explains the phenomenon of successive searching (see Spink, 1996), where information seekers are seen to carry out more than one search for information relating to their problem.

Within this general model, Kuhlthau's (1994) model of the stages in information seeking is specifying the stages an information seeker may proceed through in any one search activity, while Ellis's (1989) search 'characteristics' are seen as applying to either searching during the different problem stages or during any one of Kuhlthau's stages.

The framework, therefore, seeks to bring together a number of different ideas from ISB research.

RESEARCH QUESTIONS AND HYPOTHESES

The aims of the research can be set out under these two headings. From the perspective of this paper, the theoretical frameworks under test gave rise to two research questions:

• Is the problem-solving stage model recognised by clients as appropriate for recording their progress on a project?

The problem-solving stage model had been suggested by Wilson (2) as a generalised basis for explaining the fact of successive searches by clients dealing with a particular issue, and it was of interest to the research team to determine whether it offered a way of categorising clients that would provide a useful analytical variable.

• Is the concept of uncertainty recognised by clients? Can they use a presented scale to indicate how certain/uncertain they are about their problem stage and about the availability of information to assist them in solving their problem?

Similarly, Wilson had also suggested that the concept of 'uncertainty' could be useful in categorising the cognitive state of the client and that, if it could be operationalised, it might provide a useful analytical variable. It was important, therefore, to discover whether these concepts could be operationalised and then, by testing them as part of a number of hypotheses, determine whether they provided useful analytical variables.

Hypotheses

Given the complexity of the research design and the number of issues that could be explored, the number of variables to be tested is too high to be presented in a conference paper of limited length. We have made a selection, therefore, of some of the hypotheses (presented below in null form) related to the concepts of uncertainty and the problem-solving stage model. The intention was to test relationships that would help to identify useful aspects of the various models, particularly those of Kuhlthau and Ellis, in the contexts of problem solving and uncertainty. In addition, various demographic characteristics were explored to discover whether the information-seeking process (in the case of these clients) was affected by such characteristics and whether the problem-solving stage model and the concept of uncertainty held for clients in different countries, of different ages and sex, and in different disciplines.

• Hypothesis 1: There is no significant difference between disciplines as to the problem-solving stage identified by clients.
• Hypothesis 2: there is no difference in the state of uncertainty expressed at the different problem stages.
• Hypothesis 3: there is no difference in the overall level of uncertainty expressed by the different sexes.
• Hypothesis 4: there is no difference in the overall level of uncertainty expressed by clients in different age groups.
• Hypothesis 5: there is no difference in the level of uncertainty expressed in relation to the problem stages by clients in different disciplines.
• Hypothesis 6: there is no difference in the level of uncertainty expressed by clients before and after the on-line search.
• Hypothesis 7: there is no difference in the overall level of uncertainty expressed by clients with different levels of knowledge of their domain.
• Hypothesis 8: there is no difference in the feelings expressed by clients having different levels of uncertainty.
• Hypothesis 9: there is no difference in the overall level of uncertainty expressed by clients engaged at different stages of the Kuhlthau model.
• Hypothesis 10: there is no difference in the overall level of uncertainty expressed by clients engaged in different activities as expressed in Ellis's behavioural model.

DATA COLLECTION

Data were collected in three stages:

• pre-search interview: during which a detailed description of the client's problem was obtained, together with responses to interview questions and responses to a questionnaire, which covered, for example, problem stage, Kuhlthau's stages, feelings about the progress of the work, other information seeking activities, and uncertainty.
• on-line search and post-search interview: Immediately before the search, the client completed a test mounted on the PC, which automatically recorded various dimensions of cognitive style. During the search, computer logs were kept, together with audio-tapes of the interaction between client and searcher. After the search the client completed another questionnaire on aspects of the search and, again, on their certainty/uncertainty with regard to different stages of problem resolution. The searcher also completed a search assessment instrument.
• follow-up interview: conducted a minimum of two months after the search seeking an evaluation of the retrieved material, the client's present position in the problem solving process, state of uncertainty, feelings about the project or problem, etc.

Very similar information was collected at North Texas and there was enough commonality for the North Texas cases to be included in the overall analysis.

Data were collected on a total of 198 cases: 87 at North Texas and 111 at Sheffield. The demographic characteristics of the two sets of cases were as follows:

• the age range was very similar in both UNT and Sheffield, as shown in Table 1. UNT had a slightly higher proportion of clients aged under 30, and a slightly higher proportion of clients in the 40 to 49 group, while at Sheffield a higher proportion was in the 50 to 59 age group. However, the differences are very small and not statistically significant. (Chi-squared = 3.08, sig. >.05)

• the sex distribution, however, was rather different, as shown in Table 2. The UNT sample had a much higher proportion of female clients than that of the Sheffield sample. Again, however, the difference is not statistically significant (Chi-squared = 3.08, sig. >.05)

• clients were classified by broad discipline, i.e., humanities; 'pure' social sciences, such as economics, political science, sociology, etc.; applied social sciences, such as social welfare and social administration; pure science; medicine; and engineering. The numbers of humanities and medical clients were rather small and the former were incorporated into the pure social sciences group, while the latter were included in the pure science group. This gave four discipline categories. The UK and US clients were distributed over these four categories as shown in Table 3. The differences are significant, Chi-squared = 17.036, sig. at .01 level.

RESULTS

Problem-solving stage

Clients were asked "What stage are you at in terms of defining or resolving the problem, or in presenting the answer?" and given a list of the problem solving stages with definitions (Appendix 1).Very few clients had any difficulty in fitting the progress of their work into one or other of the stages offered. All were given the opportunity to select an alternative position on the scale - generally between two of the stages - and only 11 did so. The results are shown in Table 4 below. As may be seen from the table, and not surprisingly (since we might expect the greatest intensity of information seeking to occur at these stages), the majority of clients located themselves in either the problem definition stage or the problem resolution stage.

The problem stage can also be used to show the progress of clients as they move through the information seeking and use process. Figure 1 shows the pre-search and follow-up stages of the clients who were involved in the study at both of these phases of the research. The anticipated shifts are shown to take place, that is, fewer clients were in the early stages and more were in the later at the time of the follow-up.

Uncertainty

Again, the vast majority of clients were able to use the scale presented to identify the state of their certainty/uncertainty for each stage of the problem-solving process and for the likely availability of information sources (see Appendix 1). For example, Table 5 shows the distribution of results for uncertainty about the definition of the problem. Appendix 1 shows the questions used to elicit certainty/uncertainty.

Hypothesis 1

• There is no significant difference between disciplines as to the problem-solving stage identified by clients.

This hypothesis was supported by the data: in a cross-tabulation of problem stage against discipline, the chi-squared value was 5.7, which was not significant. The four discipline groups mentioned above were used and the only difference found, which was slight, was that more applied social scientists identified themselves as being in the problem resolution stage, whereas the largest category for the rest was problem definition.

Hypothesis 2

• There is no difference in the state of uncertainty expressed at the different problem stages

This hypothesis proved difficult to test because of the highly skewed nature of the distribution - the greater majority of people were relatively certain about the progress of their work. However, when the mean value of the median uncertainty scores of people in different stages of the problem-solving process were calculated, the result shown in Figure 2 was obtained.

References

• BELKIN, N.J.(1980) Anomalous states of knowledge as a basis for information retrieval. Canadian Journal of Information Science, 5, 133-143
• WILSON, T.D. (1999) Models in information behaviour research. Journal of Documentation, 55 (3), 249-270
• SPINK, A. (1996) A multiple search session model of end-user behavior: an exploratory study. Journal of the American Society for Information Science, 46, 603-609
• KUHLTHAU, C.C. (1994) Seeking meaning: a process approach to library and information services. Norwood, NJ: Ablex.
• ELLIS, D. (1989) A behavioural approach to information retrieval design. Journal of Documentation, 46, 318-338.
• WILSON, T.D. (1981) On user studies and information needs. Journal of Documentation, 37, 3-15.

* David Ellis is now Professor, University of Wales, Aberystwyth