vol. 13 no. 4, December, 2008
vol. 13 no. 4, December, 2008 | ||||
Exploratory search processes, in which searchers look for information on the Web to solve their own problems, represent a type of e-learning generated and controlled by the searchers themselves. Current research suggests that exploratory searchers modify their knowledge by gradually acquiring information and using the modified knowledge to explore problems further until they either solve or abandon them (Bates 1989; Marchionini 2006; Sihvonen and Vakkari 2004). To date, a combination of observing (video-recorded) search processes, a think-aloud protocol and post-search interviewing has been used to reconstruct, in as much detail as possible, searchers' internal micro-level processes, with limited success. This strategy was used mainly because most Web search processes are embedded deeply in everyday activities. Often, searchers have difficulty recalling and articulating what is happening or has happened in their mind while looking for information and building knowledge (Miwa and Kando 2007).
In contrast to previous research, this study used a multi-method approach to capture observable search behaviour as well as the searchers' internal processes. Specifically, we introduced the concept of a view as the unit of analysis, using an eye-tracker to capture searchers' eye movements. We also showed searchers their video-recorded eye movements to stimulate their recollection of cognitive and affective states upon which their behaviour was based and to get reliable data about their internal processes. This information was used to make sense of the searchers' observable behaviour during the search processes. Consequently, the aim of this study was to describe, in as much detail as possible, exploratory Web-search processes by acquiring patterns of knowledge acquisition and modification, in order to develop models.
In the following, exploratory search processes on the Web and a view are defined as units of analysis for browsing behaviour. This was done to relate this study to existing studies and to describe methodological improvements for capturing micro-level processes and for expanding our understanding of searchers' knowledge-building behaviour.
The concept of exploratory search processes is based on three research traditions: self-regulated learning, interactive information retrieval and information-seeking behaviour. Self-regulated learning shows how active, self-regulated and motivated learners engage cognitively in learning tasks (Corno and Mandinach 1983). Interactive information retrieval demonstrates how people's information needs are developed while they are interacting with a variety of information resources during information retrieval (Vakkari and Järvelin 2005). Information seeking behaviour shows how an individual's knowledge of everyday life can be modified by interactions with literature, media and social environments (Bates 1979, 1989; Todd 1999).
Self-regulated learning refers to learning that is influenced mostly by the learner' self-generated thoughts, feelings, strategies and behaviour, which are oriented towards the attainment of goals (Zimmerman and Campillo 2003). Corno and Mandinach (1983: 95) argued that self-regulated learning deepened and modified the associative network in a particular area of the brain, as well as monitoring and improving this deepening process. Their self-regulated learning model identified the two all-encompassing processes of acquisition and transformation. These involved five embedded strategies: alertness (receiving incoming stimuli; tracking and gathering information), selectivity (discriminating between stimuli; distinguishing relevant from irrelevant information), connecting (searching for familiar knowledge; linking familiar knowledge to incoming information), planning (organizing how to perform tasks and in what order) and monitoring (continual tracking of stimuli and transformations; researching; planning; self-checking). Rogers and Swan (2004) explored the application of Corno and Mandinach's model to Internet searching and found that the model's components described and predicted the behaviour of searchers engaged in learning from Internet resources. Thus, the self-regulated learning framework appears adequate for describing how people learn while searching for information on the Internet.
Researchers with cognitive viewpoints who assume a dynamic basis for information acquisition have proposed that information retrieval involves communication processes in which the users' pre-existing vocabulary is the key to effective interaction with search engines (Belkin 1993; Ingwersen 1996). By comparing thesaurus terms used by novices and experts in a given domain, Sihvonen and Vakkari (2004) concluded that only those users with sufficient domain knowledge might improve their search results by using thesaurus terms in query expansion.
The increasing availability and accessibility of Web resources for the general public allow domain novices a greater opportunity to interact with document collections. In response to such trends, the main role of information retrieval seems to be shifting from finding a particular information source to enabling ad hoc information lookup as required for learning during general browsing and exploration (Marchionini 2006). This transition is reflected in navigation tools that allow for the seamless combination of browsing and searching functions. For example, the Flamenco system allows searchers to browse a large collection of architectural images using hierarchically faceted metadata (Hearst et al. 2002). JINII Plus, an information-access interface for the shared portal of the Japanese academic community, aims to provide seamless switching between searching and browsing (Kando et al. 2006). These trends suggest a need for further research into how Web users learn and investigate while searching and browsing.
Bates (1979) identified twenty-nine search tactics within the four categories of monitoring, file structure, search formulation and term, which can be used for online and manual bibliographic searching. Bates (1989) described the evolving nature of information-seeking processes in her berrypicking model, which uses the analogy of picking hackberries in a forest. According to this model, searchers initiate their information search by focusing on one aspect of a broader question. Each new piece of information they encounter gives them new ideas and directions to further the search process and, consequently, continue generating new questions and queries. Searchers may gradually identify useful information at each stage of the process, rather than obtaining all the necessary information from a single information source. In this process, the query is satisfied not by a single piece of information but by accumulated information obtained at every stage of a continually modified search process.
Based on the berrypicking model, Bates suggested a variety of design features for information retrieval systems. Using data from a study of the information-seeking behaviour of secondary school students who were writing papers for a class assignment, Kuhlthau (2004) developed the Information Search Process model. Based on Kelly's (1963) personal construct theory, this model depicts the constructive nature of students' information seeking processes.
The Information Search Process model contains six stages:
Kuhlthau's model has been tested in different contexts, and the author developed the concept of a zone of intervention for identifying the role of teachers and librarians in students' information search processes. This concept is based on Vigotsky's (1978) zone of proximal development. These two models not only describe the evolving and constructive nature of information search processes, but also suggest that people learn gradually while they look for information from a variety of sources.
The findings from three research traditions, self-regulated learning, interactive information retrieval and information search processes, must be combined to develop an integrated conceptual model for designing an information-access interface to support learning and investigation or exploratory searching.
Based on these ideas, the aim of this study was to explore, at a micro-level, how students modify and use knowledge while browsing and interacting with Web resources for career planning. In this study, exploratory search processes on the Web is defined as a type of e-learning in which searchers look for information to solve problems that are generated and controlled by the learners themselves. The main focus of the study was to develop a framework or taxonomy to capture how searchers acquire, modify and use their knowledge of exploratory search processes on the Web.
Traditionally, most research tends to distinguish work-task knowledge (domain knowledge) from search-task knowledge. By contrast, this study investigated both types of knowledge, as articulated by searchers, in order to provide a holistic view of knowledge acquisition, modification and use during Web searching. We used the concept of a view to develop the methodology for this study. Kwasnik (1992) defined a view as 'what a person articulates as seeing at one time (in his/her physical as well as cognitive world), that is, a span of attention' and proposed using it as a unit of analysis in browsing. By expanding physical eye movements into the internal states of cognitive and affective worlds, a view can represent a synthesized segment of combined external and internal behaviour at each point in an exploratory search process. We therefore introduce the concept of a view as the unit of analysis for the holistic capture and analysis of exploratory search processes on the Web.
We recruited fourteen students (twelve undergraduates and two graduates; eleven females and three males) to participate in this study using a Website that offered part-time employment. Each student was given a time-based payment for participating in a series of data collection activities, described in Figure 1 as pre-search activity, recording exploratory search processes and post-search activity.
After being invited to attend the laboratory individually, each participant completed a short questionnaire requesting information on the frequency of his or her Internet use, favourite browsers and search engines, current stage and future prospects in career planning, and demographic information.
An in-depth interview of thirty to forty minutes was conducted to obtain contextual information for interpreting the participant's exploratory search processes on the Web. Specifically, the interview sought information on the participant's personal characteristics, family structure and family history focusing on job hunting and career development, plus their views on occupational life, their work and job-hunting experiences and whether they desired to attend graduate school.
The participant conducted a Web search on topics of their choice within the scope of career planning (e.g., job hunting and/or choosing graduate schools) for approximately twenty minutes, using a familiar browser and search engine. After completing the initial Web search, participants were allowed to conduct additional Web searches on whatever topic they wished.
Screen shots and mouse movements during exploratory search processes were captured using HyperCam (a screen-capturing software package). The participants' eye movements were measured using the EMR-NL8B eye-tracker, which required the participants to rest their jaws on stands so that no head movement could occur.
The researcher stood beside the participant and observed the exploratory search processes while watching the participant's eye movements and recording their activity on the timeline memo, in preparation for the post-search interview.
After completing the exploratory search processes session on the Web, the participant was interviewed. During the interview, we showed each participant their video-recorded eye movements superimposed on the screen shifts to stimulate recall of their cognitive and affective states. In this way, we captured the participant's view at each observable point in their behaviour. Specifically, we stopped the video when each move occurred (typing, clicking, browsing, copying, pasting, bookmarking, closing windows and stopping) and asked for the participant's reasons for the move as well as any accompanying thoughts and feelings they had at that moment.
The pre-search and post-search interviews were voice-recorded and fully transcribed. The transcribed interview data were coded for patterns of knowledge acquisition and utilization, using ATLAS.ti software for qualitative data analysis, following a bottom-up strategy using the constant comparative technique. A timeline matrix was developed for each participant's exploratory search processes, in which post-search articulation, knowledge acquisition and knowledge utilization at each move were arranged in chronological order to facilitate the identification of patterns in knowledge modification and utilization. Using this analysis, the taxonomy of knowledge modification (Miwa and Kando 2007) was expanded and a new taxonomy of knowledge utilization developed.
Table 1 contains each participants personal characteristics based on information obtained from the short questionnaire.
| Code | Grade | Sex | Major | Web use | Future career plan |
|---|---|---|---|---|---|
| A | 3rd (U) | Female | Earth science | 2/week | Graduate school |
| B | 1st (G) | Female | Informatics | 2/week | Private company |
| C | 3rd (U) | Female | Science | 2/week | Graduate school or public administration |
| D | 3rd (U) | Female | History | Every day | Private company |
| E | 3rd (U) | Female | Cultural studies | Every day | Undecided |
| F | 3rd (U) | Female | Humanities | 1/week | Undecided |
| G | 4th (U) | Female | International studies | Every day | Private company |
| H | 3rd (U) | Female | Medicine | 2/week | Hospital (medical doctor) |
| I | 2nd (U) | Female | Education | Every day | Graduate school |
| J | Graduated | Female | Media | 1/week | Undecided |
| K | 4th (U) | Male | History | Every day | Public administration; librarian |
| L | 2nd (G) | Male | Science | Every day | Private company |
| M | 3rd (U) | Male | Economics | Every day | Undecided |
| N | 4th (U) | Female | English language | Every day | Graduate school |
In the post-search interview, participants articulated what they thought, felt, expected and intended to do at each move in their exploratory search processes, either voluntarily or in response to the interviewer's prompting. Analysis of the interview data allowed us to elaborate the taxonomy of knowledge modification and identify the stages of exploratory search processes at which participants tend to acquire knowledge. The analysis also permitted the development of a taxonomy of knowledge utilization.
The initial taxonomy of knowledge modification was developed by the author in previous research reported elsewhere (Miwa and Kando 2007). The taxonomy was expanded in this study, as shown in Table2. The six types of knowledge modification in italics correspond to the initial taxonomy, while the four other types were identified in this study. Among these ten types of knowledge modification, the first four types of adding, limiting, relating and specifying may be performed solely based on user-search engine interactions that use newly acquired knowledge from the ongoing search processes. Therefore, these may be readily performed by novice domain searchers.
Among these four types, limiting and specifying can easily be confused. In short, limiting occurs unexpectedly, while specifying occurs intentionally. We use limiting to represent the situation in which the initial query provides either a wide range of search results or irrelevant information, compared with what the searcher had anticipated. At this point, the searcher narrows down the conceptual knowledge represented in the initial query. This situation tends to occur unexpectedly as the searcher examines the search results. This is considered a type of focused learning of conceptual knowledge. The example presented in Table 2 is of a searcher (a sophomore student) looking for a graduate programme. She combined graduate programs and earth science (her undergraduate major) in the initial query, which provided her with mostly doctoral programmes. By examining the search results, she learned that the term graduate programs included both master's and doctoral programmes, and so she used master's course instead of graduate programs in the next query.
On the other hand, we use specifying to represent the situation in which the searcher intentionally seeks information that may describe the conceptual framework more specifically. The example presented in Table 2 is of a searcher looking for detailed information on a company at which she seeks a position by using the company's name. When she found that the geographic location of the company was downtown Tokyo, she obtained more specific knowledge that helped her to decide whether to apply for the job. She said, 'If the company is located in a local city or the suburbs, I would not apply.' Here, the knowledge modification was intentional.
The latter six types of knowledge modification require different levels of domain knowledge. Transforming and interpreting seem to require a high level of domain knowledge. In order for searchers to understand a concept in a different framework (transformation), they need not only recall relevant existing knowledge and compare it with the newly acquired knowledge, but also identify the different framework from their existing knowledge in order to reclassify the existing and new knowledge appropriately. Interpreting also requires a high level of domain knowledge since the searcher should be able to locate existing knowledge relevant to the newly acquired knowledge in order to construct a new meaning.
Less comprehensive domain knowledge may be required for recalling, correcting, clarifying and verifying. Recalling may occur in response to some stimulus gained from newly acquired knowledge that makes searchers remember somewhat relevant knowledge. Correcting, clarifying and verifying represent more or less the deliberate examination of existing knowledge in response to new knowledge acquisition.
| Type | Definition | Example of expressions* | |
|---|---|---|---|
| No domain knowledge required | Adding | Acquiring novel information to increase knowledge | There are two national universities in the Miyagi prefecture. |
| Limiting | Narrowing the scope of the concept | I want to obtain information on master's courses but not doctorates. | |
| Relating | Relating a concept to another concept | Queens Court is a member of the Java group. | |
| Specifying | Increasing the depth of meaning of the concept by increasing specificity | The company office is located in downtown Tokyo (not by text but) from the map. | |
| Domain knowledge required | Recalling | Remembering or merging existing knowledge | Proofreading is a job for a newspaper journalist. |
| Clarifying | Ambiguous knowledge becomes clearer | MD stands for merchandiser, a person who develops a marketing strategy. | |
| Correcting | Clearing up misunderstanding | There is no entry for 'graduate school' in the Yahoo category menu. | |
| Transforming | Understanding a concept in a different framework | I could find the way to the examination site. | |
| Verifying | Confirming or validating existing knowledge | X University was found, as I anticipated. | |
| Interpreting | Combining existing knowledge and acquired knowledge for heuristics | An idea of using the local government's homepage came to mind because I have experience in using another local government's homepage for a similar purpose. | |
| * Translated from Japanese by the author. | |||
Common steps in the exploratory search processes, identified by analysing the timeline matrix, indicated sequential shifts in behaviour, as follows.
We identified that most knowledge modifications occur at step 7, browsing lists of search results, and at step 10, browsing site contents.
When browsing lists of search results, searchers acquire (a) terminological knowledge (terms that may lead to sites that are more relevant) and (b) search results knowledge (the relevance of sites and/or number of hits). The former is used to modify keywords, and the latter is used to modify the search strategy or choose another search engine. When browsing the contents of the relevant site, searchers acquire (c) subject knowledge that might lead to modification of their information needs, (d) functionality knowledge of the site (functions and operations of retrieval, full-text search and pull-down menus) and (e) knowledge of other possible information sources. Acquisition of subject knowledge includes all the types of knowledge modification presented in Table 2.
In many Web searching processes, participants used knowledge acquired at an earlier point in the Web search process. Table 3 was developed by analysing how such knowledge is used. It presents the taxonomy of patterns of knowledge utilization. Of the patterns described, all but the last represent utilization of knowledge acquired in an earlier process of the same exploratory search processes.This indicates that knowledge acquired during exploratory search processes is evaluated and interpreted by the searchers and then used in a later process in the same exploratory search processes, e.g., browsing a site is initiated by knowledge obtained from previous browsing of the search results lists.
The termination of search behaviour occurs at three levels, in-site browsing, in-site retrieval (when such a function is available) and searching for the topic. Termination of in-site browsing appears to occur when it is recognized that the information contained in the site either does or does not meet the searcher's information needs. Termination of in-site retrieval occurs either when searchers realize that they cannot obtain the expected results using the retrieval function or when the required information has been obtained. Termination of searching for the topic occurs either when searchers realize that there is no useful information on the Web or when the information needs have been satisfied
| Type | Definition | Example of knowledge utilization | |
|---|---|---|---|
| Start browsing | Begin browsing within the site | After acquiring increasingly more important knowledge about Master's courses in the results lists, the participant clicks the link to the site and starts browsing its contents. | |
| Stopping | Stop browsing | Stop browsing within the site | When a participant learns that most students at the University of Air (a Japanese open university) are full-time workers, she stops browsing, indicating that she is not interested in the programme. |
| Stop retrieval | Stop retrieval within the site | Realizing that there is not much of a match for the search topic at a blog site, the participant stops retrieval at that site and moves to another blog retrieval site. | |
| Stop searching | Stop searching for the topic | Realizing that there is not much information on the search topic on the Web, the participant stops searching for the topic and moves to the next topic. | |
| Further searching | Continue exploratory search processes to further the search for the topic | When the participant has determined that the employment test will occur in the near future, she moves on to find the application procedure. | |
| Changing | Change keywords | Change or modify keywords | When the participant realizes that the initial keywords were incorrect, the keywords are modified before continuing the search. |
| Change logic | Change or modify search logic | When the participant determines that there are too many job descriptions for temporary staff, she modifies the search logic to exclude the term 'temporary' and limits the search to full-time jobs. | |
| Change strategy | Change the entire exploratory search processes strategy for the topic | When a participant realizes that there is no information on parking lots for bicycles, she recalls her previous experience when using a municipal government site and goes there to find the information. | |
| Planning for post-search activities | Plan what to do after using exploratory search processes on the Web | When a participant realizes that there is no information on a graduate course on the Web, she decides to ask a faculty member from her university. | |
Changes in search behaviour occurred at three levels: changes in keywords, changes in query or search logic and changes in the entire search strategy. All levels of change were initiated by the realization that the keywords, query or search strategy did not lead to the expected results. Therefore, we can conclude that searchers change any of these three strategies to optimize the search.When searchers recognize that their information needs may not be met by exploratory search processes on the Web or find references to more relevant information sources external to the Web, they stop and decide to explore other information sources, including libraries and people
These results indicate that modification of knowledge tends to occur when searchers are browsing either search results lists or site contents. This leads to either modification of search behaviour or termination of the search.
This study's results may have theoretical implications for Bates' (1979) search tactics and her berrypicking model (Bates 1989), as well as for Sihvonen and Vakkari's (2004) use of thesaurus terms.
For search tactics, the knowledge modification category of specifying, as identified in this study, is related to Bates' tactic of specifying. In a sense, these tactics are similar, since searchers intentionally 'search on terms that are as specific as the information desired' (Bates 1979: 208). Since the present study focused on how searchers' knowledge is modified during information acquisition, our explanation is that searchers use the specifying tactic to identify more specific information. Searchers' knowledge is modified to become more specific than its initial state when the desired information has been successfully identified.
The Web-search processes in our study resemble those in the berrypicking model. Searchers initiate their search for information on the Web using a broad question (e.g., to which job or graduate programme they should apply), with one part of a question (e.g., what is a good keyword to describe the job) and each new piece of information they encounter (e.g., a word found in Wikipedia) providing them with new ideas and directions in which to advance the search process. Searchers generate new questions and queries continually and they gradually identify useful information at every stage of the search, rather than obtaining all the necessary information in one stage using a single keyword search. In a sense, a Web search is a micro-level information search process with a single source of information embedded within a broader berrypicking process that can be used, for example, for job hunting. Thus, we may think of the exploratory search processes for job hunting as a macro-level berrypicking process, consisting of multiple micro-level berrypicking processes, each of which uses a different source of information.
Several Web-search processes were initiated by visiting the Wikipedia Website. During post-search interviews, all participants reported using Wikipedia to identify appropriate keywords for the search concept. This behaviour seems to be identical to the use of a thesaurus for online searching. Experienced searchers appeared to use the tactic of visiting Wikipedia to gather useful terms during the initial stage of their Web-search processes. If our observation is correct, experienced searchers obtain subject knowledge or keywords at the initial stage of their Web searching to improve interactive query expansion, a strategy proposed by Sihvonen and Vakkari (2004) for online database searching.
Fourteen students' Web search processes for career planning were obtained from a post-search interview, which was accompanied by video-recorded eye movements superimposed on the screen shifts to stimulate recall.
Following an analysis of the transcribed interview data, we modified the taxonomy of knowledge modification patterns and developed a taxonomy of knowledge utilization patterns during exploratory search processes. The results indicate that knowledge modification occurs when browsing either the search results lists or the site contents, leading to either modification of search behaviour or termination of the search.
The ten patterns of knowledge modification and nine patterns of knowledge utilization identified in this study seem to represent adequately micro-level exploratory search processes on the Web. The study's findings confirmed that a searcher's knowledge is modified throughout the exploratory search processes by their gradually acquiring information and that the modified knowledge is used to explore further. Thus, the findings provide concrete evidence to support existing models of exploratory search (Marchionini 2006), interactive information retrieval (Vakkari and Järvelin 2005), information tactics (Bates 1979) and berrypicking (Bates 1989).
Further examination led us to recognize that some patterns of knowledge modification (e.g., transforming and interpreting) require searchers to have a high level of domain knowledge, whereas other types of knowledge modification may be performed by searchers with little domain knowledge. We will further develop this research by exploring differences in knowledge acquisition patterns between domain novices and domain experts. We expect to determine the relationship between domain knowledge and exploratory search skills.
| Find other papers on this subject | ||
© the authors, 2008. Last updated: 7 December, 2008 |
|
