header
vol. 17 no. 1, March 2012

 

A study of the information search behaviour of the millennial generation


Arthur Taylor
Computer Information Systems, College of Business Administration, Rider University, Lawrenceville, New Jersey, USA


Abstract
Introduction. Members of the millennial generation (born after 1982) have come of age in a society infused with technology and information. It is unclear how they determine the validity of information gathered, or whether or not validity is even a concern. Previous information search models based on mediated searches with different age groups may not adequately describe the search behaviours of this generation.
Method. The longitudinal study discussed here examined the information behaviour of undergraduate college students who were members of the millennial generation. Data were collected from the students using surveys throughout an information search process as part of an assigned research project.
Analysis. Quantitative analysis was carried out on the data, which related to 80 individual subjects and evaluation of 758 documents.
Results. Statistically significant findings suggest that millennial generation Web searchers proceed erratically through an information search process, make only a limited attempt to evaluate the quality or validity of information gathered, and may perform some level of 'backfilling' or adding sources to a research project before final submission of the work.
Conclusions. These findings indicate that the search behaviour of millennial generation searchers may be problematic. Existing search models are appropriate; it is the execution of the model by the searcher within the context of the search environment that is at issue.


Introduction

The generation born between 1982 and 2000 has been identified as the millennial generation (Strauss and Howe 1991; Howe and Strauss 2000). They have come of age in a digital world with ubiquitous information sources. The libraries with their strong mediated search support are no longer the primary sources of information for them. Consequently, existing information behaviour models may not adequately describe their approach to filling information needs. The goal of this research was to explore the information search behaviour of this generation in relation to commonly cited information search behaviour models.

The longitudinal study detailed in this paper evaluated the search behaviour of millennial generation students conducting information searches in a naturalistic environment. In the study participants provided detailed information on their search choices and judgments of document relevance over a five week period. The findings based on these empirical results provide some indication of how members of the millennial generation progress through a search process. The subjects in this study proceeded erratically through their search process and did not appear to validate sources or information gathered.

Information search process models provide a framework for the study and evaluation of information searches. Models such as Kuhlthau's (1993), Ellis's (1989) and Wilson's (1999) provide some guidance on the search process but have limitations in their application. These models were developed prior to the widespread use of the Internet as an information resource. Ellis's subjects were adult professionals, not millennial generation students. Kuhlthau's subjects were from a variety of age groups and may have included some young millennial generation subjects, but the research was done before the availability of the Internet.

Research has provided little insight into how information behaviour differs on the Internet, specifically with the millennial generation, and whether or not previously identified models are appropriate. millennial generation users are more likely to search for information on the Internet using commercial search engines (Abram 2006; Oblinger 2003). Research on how these searches are conducted could inform us on the current information behaviour of the millennial generation. Results could provide the basis for improved information search process models which better reflect current technology and the generation of information seekers raised with this technology.

Literature review

In Generations Strauss and Howe (1991) present a broad historical panorama of generations and their influence on history. The authors identify the millennial generation as those born in or after 1982. In millennials Rising the authors identify the same start date for the generation and infer a cut-off date of 2000. The authors present theories concerning the influences of a generational cohort and the environment in which the cohort is raised and comes of age. The millennial generation is a group raised during a boom period. This prosperity created significant social and technological shifts leading to a distinct cultural identity.

The millennial generation is only familiar with a world where personal computers and information are easily accessible, and has distinct expectations concerning technology, communication, and access to information. Abram (2006) notes that millennial generation students expect instant access to information (cell phones, portable computers, Internet access), and tend to procrastinate with research and the choices it requires. They tend to be multi-taskers, often doing several tasks at once. They exist in a noisy, media-driven world, a condition that may lead to issues with filtering what is valid and important to their task. Constant socializing in a connected world leads to persistent distractions from any assigned task (Essinger 2006). The prevailing educational view is that students in undergraduate programs have incomplete cognitive thinking skills. This creates difficulty in discerning valid information from invalid information. In a mediated search environment, a librarian would help the student identify valid information sources and provide guidance on how to identify valid and useful information. This mediation is lacking in an environment where the student searches the Web unaided (Wieler 2004).

Gross and Latham (2011) conducted an investigation of experiences and perceptions of information with first year college students, members of the millennial generation. The researchers indicated that subjects viewed finding information as a product, not a process. Subjects in the study indicated that they did not perceive finding information as being difficult or requiring any particular skill set. One subject commented that 'the computer does all the work for you basically'. Subjects also felt the evaluation of information was subjective, not objective, suggesting a postmodernist view of information and knowledge in general. The focus on a search product and dismissal of a process for evaluating and verifying content by subjects is reflective of the millennial generation's neoliberal social view (Howe and Strauss 2000) where production and efficiency are important, and what appears to be a consumerist approach to education is a side effect of this view.

In Rowlands et al. (2008), the search behaviour of a subset of the millennial generation is examined in a broad, multi-method study. Researchers used a sample of those born after 1993, young members of the millennial generation. They noted that searchers of this generation tended to search horizontally rather than vertically, skimming content, viewing just one or two pages, and making quick relevance judgments based on this review. This thin, surface-level review of content is related to the lack of concern, or inability to discern, the authority of sources and the quality of content, as revealed in other studies of members of the millennial generation (Williams and Rowlands 2007; Hirsh 1999; Grimes and Boening 2001; Lorenzen 2001). The familiarity and comfort with the Web as an information search conduit creates other problems. The hypertext interface that is the foundation of the Web creates a fragmented view of information. The millennial generation is at ease in this environment, often not recognizing incomplete fragments as such. Each fragment of information appears to be as valid as any other. In Lorenzen (2001), subjects were clearly hesitant when asked how they evaluate the quality of a Website, some suggesting that a .com domain implied it was a reliable information source, and others indicating that a .com site was not a reliable source. Some subjects in the study felt that if a site was indexed by Yahoo! it had to be authoritative.

From a broader perspective, the culture within which the millennial generation has learned cognitive skills may be partly to blame for these problems. Philosophers have identified the postmodern condition as a rejection of the objective, scientific evaluation of knowledge, and the general acceptance of a an open, subjective, uncritical view of knowledge. There is growing concern that members of the millennial generation have adopted this uncritical view of information (Harley et al. 2001). Millennial generation information searchers appear to be more concerned with the time it takes to find and evaluate content than the overall validity or quality of that content, and generally prefer visual information to textual (Harley et al. 2001; Vondracek 2007; Wieler 2004). An additional cultural influence is the consumerist society within which the millennial generation has been raised. In Harley et al. (2001), the postmodern condition as applied to the dissemination of information is considered a disruptive force in the growth and distribution of knowledge in our culture. Consumerism, superficiality and knowledge fragmentation are a result. In the information economy, information is just another economic commodity which is consumed at the lowest cost. The cost in this context is effort, and when seeking information, the millennial generation individual often perceives the lowest cost as the most convenient, readily available information with limited consideration for quality (Buczynski 2005; Thompson 2003; Young and Von Segern 2001).

Information search process

How millennial generation searchers work to fill their information need can be examined within the framework of an information search process model. Kuhlthau (1993, 1991) modelled information seeking as a series of stages. Her model was built on personal construct theory, Taylor's (1968) stages of need formation, Belkin et al.'s (1982) anomalous states of knowledge, and theories and models of expression and mood. Though developed in 1991, more current research continues to validate the basic tenets of the model (Kuhlthau et al. 2008). Kuhlthau's model is often interpreted as sequential, but the model does specifically allow backtracking and iteration. Ellis (1989, 1997) identified several search patterns used for gathering information to fulfil an information need. His model emphasized the subjective, iterative nature of the search process and avoids suggesting there is a consistent sequential process taking place. Wilson (1999) provided a synthesis and comparison of Ellis's and Kuhlthau's models which maps Ellis's behaviour into Kuhlthau's stages.

Research questions

In order to better understand the information behaviour of the millennial generation, this research tracked millennial generation subjects as they progressed through a five week research project. The goal was to gather information to answer the broad question of how information seekers of the millennial generation progress through the process of finding information. Specifically, do they proceed through an orderly search process, or is their behaviour somewhat erratic and symptomatic of procrastination? Do millennial generation information seekers make some effort to discern the quality of documents reviewed on the Web, and if so, are they discerning throughout the search process? In general, is the search process used similar to previously identified search process models, or are there indications that their search behaviour has changed? These broad questions lead to the following specific research questions.

1. Do millennial generation information seekers progress through a search process similar to that of Wilson's (1999) consolidated model, or is their search behaviour different?

2. Do millennial generation information seekers evaluate the quality of Web resources? Are they discerning about quality-related attributes of the sources retrieved from the Web?

3. How do millennial generation information seekers make use of general information Websites such as Wikipedia? At what stage(s) in the search process are these pages used?

Research design

The study involved the collection of data based on user searches. Users were millennial generation students in an undergraduate business course. Subjects were given an assignment to create a presentation on a computer technology. The assignment was structured as a research report and subjects had to complete a bibliography of their sources. Most subjects had little knowledge of the research topic they were assigned. Data were collected on the results of each subject's information search. All searches were conducted on the Internet. Specifically information was collected on the document selected, the relevance judgment for the document (was the document relevant or not), the criteria used to judge document relevance, and the subject's stage in an information search process.

Data were collected using a Website with a modified search engine which executed searches using the Yahoo! search engine and then reformatted the results page to provide Web-based data collection for this study. Subjects were able to access this search engine interface over the Internet, thus allowing them to work in a naturalistic environment at their own pace. Subjects were not required to use the Website for searches, but they were required to use the Website to provide data on the results of their searches. Those who did not use the default search engine entered information on their sources using survey instruments identical to those used to enter the information from the default search engine. Approximately four source documents were entered by each individual using this mechanism. Almost all subjects chose to use the default search engine and almost all entries were entered using the survey instruments on those pages. It is worth noting that good sources of information could be found on the Web for the computer-related technical topics assigned to the subjects.

Approximately 80 subjects were drawn from a convenience sample of junior and senior undergraduate students at an American university. Data were collected in 2007 and subjects were on average 19 through 22 years of age, and were therefore born between 1985 and 1988, members of the millennial generation. Subjects signed an informed consent form which explained the purpose of the research and that the information they provided would be treated anonymously. Subjects were allowed to choose a research topic from a list of topics (see Appendix A). Topics were all of the same approximate level of difficulty and were related to course content. Subjects were given several weeks to complete their research assignment and were required to complete specific interim assignments during that time period. A project abstract was due the first week, a detailed outline was due next, the rough draft of the presentation slides due the next week, and the final presentation slides were due the final week.

Project data were collected anonymously using survey instruments integrated into the Web search engine interface (Figure 1).

Search engine interface

Figure 1: Search engine interface

Subjects examined the documents returned from the search engine and then used the Website to indicate the relevance of the documents they examined (relevant, not relevant, partially relevant/not sure about relevance), their current stage in an information search process selected from a predetermined list of search stages, and the criteria used to make that relevance judgment selected from a predetermined list of relevance criteria (Figures 2 and 3).

Search results page

Figure 2: Search results page
* Note: relevance judgment choices are presented in a drop-down list which presents a mutually exclusive choice of relevant, not relevant, and partially relevant or unsure about relevance.



Figure 3: Recording the search stage



Figure 4: Criteria for relevance judgements

The Website used for data collection contained detailed instructions on the use of the site and how to record the information required. Subjects were given instructions on the use of the Website and the meaning of the terms used on the site and help pages were provided to explain the search stage choices, relevance judgment choices, and criteria for relevance judgment choices used by the subjects. The Website captured data for each subject using an anonymous user ID for search stage choices, the document selected as a uniform resource locator (URL) and the relevance judgment.

Data collected on the search stage provided an indication of how subjects were progressing through an information search process and enabled the evaluation of research question one. The relevance criteria related to quality of the document (accuracy, source quality, etc.) provided information for evaluation of research question two, and also provided some indication of how and when subjects were evaluating documents, data relevant to question one. Data collected on Websites visited (URLs) provided information for research question three. When these Websites were being visited and the criteria used during each search stage also provided information about how subjects were progressing through the search process, data relevant to research question one.

Search process, stage in task completion, and relevance criteria choices

The search stage model used for this research (see Table 1) was developed from the phases suggested by Wilson (1999) which synthesized the information behaviour of Ellis (1997) with the search process of Kuhlthau (1993). The list presented to the subjects is shown in Description Displayed to Subject column in Table 1. The term initiation was selected because it was considered clearer than beginning or starting. The term browsing was avoided because this term is now commonly used for all Web-related searching and would potentially be confusing to subjects. Instead, the term exploration was used to describe the process of scanning and gathering information. The remaining terms, differentiating, extracting and verifying are from Ellis (1997) and were used in lieu of Kuhlthau's terminology because they were considered more precise. The process of ending the search was considered to be implied by the submission of the final deliverable and would not have generated data relevant to the research questions; it therefore was not presented as a choice to the subjects.

Table 1 contains the criteria identified by subjects as having some bearing on their relevance decision (relevant, not relevant, don't know or partially relevant), the decision whether or not the document (or Website) is relevant to their information need. The relevance criteria chosen by subjects were those factors that contributed to their relevance decision. Prior research by Barry (1994), Barry and Schamber (1998), and Cool et al. (1993) identified the criteria listed in Table 1. A number of studies have identified these criteria and have provided some confirmation as to their consistency across information retrieval tasks (Barry and Schamber 1998; Park 1993; Schamber 1991; Schamber and Bateman 1996; Xu and Chen 2006). A subset of these criteria was presented to subjects not as specific criteria but using the contents of the Description displayed to the subject column in Table 3 (see Appendix C). Subjects were allowed to choose one or more criteria which they felt contributed to their relevance decision.

The criteria selected for this research are a subset of those identified in Table 1. The reasons for these selections are as follows. The Source column in Table 1 identifies the source of the relevance criterion: Barry (1994), or Barry and Schamber (1998) , or Cool et al. (1993). To reduce the potential for confusion and survey exhaustion on the part of the subject, the number of relevance criteria presented was limited to fifteen. In addition to some criteria from the Cool study, criteria identified in the Barry (1994), and Barry and Schamber (1998) studies were used. Excluded were those criteria that were specific only to Schamber's (1991) earlier study and related to document qualities specific to her topic (weather reports) and did not apply to the topics used in this study.


Table 1: Relevance criteria
* from Barry (1994: 154); Barry and Schamber (1998: 226); and Cool et al. (1993: 3).
Criteria Type Source* Used Description displayed for subject
Depth, scope or specificity Document Barry, Cool Yes Document contains good depth on the topic
Accuracy or validity Document Barry, Cool Yes Document appears to be accurate
Currency (recency) Document Barry, Cool Yes Information is current, recent, up-to-date
Tangibility Document Barry, Cool Yes Information relates to real, tangible issues; not esoteric or theoretical
Quality of sources (source quality) Document Barry, Cool Yes Source is reputable, trusted, considered expert
Availability of information Situation Barry, Cool Yes The extent to which the information is available
Verification Document Barry, Cool Yes The information is consistent with the body of knowledge the field; the information supports the user's point of view
Affectiveness Document Barry, Cool Yes The user's emotional response to the information; pleasure, enjoyment, entertainment
Effectiveness of proposed approach Document Barry Yes How effective is the approach proposed?
Consensus within the field Document Barry Yes How much consensus there is in the field for what is proposed in the document
Time constraints Situation Barry Yes How much time is allowed for the task to be completed?
Background, experience or ability to understand Situation Barry Yes Expression of concern over the ability to understand a document (same as 'understandability')
Novelty, content novelty or source novelty Document Barry Yes The source or content of the document is new to the subject.
Geographic proximity Document Schamber No Refers to weather information in a geographic location
Dynamism Document Schamber No Refers to the ability to dynamically manipulate the information in a document
Presentation quality Document Schamber No Iindication that the source of the information could be manipulated in some way.
Structure Document Cool Yes The structure of the document; how the information is presented and organized
Amount of information Document Cool Yes Document provides sufficient information.
Depth Document Cool Yes Document covers the topic in good depth.
Timeliness (age of document) Document Cool Yes Is the time frame of the document appropriate; (current where recent information is required; written in a certain time period for historical significance)
Understandability Document Cool Yes The document is understandable by the subject (ability to understand)
Bias Document Cool Yes The Document is written with a particular viewpoint.
Authority Document Cool Yes The author or publication has a good reputation in this field.

Deliverables required of the subject and the month and day they were due are identified in Table 2. Some feedback was provided upon submission of the abstract and detailed outline deliverables to encourage the students to produce a thorough and well-researched presentation. Feedback concerned only the technical aspects of the topic and the scope of their final presentation. Feedback did not concern choice of Websites, search stage choices or relevance criteria choices made by the subjects.

The final deliverable was a ten to thirty slide PowerPoint presentation. Subjects were encouraged to go beyond general information sites (such as Wikipedia), and to use quality sources such as trade journals whenever possible. (Computer trade journal content is widely available on the Web.) Subjects were also encouraged to use between ten and twenty sources to prepare their presentation.


Table 2: Deliverables used
Deliverable Description Due date
Abstract Several paragraphs explaining what will be covered in the presentation. 2 November
Detailed outline One or more pages of outline text which explain the topic, and subtopics which to be presented and the flow of the presentation 10 November
Rough draft of presentation slides PowerPoint slides containing the rough draft of the presentation 20 November
Final presentation slides PowerPoint slides containing the final presentation 7 December

Results

A total of 80 subjects participated in the study and examined and provided 758 distinct Web page evaluations. Table 5 identifies the number of documents evaluated by subjects for each project deliverable. This table shows that the number of documents evaluated varied significantly for each deliverable (χ2 = 98.9, df = 3, p-value <0.001). Subjects evaluated a total of 81 documents for the project abstract deliverable, but evaluated 225 documents for the detailed project outline. This finding is not unexpected considering the subjects' instructions for the preparation of the report abstract were to create a brief explanation of the project and thus did not require a detailed knowledge of the topic. Table 3 also shows that document selection was evenly distributed over the duration of the research project.


Table 3: Document count by deliverable due
Project deliverable   % of total
Abstract 81 10.69%
Detailed outline 225 29.68%
Rough draft 187 24.67%
Final presentation 265 34.96%
Total 758 100.00

Table 4 lists the number of search stages reported by subjects. As this table indicates, only five subjects reported being in all the search stages provided as options, and a significant number of subjects (67%) reported being in fewer than four (χ2 = 9.6, df = 1, p-value <0.001).


Table 4: Search stages reported by subjects
Stages reported Users reporting
5 5
4 22
3 20
2 20
1 13
Total 80

Table 5 and 6 identify the percentage of documents evaluated in each search stage. Table 5 lists percentages within search stage, and Table 8 lists percentages within project deliverable. This provides some indication of how subjects proceed through the information search process. As subjects evaluated documents, they indicated which search stage they were in based on a list of search stage explanations (Table 2). Expectations are that research done for preparation of the first deliverable, the project abstract, would be the initiation stage (identified as the 'initial search; start of the search process'). As Table 7 indicates, slightly less than half of the subjects indicated that they were in the initiation stage while preparing the project abstract. This finding indicates that over half the subjects reporting for the project abstract, the first deliverable, reported being in a search stage other than the initiation stage.

Table 6 provides another perspective on the initiation stage and the relationship between search stage and deliverable. As a percentage of documents evaluated within a search stage, during the preparation of the detailed outline, approximately 50% of the subjects reported being in the initiation stage, and they reported this after completing the project abstract, the first deliverable. Additionally, 13% reported being in the initiation stage during the preparation of the rough draft, and 11% reported being in the initiation stage during the preparation of the final presentation. This indicates a non-serial progression through the information search process in relation to project deliverables.

As Table 5 indicates, only 16% of the subjects reported being in the verifying stage while preparing the final presentation deliverable. Subjects were more likely to identify extracting as their current search stage (defined as ‘extracting information to answer the question’) during this time frame. This indicates that these subjects were not concluding the search process in a manner consistent with Kuhlthau's information search process which has an expectation that the subject will perform some verification or evaluation of the research collected in the final stage.


Table 5: Percent of documents evaluated by search stage within deliverable
Search stage Abstract Detailed outline Rough draft Final presentation
Initiation 47.08% 25.32% 16.25% 8.89%
Exploration 22.81% 25.67% 9.96% 12.72%
Differentiating 7.48% 10.57% 2.85% 9.22%
Extracting 20.62% 35.69% 65.84% 53.18%
Verifying 2.01% 2.75% 5.10% 15.99%
Total 100.00% 100.00% 100.00% 100.00%



Table 6: Percentage of documents evaluated by deliverable within search stage
Search stage Abstract Detailed outline Rough draft Final presentation Total
Initiation 25.75% 49.70% 13.67% 10.88% 100.00%
Exploration 14.37% 58.05% 9.66% 17.93% 100.00%
Differentiating 10.62% 53.89% 6.22% 29.27% 100.00%
Extracting 5.59% 34.72% 27.45% 32.25% 100.00%
Verifying 3.62% 17.76% 14.14% 64.47% 100.00%

Evaluation of general information sites

The selection of general information sites (defined in this study as the Wikipedia site) varied significantly in relation to project deliverable due (χ2 = 12, n=80, df = 3, P <0.01) as shown in Table 7. This site selection was approximately 10% of the total documents chosen overall (80 out of 758). Subjects were instructed to search beyond basic informational sites, and there were numerous sites available for their assigned topics. As Table 7 indicates, these sites were used consistently during the preparation of the abstract and rough draft, ranging between approximately 6% and 10% of the total sites chosen. There was a significant increase in the usage of these sites during the preparation of the final presentation. Table 8 provides additional insights into the use of these sites, showing a statistically significant relationship between the search stage and the selection of Wikipedia Web sites (χ2 = 15.7, df = 3, p-value < 0.001). These results indicate that Wikipedia sites represent approximately 40 % of the of the sites evaluated when subjects reported being in the 'extracting' stage (identified as "extracting information to answer the question").


Table 7: Wikipedia pages reviewed by deliverable due
* Percentage of Wikipedia pages within all pages evaluated for the deliverable due
Deliverable due Count Percentages*
Abstract 8 9.30%
Detailed outline 29 9.90%
Rough draft 15 5.84%
Final presentation 28 14.97%
Total 80  



Table 8: Wikipedia pages reviewed by search stage
* three subjects did not report a search stage
Stage code Count Percentages
Initiation 20 25.97%
Exploration 17 22.08%
Differentiating 6 7.79%
Extracting 31 40.26%
Verifying 3 3.90%
Total* 77 100.00%

Evaluation of criteria used to judge relevance

An examination of the criteria used by subjects to determine relevance provides some indication of their reasons for selecting or rejecting documents as they progressed through an information search process. Since there were a large number of criteria examined by the subjects, Table 11 is simplified to identify those criteria which changed most in terms of their selection as the subject moved through the search process. As this table indicates, subjects were more likely to select amount of information and depth as criteria in the earlier search stages (initiation and differentiating) than in later stages (initiation and verifying). The criteria of novelty of sources, the structure of the document, and time constraints of the subject was selected more in later stages. This may indicate that in later search stages the subjects are looking for new sources of information and possibly documents of different structure, providing a different approach and new information for their subject area. The criterion of time constraints was also more likely to be selected later in the process as deadlines loom. The criteria of  and depth were selected less later in the search process, when other criteria such as authority and structure were more likely to be selected. Though authority was selected more in later stages, the criterion of source quality was selected less, suggesting that based on the subjects' selections, subjects may not apply equal weight to authority and source quality as criteria.


Table 9: Stage/criteria percentages (reported as a percentage of the total choices within a search stage)
Relevance criteria Initiation Differentiating Extracting Verifying
Affectiveness 0.57% 4.37% 2.77% 3.52%
Amount of information 8.92% 12.23% 10.77% 7.42%
Authority 1.90% 3.49% 2.65% 3.91%
Bias 1.90% 1.75% 1.94% 3.13%
Depth 8.92% 12.23% 9.03% 7.42%
Novelty 1.14% 3.93% 2.06% 2.73%
Recency 8.35% 7.86% 6.97% 6.25%
Source quality 4.93% 3.49% 4.06% 3.13%
Structure 6.26% 5.68% 7.55% 8.20%
Time constraints 0.57% 1.31% 1.29% 2.34%

An examination of results grouped by deliverable required provides a slightly different perspective on the progress of the subjects, as shown in Table 10. This table examines criteria selected based on the date the subject made the evaluation. The date was then correlated with the deliverable due in that time frame. Based on this evaluation, subjects working on the final presentation, subjects who were approaching the end of their research effort, were more likely to select amount of information and understandability as criteria in their document choice. The accuracy, source quality, depth and breadth, however, were less likely to be selected by these subjects.


Table 10: Comparison of criteria codes for rough draft and final presentation
Criteria code Rough draft Final Presentation
Accuracy 14.66% 11.71%
Affectiveness 3.02% 5.43%
Amount of information 12.07% 14.00%
Authority 3.23% 4.29%
Bias 3.23% 2.14%
Breadth 7.76% 6.71%
Depth 13.36% 10.29%
Novelty 1.94% 3.14%
Recency 8.84% 9.14%
Source quality 5.17% 4.00%
Structure 8.41% 9.14%
Time constraints 2.80% 2.43%
Understandability 15.52% 17.57%

Discussion

In evaluating results in reference to research question one, these empirical results suggest that millennial generation subjects do not proceed in an orderly fashion through an information search process. Though models such as Kuhlthau's (1993) did not predict a strictly linear progression through the information search process, it was implied that subjects would move through each of the various steps in the process. Subjects in this sample gathered documents consistently over the course of the assignment (see Table 3). Subjects should have acquired good knowledge of the topic by the time they completed their rough draft, but with this sample approximately 35% of the documents used were retrieved after the rough draft was completed, suggesting procrastination or backfilling (adding sources late in the research process as they finalize their report) on the part of subjects. A significant portion of subjects (67 %) reported being in fewer than four of the search stages studied (Table 4) also providing some indication that subjects may not be moving through a search process in an manner consistent with the search stages used in this study.

If subjects had made an orderly, pragmatic progression through an information search process, it would be expected that they report they were ‘verifying information gathered previously during the preparation of the final project deliverable. This search progression would be based on the expectation that subjects had gathered research previously, and were now examining and verifying that research as part of completing the final deliverable. The empirical evidence gathered with this subject pool suggests otherwise. The majority of subjects (53%) evaluating documents for the final presentation identified themselves as being in the extracting stage, described as ‘extracting information to answer the question’. Only 16% of subjects preparing the final deliverable identified themselves as being in the verifying stage. A closer examination of the verifying stage indicates that the selection of this stage as a percentage of the stages reported for a particular deliverable never exceeded 15% of the total (see Table 5), an indication that subjects were not ‘verifying information that has been gathered previously’. Most of the documents evaluated by the subjects (approximately 70%) were evaluated in the final two stages, the latter portion of the project's duration. This amount demonstrates a statistically significant correlation with search stage (χ2 = 28.1214, df = 1, p-value <0.001) suggesting that subjects may be procrastinating or backfilling.

Over half the subjects reporting for the first deliverable reported being in a stage other than the initiation stage. This finding could be an indication that in this time frame, subjects found several documents and reported being in the initiation stage, and then after absorbing those initial set of documents, considered themselves out of the initiation stage and in some other stage. Subjects may have performed some significant portion of their research in this early stage, quickly moving beyond the initiation stage.

In reference to research question two, examination of the results concerning subjects' evaluation of the quality of Web resources provides some indication that subjects were not concerned with the quality, validity, or authority of the documents selected. Subjects selected the categories of source quality and authority (of sources) at a rate of between 55% or less than that of amount of information and depth (see Table 9) suggesting that these criteria were not as important to subjects as simply retrieving some quantity of information on their topic. Categories such as structure, amount of information, recency (currency) and depth were consistently selected more often by subjects, suggesting these criteria were more important to the subjects than source quality or the .

Subjects did not select verifying (presented as 'verifying information that has been gathered previously') for any particular deliverable at a rate higher than 16% (Table 5), suggesting that subjects generally found some other search stage category provided a better description of their behaviour. Based on these results, it appears subjects did not attempt to verify their sources, and it can be assumed they considered the information valid or were not concerned about the validity of the source.

In reference to research question three, subjects did appear to make use of general information sites such as Wikipedia, but they did not select these sites consistently throughout the search process. The instructions presented to subjects on the use of general information sites is obviously an intervening factor to be considered in the evaluation of this statistic. However, the timing of the selection and use of these sites by the subjects suggests that they were likely to select these sites in the extracting stage. Also, the selection of Wikipedia for approximately 15% of sites for the final deliverable suggests that for some subjects, these sites were useful in the later stages of the research project. The difference between the selection of these sites in early stages versus late stages, however, is not statistically significant. For this sample, subjects appeared just as likely to use these sites in early search stages (initiation, exploration) as in later search stages (differentiating, extracting, verifying).

Limitations

The subjects in this study were from a university in the USA. The articles referenced in the literature review also cited studies performed on subjects in schools in the USA. It is not clear that search behaviour, or more broadly, the behaviour of millennial generation subjects, will be the same in other cultures. Subjects were also undergraduate students with a mixed scholastic background (below average, average, above average). Graduate students or professionals may perform differently.

The task assigned students provided specific instructions to encourage students to perform research. Students were told that they were being graded partially on the quality of their research, and this meant that more quality sources cited would result in a higher grade. Specific instructions given to the subjects also addressed limiting the use of general information sites such as Wikipedia. These intervening factors may have encouraged students to evaluate more documents from quality sites and perform additional research during the later stages of the project. In the absence of these instructions, it is possible Wikipedia sites may have represented a larger portion of the sites selected and referenced.

This study was designed to allow the subjects to work in an environment outside of a lab at their own pace and at their own convenience. The intention was to create an environment where the subject would feel comfortable and behave in a typical manner and provide meaningful results on the common information behaviour of the millennial generation. This meant that the subjects themselves reported their search stage and criteria and relevance as opposed to the method in many information behaviour studies where researchers provide that evaluation. Though subjects were tutored in the use of the system and provided with explanations for what was being reported, and though help prompts and help pages were available to subjects, there is the possibility that some subjects may have misinterpreted these instructions and that these prompts and requirements on reporting may have led to some non-typical behaviour on the part of subjects.

The task assigned to the students involved a subject area that was suitable for Web-based research. This assignment may have encouraged fragmented searches and retrieval since information for many of the assigned topics could be retrieved from multiple sources available on the Web. Other assignments, for example writing a paper on Western philosophy for a humanities course, might involve more focused searches with fewer sources.

The research methods employed allowed subjects to select topicality as a criterion, but did not specifically analyse topicality in relation to the selection of other criteria. This approach was based on the assumption that the document first needed to be on topic before other criteria would be considered (Wang and Sorgel 1998; Crystal and Greenberg 2006). This research focused on the use of more specific relevance criteria.

Additional influences on relevance decisions are known to be user's background or knowledge of the subject domain, and search task. The convenience sample for this research was drawn from a pool of undergraduate students who are business majors in a business school at an American university. All students were taking the same course and were given the same assignment, thus all had the same work task. These influences are mitigated in this study by drawing from a subject pool whose members have similar backgrounds, experiences, and domain knowledge. Though this aspect of the design of these studies attempted to control for variations in domain knowledge, variations in knowledge may exist within the subject pool. The choice of this convenience sample also limits the generalisability of the results.

Though subjects could use any search engine to gather sources, the Yahoo! search engine was used by default since it was easier to integrate search results into the Web-based data collection instruments used in this study. Almost all subjects chose to use the default search engine. The search engine itself and the search results were not being examined in this study, but use of a different search engine as the default (for example, Google) may have produced different results.

Conclusion

These findings suggest that the execution of searches by millennial generation searchers may be problematic. Existing search models are appropriate; it is the execution of the model by the searcher within the context of the search environment that is at issue. millennial generation searchers have come of age with different information gathering resources than their predecessors. The filter of the librarian-mediator relationship is gone and replaced by a profusion of fragmented and sometimes dubious information sources. It is within this environment that they search and gather the information they need. The results presented here provide some indication that members of the millennial generation do not consider verification of Internet sources important, indicating a non-critical view of information found on the Internet. This is consistent with results by Gross and Latham (2011) who report that subjects in their study indicated that information quality was not a concern and express surprise and incredulity that there may be an objective measure of information evaluation.

These findings point towards an information literacy problem that may be based more on a perception rather than lack of information searching skills. The millennial generation has come of age with technology that provides access to a vast variety of information sources. They constantly make choices concerning focus. They clearly know how to filter, the problem is what they choose to filter when evaluating information.

As Maybee (2006) indicated, learning involves a relationship between the learner and the subject matter. Changing the relationship involves changing the learner's perceptions. Subjects identified in Gross and Latham (2011) were not concerned with the quality of information gathered, consistent with the behaviour of the subjects in the study reported here who rarely reported validating information sources (identified as source quality). Gross and Latham further indicate that subjects did not perceive the existence of an objective standard for evaluating information (see Gross and Latham 2011: 173). Finding and evaluating information is subjective, reflecting a distinctly postmodernist view of information and knowledge.

It is this core perception of information quality as a subjective concept, and perhaps even a broader uncritical view of information in general, that must be addressed. Further research should explicate this view and perhaps the broader worldview of subjectivity versus objectivity amongst the millennial generation. A better understanding of why members of this generation resist using objective information evaluation standards may provide strategies for addressing this perception and thus improving the search skills of the members of the millennial generation.

About the author

Arthur Taylor is a Professor of Computer Information Systems at Rider University. Before joining Rider University, he worked for over 17 years in the field of information systems. His research interests include relevance, the information search process, and computer supported education. He can be contacted at ataylor@rider.edu


References
  • Abram, S. (2006). Millennials: deal with them. Texas Library Journal, 82(3), 100-103.
  • Barry, C.L. (1994). User-defined relevance criteria: an exploratory study . Journal of the American Society for Information Science, 45(3), 149-159. Retrieved 8 February, 2012 from http://www.asis.org/Publications/JASIS/Best_Jasist/1995Barry.pdf (Archived by WebCite® at http://www.webcitation.org/65Iga1qEc)
  • Barry, C. L. & Schamber, L. (1998). Users' criteria for relevance evaluation: a cross-situational comparison. Information Processing and Management, 34(2-3), 219-236.
  • Belkin, N, Oddy, R.N. & Brooks, H.M. (1982). ASK for information retrieval. Part I. Journal of Documentation, 38(2), 61-72.
  • Buczynski, J.A. (2005). Satisficing digital library users. Internet Reference Services Quarterly, 10(1), 99-102.
  • Cool, C., Belkin, N.J., Frieder, O. & Kantor, P. (1993). Characteristics of texts affecting relevance judgments. In Proceedings of the Fourteenth National Online Meeting, (pp. 77-83). Medford, NJ: Learned Information. Retrieved 8 February, 2012 from http://comminfo.rutgers.edu/~cgal/CV%20PDFs/online93_paper.pdf (Archived by WebCite® at http://www.webcitation.org/65IgvvyaM)
  • Eisenberg, M.B. (1996). Take the Internet challenge: using technology in context . Library Talk, 4(4), 5-7.
  • Ellis, D. (1989). A behavioural approach to information science retrieval design. Journal of Documentation, 45(3), 171-212
  • Ellis, D. (1997). Modelling the information seeking patterns of engineers and research scientists in an industrial environment. Journal of Documentation, 53(4), 384-403.
  • Essinger, C. (2006). X/Y: managing the millennial generation. Texas Library Journal, 53(4), 384-403.
  • Grimes, D. & Boening, C. (2001). Worries with the Web: a look at student use of Web resources. College and Research Libraries, 62(1), 11-22.
  • Gross, M. & Latham, D. (2009). Undergraduate perceptions of information literacy: defining, attaining and self-assessing skills. College and Research Libraries, 70(4), 336-350.
  • Harley, B., Dreger, M. & Knobloch, P. (2001). The postmodern condition: students, the Web, and academic library services. Reference Services Review, 29(1), 23-32.
  • Howe, N. & Strauss, W. (2000). Millennials rising. New York, NY: Vintage Books.
  • Kuhlthau, C. (1993). Seeking meaning: a process approach to library and information services. Norwood, NJ: Ablex Publishing Company.
  • Kuhlthau, C.C., Heinström, J. & Todd, R.J. (2008). The 'information search process' revisited: is the model still useful? Information Research, 13(4), paper 355. Retrieved 8 February, 2012 from http://informationr.net/ir/13-4/paper355.html (Archived by WebCite® at http://www.webcitation.org/65IhRRq6c)
  • Kuhlthhau, C. (1991). Inside the search process: information seeking from the user's perspective. Journal of the American Society for Information Science, 42(5), 361-371.
  • Lorenzen, M. (2001). Land of confusion? High school students and their use of the World Wide Web for research. ResearchStrategies, 18(2), 151-163.
  • Maybee C. (2006) Undergraduate perceptions of information use: the basis for creating user-centered student information literacy instruction. Journal of Academic Librarianship, 32(1), 79-85.
  • Oblinger, D. (2003, July/August). Boomers, gen-xers, millennials: understanding the new students. Educause Review, 37-47. Retrieved 8 February, 2012 from http://net.educause.edu/ir/library/pdf/ERM0342.pdf (Archived by WebCite® at http://www.webcitation.org/65IhoMDj0)
  • Park, T.K. (1993). The nature of relevance in information retrieval: an empirical study. Library Quarterly, 63(3), 318-351.
  • Rowlands, I., Nicholas, D., Williams, P., Huntington, P., Fieldhouse, M., Gunter, B. … Tenopir, C. (2008). The Google generation: the information behaviour of the researcher of the future. Aslib Proceedings, 60(4), 290-310.
  • Schamber, L. (1991). User's criteria for evaluation in a multimedia environment. Proceedings of the Annual Meeting of the American Society for Information Science, 28, 126-133.
  • Schamber, L. & Bateman, J. (1996). User criteria in relevance evaluation: toward development of a measurement scale. Proceedings of the Annual Meeting of the American Society for Information Science 33, 218-225.
  • Strauss, W. & Howe, N. (1991). Generations. New York, NY: William Morrow and Company.
  • Taylor, R.S. (1968). Question-negotiation and information seeking in libraries. College & Research Libraries, 29(3), 178-194.
  • Thompson, C. (2003). Information illiterate or lazy: how college students use the Web for research. Portal: Libraries and the Academy, 3(2), 259-267.
  • Vondracek, R. (2007). Comfort or convenience? Why students choose alternatives to the library. Portal: Libraries and the Academy, 7(3), 277-293.
  • Williams, P. & Rowlands, I. (2007). Information behaviour of the researcher of the future. Work package II: The literature on young people and their information behaviour. London: Joint Information Systems Committee (JISC). Retrieved 8 February, 2012 from http://www.jisc.ac.uk/media/documents/programmes/reppres/ ggworkpackageii.pdf (Archived by WebCite® at http://www.webcitation.org/65IiKxfVr)
  • Wieler, A. (2004). Information-seeking behaviour in generation Y students: Motivation, critical thinking, and learning theory. Journal of Academic Librarianship, 31(1), 46-53.
  • Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3), 249-270. Retrieved 8 February, 2012 from http://informationr.net/tdw/publ/papers/1999JDoc.html (Archived by WebCite® at http://www.webcitation.org/65ImjsKV6)
  • Xu, Y.. & Chen, Z. (2006). Relevance judgment: what do information consumers consider beyond topicality? Journal of the American Society for Information Science and Technology, 57(7), 961-973.
  • Young, N.J. & Von Segern, M. (2001). General information seeking in changing times: a focus group study. Reference and User Services Quarterly, 41(2), 159-169.
How to cite this paper

Taylor, A. (2012). "A study of the information search behaviour of the millennial generation" Information Research, 17(1) paper 508. [Available at http://InformationR.net/ir/17-1/paper508.html]
Find other papers on this subject



Check for citations,using Google Scholar

logo Bookmark This Page

Appendix

Research topics assigned to subjects


Hit Counter by Digits
© the author, 2012.
Last updated: 9 February, 2012
Valid XHTML 1.0!