What's interrupting your search?

Introduction

As mobile devices have become more powerful and wireless network connectivity has improved, they have transitioned from being personal data assistants to highly portable computers (albeit with constrained screen sizes and tap-based input mechanisms). As a result, people are increasingly using their mobile devices for tasks that used to be reserved for desktop/stationary computer use. One such task is mobile information retrieval (Crestani, et al., 2017).

While the ability to conduct a search within a mobile setting has allowed searchers to find the information they need in context, and without having to wait until they get back to their computers, it does mean that searches are now being performed in environments where there is an ever-present risk of interruption. Although recent studies have explored the impacts of interruptions, both in a general (Couffe and Michael, 2017) as well as in a mobile search settings (Hoeber, et al., 2021) - and have found that they can have profound impacts on both objective and perceived search performance (Harvey and Pointon, 2017) - little has been done to assess the complex environment of mobile search and measure the prevalence of interruptions that can occur in the real world. Further, there has been little attempt to codify interruptions into those that can generally affect search tasks and those that are specific to a mobile context of use.

Interruptions have four unique features: (1) they are induced by an event; (2) the event may be either internal or external to the user; (3) the primary task is temporarily suspended; and (4) the user has an intention to return to the primary task (Couffe and Michael, 2017). The first two points highlight the event-driven nature of interruptions; the last two points highlight the difference between someone being interrupted or choosing themselves to undertake a new task.

In this study, we address the following three research questions:

• RQ1: What is the nature of the interruptions that searchers experience in mobile settings?
• RQ2: Are the search interruptions internally or externally induced?
• RQ3: Are the search interruptions mobile-specific or not?

Using a diary study approach we collected data on the everyday interruptions participants experienced when searching using their mobile phones. In addition to using an inductive open coding approach to identify and categorise the types of interruptions (RQ1), we also used deductive coding with an a priori coding scheme to assess whether each interruption was internally- or externally-induced (RQ2), and whether it could occur generally or is unique to mobile contexts (RQ3). Together, these results paint a picture of how mobile web searchers are being interrupted, providing an empirical basis for the design of future surveys and controlled laboratory studies on mobile search interruptions, and the design and study of mobile search interfaces that may mitigate the effects of the most common types of interruptions.

Related Work

Since 2015 more than half of all web searches have originated from mobile devices (Dischler, 2015) and, as a result, user behaviour during mobile search has become a major research topic (Crestani, et al., 2017). Searching on a mobile device is quite different from performing the same act on a desktop: it requires information to be integrated into a smaller screen, causing users to be more purposeful and focused more on their search results (Ong, et al., 2017); and the fact that it is frequently done ‘on-the-go’ means that both distractions and interruptions from the environment are much more common and more detrimental (Harvey and Pointon, 2017). Previous work has shown that over a third of mobile searches occur when people are on the move (Heimonen, 2009), including when people are using public transport (Church and Oliver, 2011; Hoggan, et al., 2009), and even driving vehicles (Larsen, et al., 2020).

People use mobile devices to perform searches in a wide variety of different locations and contexts, and these differing contexts of use can influence which kinds of information needs arise (Sohn, et al., 2008). Komaki, et al., 2012 showed that users’ searches on the move often fail, and suggested that the location of the search greatly influenced the user’s judgement. They also found that commuters on the move often failed their searches for a number of different reasons, although they did not specifically identify what these reasons were. Sohn, et al., 2008 demonstrated that external factors, such as a lack of Internet connection or being on transport, often caused people to postpone their attempts to resolve these needs until later, a finding corroborated by more recent work (Aliannejadi, et al., 2019), which also showed that complete abandonment of search tasks is common in certain mobile contexts.

Many of these different contexts of use introduce the risk of being interrupted. The detrimental effects of these interruptions on mobile search performance, behaviour, and people’s perceptions thereof, have been demonstrated through simulated laboratory studies (Harvey and Pointon, 2017, 2019; Hoeber, et al., 2021), ‘in-the-wild’ field experiments (Aliannejadi, et al., 2019; Hoggan, et al., 2009; Komaki, et al., 2012), and observational studies (Komaki, et al., 2012). Such situations increase cognitive load and, therefore, reduce querying effectiveness (Harvey and Pointon, 2017). These mobile-specific contexts can also impact user search behaviour in terms of changes in people’s hand movements and grip, and an increase in error rates (Harvey & Pointon, 2019). Auditory distractions can be stressful for users, causing them to feel additional time pressure (Harvey and Pointon, 2019; Hoggan, et al., 2009), and the desire to search for and access information on-the-go can sometimes be very dangerous (Simmons, et al., 2020). The perception of the impact of the interruption of the task can be significant, even surpassing the actual objective impact itself (Hoeber, et al., 2021).

We know that people search frequently in mobile contexts, that these contexts can cause people to change, postpone, or even abandon their searches, and that interruptions can have many negative effects. However, little research has been done to categorise the nature of real-world search interruptions, identify whether they are induced by internal or external factors, or whether they are specific to conducting the search in a mobile context.

Method

We used a diary study approach to collect data about participants’ everyday mobile search interruptions. Diary studies are a good way to collect information about user behaviour in context because they permit the capture of data in a natural setting, without the distracting influence of an observer, or the contrivance of a laboratory study (Bolger, et al., 2003). Such an approach has been used in previous work to obtain first-hand information about users’ search behaviour (Komaki, et al., 2012; Sohn, et al., 2008) and to inductively develop taxonomies of search tasks (Elsweiler, et al., 2007). They can, however, suffer from missing data, as participants do not always remember to complete an entry when something relevant happens or it has been deemed to be too insignificant to report, and their recollection of events can be flawed. Participants may also lose motivation in the study as it goes on and cease the collection of data (Bolger, et al., 2003).

In order to mitigate some of the aforementioned issues, we limited the number of questions in each diary entry, instructing participants that they they should note up to two or three examples per day, and sent daily reminder emails to each participant. We also prepared a participant information sheet, which provided a clear and detailed description of what participants should record and submit during the study, and when. Although we recognise that there are a variety of forms of ‘mobile’ Internet-enabled devices (e.g., tablets, smartwatches, etc.), for the purposes of this work, we asked participants to only submit diary entries for when they were using their mobile (smart)phones. We asked participants to record their diary entries at the end of each day (a retrospective, interval-contingent diary approach). Although this can result in issues stemming from imperfect recall, it reduces load on the participants, increasing the probability that they will complete the study, and prevents the requirement to diary immediately from influencing the experience of the interruptions themselves (Elsweiler, et al., 2007) and possibly becoming an interruption of its own making.

Participants were asked to briefly record details of interruption events they experienced during the day whilst using their mobile devices to search the web, using a template we provided. More, specifically they were asked to answer the question: ‘What was the nature of the interruption?’. Diary entries were sent to the research team on a daily basis via email.

An open coding approach (Benaquisto, 2008) was used to analyse the nature of the interruptions described in the diary entries, in which categories and sub-categories were identified and developed inductively. The data were coded in multiple rounds by two of the authors working together in a synchronous manner so that individual entries could be carefully considered, new categories and sub-categories could be created as needed, and in these cases, previously coded data could be reconsidered in light of the updated categorical structure. The end result was that each interruption was assigned a category/sub-category that described the nature of the interruption. We note that, although we asked participants to only report on events that occurred when using their mobile phones, these will not necessarily have all been in a mobile context (e.g., people commonly use their smartphones to search the web at home). In a second round of deductive coding (Gilgun, 2019) (also conducted by two of the authors in a synchronous manner) we classified the data along two contextual dimensions using two sets of prior codes: whether the interruption was induced by internal or external factors, and whether the interruption was of the type that is normally unique to a mobile search setting or could happen anywhere. We acknowledge that such qualitative data analysis processes may introduce bias into the resulting taxonomy; however, our position as researchers in the field and the subjectivity the process brings is an asset, since it has allowed us to be sensitive to the broad ways in which mobile search can be interrupted.

Participants were recruited using convenience and snowball sampling through two social media channels: WeChat and WhatsApp. A total of 22 volunteers were recruited, two of whom dropped out of the research halfway through the process, resulting in 20 volunteers participating in the full diary study. Note that the research was conducted in the Information School of a major UK University, where a large proportion of the enrolled students during the 2020/21 year group were from China. Data was collected over a 10-day period between the 10th and 19th of July, 2021. Ten of the participants were in the UK during the study period, and ten were in China; half were female, and the modal age group was 19-24 years (13 participants were in this age group; 2 were over the age of 30). Any diary entries written in Mandarin were translated into English by a native Mandarin speaker, and were translated in such a way that the meaning of expressions was preserved but not necessarily the exact terms used.

We note that ethical approval was obtained from the first author’s university before proceeding with data collection.

Results

The study generated a total of 511 diary entries with a median of 26 entries per participant (min: 19; max: 30). The majority of participants (15) entered at least one diary entry for each of the ten days, the other five each missed just a single day. This suggests that our retrospective once-a-day diarying approach was successful in maintaining engagement. The data collected revealed a wide range of different types of interruptions experienced by the participants, which we have organised into a coherent taxonomy based on the nature of the interruptions (six categories and 29 sub-categories in total); and then classified each based on whether they were internally or externally-induced, and whether they were general or mobile-specific interruptions.

The results are shown in Table 1, which lists each of the categories and sub-categories, an example for each sub-category, the number of diary entries assigned to each, the percentage of each category’s entries that were identified as being internally-induced, and the percentage that were identified as being mobile specific.

Inductive categories and sub-categories

The most frequent category was New Information (23%), when an instant message, app notification, email, or alert popped up on the device interface, or an incoming phone call was received. These were all external and not mobile-specific, as they could just as easily have occurred when, for example, sitting at a desktop computer. We divided these into five sub-categories based on the source of the information: 34% of these, the most commonly-occurring, were phone calls; 28% were instant messages or SMS texts; 20% were pop-up app notifications (e.g., from news apps or delivery notifications); 13% were were triggered by the receipt of an email; and the remaining 6% were alerts (alarms). Previous work has noted the disruptive nature of such notifications and has shown that they can have a negative impact on user experience and performance (Sahami Shirazi, et al., 2014). Other research has shown that mobile phone users receive a large number of notifications every day (a median of 56) and that, with the exception of instant messages, the majority are ignored and/or dismissed (Pielot, et al., 2018), This may have lead to a under representation of such events in our diary logs.

Change in Environment (20%) were also very frequently observed. This category refers to instances where the participant was interrupted by something in the surrounding physical environment but not by another human being (such events are assigned to their own category ‘Other People’). Examples include: a sudden change in the weather, seeing something interesting and wanting to take a photo of it, being distracted by a loud noise or an interesting smell, or being distracted by an obstacle, animal, or task. The most frequent sub-category of such interruptions were those that required the participants to make a payment/complete a transaction. All of these entries were mobile-specific and most were external in nature. We note that, with the exception of transport-related noises (e.g., in Harvey and Pointon, 2018 and Hoggan, et al., 2009), such interruptions have not been the focus of prior research on mobile search to date but are likely to cause an interruption of considerable duration, and which will almost certainly require cognitive effort to resume the primary task. We note that this category does not include entries that relate to transport and mobility interruptions, unless it was an interruption caused by sounds made by a vehicle.

Transport and Mobility was, perhaps unsurprisingly, a common category (18%) and covered interruptions caused by bumpy vehicles, the need to find and use public transport, disembarking a vehicle, and the need to cross a road or intersection whilst on foot. These were all external in nature and mobile-specific. Mobile situations involving public transport have frequently been attested to and/or simulated in previous work (e.g., Harvey and Pointon, 2017; Hoggan, et al., 2009; Komaki, et al., 2012). Such situations have been shown to have a considerably negative effect on user interaction with mobile devices and result in frequent interruptions, decreased input accuracy and increased error rates (Hoggan, et al., 2009). However, research work has not yet explicitly considered the frequent interruptions caused by pedestrians having to stop walking and wait to cross the road, which was by far the most common sub-category (76% of entries).

In 14% of entries, participants were interrupted by Other People, including people asking for directions or other assistance, meeting a friend or colleague in the street, or other physical behaviour such as people arguing or being drunk. Five participants even reported being distracted by an attractive person walking by. These were all externally-motivated and the majority (88%) were mobile-specific, with exceptions being an interruption that could happen anywhere (e.g., ‘My child accidentally kicked me' [P12]). Some of the simulated interruptions used in a recent study were motivated by these types of interruptions (Hoeber, et al., 2021).

Participants frequently reported interruptions caused by Technology Issues, such as an unreliable or entirely disrupted signal, devices running out of battery power, dropping the device, and, in three instances, phones shutting down due to overheating. Of the 32 reported signal issues (weak signal, network connectivity), 14 were caused by the participant being on some form of transport. All technology issues were deemed to be external in nature and mobile-specific. Although these issues could arguably also happen in a home/work environment (e.g., a wired Internet connection failing or power disruption), these are generally rare occurrences. This corroborates the findings of Sohn, et al., 2008 and Aliannejadi, et al., 2019, and is problematic as such interruptions often make resuming the search challenging and may event lead to search abandonment.

Self-interruptions (11%), as the name suggests, is any instance where the factor interrupting the search was instigated by the participants’ themselves. This may have been, for example, a physical sensation such as thirst or pain, an urge to visit a bathroom, or a sudden recollection that the participant needed to call or message someone. This category also includes instances where the participant was listening to music on their phones whilst searching when a new song began playing that they did not like, and therefore, had to interrupt their search in order to skip. Only 7% of these were identified as being mobile-specific (e.g., ‘Long time looking down and searching causes pain, so want to stretch to relieve the pain' [P11]). Although such self-interruptions have been studied in a general context (Adler and Benbunan-Fich, 2013), they have yet to be studied in a mobile search context. Although this was the least frequently-occurring category in our data, it nonetheless appeared quite frequently and was experienced by all but one of our participants during the study period.

Deductive classifications

At an aggregate level, we considered the distinctions between how the interruptions were induced (internally or externally) and whether they are general or specific to a mobile context. We note that all percentages expressed in this section are in comparison to the whole set of interruption events logged by participants. Overall, most were externally-induced (446 or 87% of the total), and a large proportion (329 or 64%) of all interruption events were identified as being specific to a mobile context of use.

Among the externally-induced interruptions, more than twice as many were attributed to the mobile context (315 or 62%) than those identified as non-mobile (131 or 26%). This highlights the fact that being mobile while searching places the user in more risk of being interrupted by events occurring in the external environment than if they were in a static setting.

Although internally-induced interruptions were generally rarer (65 or 13%), most of these were not specific to the mobile context (51 or 10%). This indicates that the nature of the internally-induced interruptions is not dependent on mobile search setting, and could interrupt any activity or situation. The remainder (14 or 3%) were the rare cases of being internally-induced and unique to a mobile context (e.g., changes in surroundings - check map). Note that this same sub-category also accounts for the unusual case where a changes in surroundings was internally-motivated. The other non self-explanatory cases were those in the self-interruption category that were determined to be externally-induced; these were all instances where the user chose to skip a song that had been played by their device at random.

Limitations

The primary limitation of this study is with the retrospective recall, which is inherent to the diary study methodology (Bolger, et al., 2003). In particular, notable events are likely to be remembered more easily than commonplace occurrences (Sohn, et al., 2008). Given that we primed the participants to be observant of the interruptions to the search activities, we may consider these to be notable events, partially mitigating the impact of this limitation. As a result, while we acknowledge that there may be some degradation to the construct validity (the trustworthiness of the measurements), the trade-off of using a diary study method rather than other alternatives (e.g., detecting interruptions and forcing the participants to complete a survey) is that the ecological validity remains high (what was found is representative of reality).

Another potential limitation is the somewhat biased sample demographics in terms of ages, as the vast majority of our participants were young adults between the ages of 20 and 30. It would be beneficial in future work to study mobile interruptions encountered by older people, as they may use their devices in different settings that younger participants would. In this case, we might expect the relative frequency of various interruption types to differ by age but would expect the cardinality and content of the taxonomy of interruptions to be broadly the same. It would also be useful in future work to try to widen the demographic scope in terms of ethnicity/cultural background (many of our participants were Chinese).

Discussion and conclusions

The findings from this study provide a meaningful contribution to the literature on the complex environment of mobile search and interruptions. Regarding RQ1 (‘What is the nature of the interruptions that searchers experience in mobile settings?’), our data and analyses provide empirical evidence of the broad nature of everyday interruptions experienced by our participants. While the sample size and data collection method does not permit us to conclusively generalise about the frequency of occurrence for each (sub)category, the relative counts observed likely serve as useful proxies to this. We note, for example, that self-interruptions, technology-related issues and other people being distracting were fairly common in our data, yet are rarely simulated in existing experimental studies (e.g., Harvey and Pointon, 2017; Hoeber, et al., 2021; Hoggan, et al., 2009). These focus on transport and mobility-related distractions, which, although they also occurred frequently and may be especially disruptive, do not represent a large proportion of the whole range of different interruptions our participants encountered. As such, the categories and sub-categories identified in this study provide a taxonomy of mobile search interruptions that may inform the design of future survey-based research and observational studies. Furthermore, as the activity of searching the web served as an anchor for identifying when interruptions were occurring, there is nothing specific about the activity of searching that influenced the interruptions themselves. As such, we expect these findings to be transferable to interruptions that occur during general mobile device use.

Our findings also provide evidence as to the the extent of internally- v. externally-induced interruptions, and whether the types of interruptions are unique to a mobile context (RQ2 and RQ3). Such information can inform the design of future controlled laboratory studies and simulation studies, basing the interruptions used on what we have found rather than relying on intuition or self-observation. Search interface designers may use the interruptions that are unique to the mobile context as motivation to design and study new search interface approaches that mitigate the impacts of these types of interruptions (e.g., detecting transport and mobility interruptions using the motion sensors on the mobile device, and then highlighting the most recent queries and search results to enable search task resumption).

While this study has provided a taxonomy and classification of the interruptions that occur while searching the web using mobile devices, it did not collect other contextual information that would be helpful to understand the purpose of the search activity and the impact the interruption actually had on those activities. Further studies are needed to understand this bigger picture; controlled laboratory studies may then be designed to conduct experiments to assess whether the interruption categories lead to different impacts on the primary search tasks; and for those interruption types found to have the most significant impacts, new search interfaces may be designed and studied to mitigate these effects.

About the authors

Mengjia Lu completed a Master’s degree in Information System at the iSchool at the University of Sheffield in September 2021. During this time she worked on a dissertation about the interruptions encountered by people when using their mobile devices in everyday situations. The data collected, and some of the findings, during this dissertation project were subsequently used as this basis for this work. She can be contacted at lumengjia_19@163.com.

Morgan Harvey is a lecturer in data science at the iSchool at the University of Sheffield. He conducts research and publishes work in the fields of information retrieval, information behaviour, and recommender systems. Although his research is applied in a broad array of different contexts, at its core is the idea of helping to connect people to the information they need or want. He is the corresponding author and can be contacted at m.harvey@sheffield.ac.uk.

Orland Hoeber is a Professor in the Department of Computer Science at the University of Regina. His primary research interest is at the intersection of interactive information retrieval and information visualisation. He has an active research team working on the design, development, and study of visual and interactive software to support exploration, analysis, reasoning, and discovery in a broad range of information-centric domains including digital library search, mobile search, social media analysis, and sport analytics. He can be contacted at orland.hoeber@uregina.ca.

Acknowledgement

The authors would like to thank our participants for freely giving up their time to help us collect data for this work.

How to cite this paper