Information Research

Vol. 29 No. 1 2024

Practices and pain points in personal records

Matt Balogh, William Billingsley, David Paul, Mary Anne Kennan, and Melanie Robertson-Dean

DOI: https://doi.org/10.47989/ir291604

Abstract

Introduction. This paper reports the findings of a survey on personal electronic records management practices focussing on records that people deal with in their everyday lives at home. The aim of this research was to determine which personal electronic records practices were most effective in averting oversights and generating satisfaction in participant’s records management practices. This paper presents one stage of a broader design science research program.

Method. The research for this paper was conducted by means of an online questionnaire using Qualtrics software and participants were recruited through social media.

Analysis. Analysis was conducted using tabular analysis in SPSS, and Principal Component Analysis in R.

Results. The research found that there is a statistical relationship between the practices that respondents adopted with their personal electronic records management and their level of satisfaction with that process. For example, respondents who saved records on a computer or in the cloud reported higher levels of satisfaction with how they managed their personal records and experienced fewer adverse incidents such as losing documents or failing to pay bills on time.

Conclusion. The paper concludes by identifying some specific personal records management practices that are likely to improve satisfaction with that task, such as saving and sorting records that need to be retained outside of email in a structured filing system.

Introduction

People and households receive and manage many different kinds of information, documents, and records as they navigate their everyday lives. In previous work, we conducted an exploratory virtual guided tour study in which eighteen respondents demonstrated and explained to researchers how they manage their personal records, such as bills, warranties, statements or any other records they might choose to keep. This revealed a variety of practices and roles that people take on within households regarding records management. Our previous research identified patterns of behaviour, such as leaving all one’s email records in the in-box or sorting records into folders within email or elsewhere, such as on a computer or in the cloud (Balogh et al., 2022b). Some of these practices are similar to those found in personal information management research conducted in the workplace (Bergman & Whittaker, 2016; Henderson & Srinivasan, 2011; Oh, 2017, 2020), while other aspects of personal records management are particular to the home environment (Balogh et al., 2022b). In this study, which uses a survey with an expanded group of respondents, we seek to understand how particular practices and pain points in personal electronic record management relate to each other.

This paper is set within a broader project regarding personal electronic records management, including exploratory research, conducted with the intention of developing better ways of providing digital support for personal records management in the home. A design science approach was used for the broader project. Design science is a research methodology which uses the process of designing and evaluating an artefact with a stakeholder community to uncover insights about the problem domain (Weber, 2018). The research in this paper contributes by providing information to inform the future development of a personal electronic records management application in a future stage.

We use the phrase personal electronic records management to reference practices at home and compare these to personal information management practices that have been the subject of research predominantly conducted in workplaces, and particularly amongst knowledge workers. Personal electronic records management does not include information and documents that one deals with in the course of work (Balogh, et al,, 2022a; Balogh et al., 2022b).

The management of personal records relating to everyday life is important for two reasons. Firstly, so that people are able to re-find information and documents that they need, when required, even if they did not previously anticipate needing that record (Balogh et al., 2022a, 2022b). Secondly, the management of personal electronic records can help people avoid oversights, such as missing paying bills, or forgetting to renew insurance. For example, in Australia, drivers were fined millions of dollars because of a reduction in the reminder process for new vehicle registrations (Mayoh, 2014), and research has found that unused subscriptions and forgotten outgoings could be costing people over $1,000 per annum (Potter & Landauro, 2023).

We refer to the information managed at home as records (from the records management literature) to reference the variety of information and documents that people retain, such as emails, bills, and notes (Balogh, 2022; Balogh et al., 2022a; Shepherd & Yeo, 2003; Yeo, 2018). The word records also accommodates the diversity of formats in which information and documents may manifest. Building on a prior qualitative stage, the research reported here explores how a larger group of respondents manage their personal electronic records and their attitudes towards their current personal electronic records management. This knowledge will assist the development of automated personal electronic records management systems designed to reduce oversights and ensure that records can be re-found when required. Previously, prototypes have been designed and tested for workplace personal information management, such as InfoMesh (Krishnan, 2010), Placeless Documents (Dourish, 2001; Dourish et al., 2000) and WorkspaceMirror (Boardman & Sasse, 2004). Research conducted on these prototypes contributes to the understanding of how applications can assist in records management. Further research is required on the specific circumstances of personal records dealt with at home.

Related work

The field of personal information management researches the practice and study of how people acquire, save, and re-find information and documents. In essence, it is about ‘keeping found things found’ (Jones, 2008).

Our research draws on the themes identified in the field of personal information management, which includes research in the workplace and academic settings. We use the phrase personal electronic records management to refer to practices for managing everyday records, such as bills, warranties, insurance renewals or health records and compare these to personal information management practices that have been the subject of research conducted in workplaces, and particularly amongst knowledge workers and students (Balogh, 2022; Balogh et al., 2022a, 2022b; Buttfield-Addison, 2014). While we acknowledge the blurring of boundaries as working from home becomes more common, we do not address the management of information relating to work at home, nor artefacts that may be catered for by specialist software, such as photo or music management applications. We do, however, draw on the literature of research into personal information management as a point of comparison in exploring people’s management of personal records at home.

Email as a personal information management tool

Email is often used as a de-facto personal information management tool (Bellotti et al., 2003; Ducheneaut & Bellotti, 2001; Whittaker et al., 2006, 2007b). Adoption of an email in-box as a defacto to-do list in the workplace was validated by research in 2014, which found that 94% of survey respondents had sent emails to themselves (Buttfield-Addison, 2014, pp. 155-156), either as reminder notes or containing URLs to check later (Bruce et al., 2004, p. 4; Jones, et al., 2002, p. 394). Oh and Belkin (2011) observed that email was chosen for keeping various forms of personal information to be able to re-find it, keep a record or archive of that information, or as a reminder.

However, similarly to personal information management, research on email usage has been largely conducted in workplaces, and mostly amongst knowledge workers and students (Balogh et al., 2022b; Buttfield-Addison, 2014). The question arises: do people similarly use their home email in-box as a to do list, and do they use home email accounts for reminders for personal records management matters? Using email for these purposes comes with inherent risks, such as an overload of emails that can make re-finding required information challenging (Whittaker et al., 2006; Whittaker & Sidner, 1996).

Organising behaviour

Research in personal information management in the first decade of this century focussed on, among other things, electronic information and records and people’s organizing behaviour. This organizing behaviour can be described in terms of how much information a person keeps, and how those records are retained (Jones, 2008, p. 122). There is great variation in how people retain information, ranging from ‘keep everything’ (given that digital storage is plentiful) to ‘keep nothing’ (relying on items being retrievable online) (Jones, 2008, p. 124). In the workplace, people can minimise the amount of electronic information and number of documents they keep on their local computer by relying on corporate intranet and storage and other external sources, thereby adopting a personal keep nothing approach. In the case of personal records at home, a keep nothing approach relies on being able to re-find records held by service providers such as banks, employers, and taxation departments. Jones noted three risks to the keep nothing approach at work. First, one may not be able to find things because they have not been labelled by others with the same terminology that the person trying to re-find the documents would use; second, people risk forgetting that the information or item is available; and third, ‘information out there is not under our control, and could change or disappear entirely’ (Jones, 2008, p. 135).

The ways in which people organise electronic files and the challenges people face in so-doing continue to be researched, building on the earlier literature and using similar language to refer to the way people might file paper documents on their desk (Alon & Nachmias, 2022; Brackenbury, Harrison, Chard, Elmore, & Ur, 2021; Hellmich & Dinneen, 2023; Ofer Bergman, et al., 2022). Just as papers can be stacked in one big pile or sorted into clearly labelled folders on a physical desktop, electronic records may similarly be saved in similar ways, neatly organised into folders on a computer or just kept in the order received, for example. The personal information management literature has used the following descriptors for different organising behaviour:

• Pilers, also referred to as no-filers, who rarely file items into sorted folders, instead just allowing them to pile in paper or electronic heaps (Henderson, 2009a, 2009b; Whittaker & Hirschberg, 2001);

• Periodic filers, also variously known as spring cleaners, who file groups of items from time to time (Jones, 2008); and,

• On-the-fly filers, referred to as filers, who file everything as they go (Oh & Belkin, 2011).

The personal information management literature has also identified similar filing and piling practices within email. People can leave all their in-bound email in the in-box or sort it into folders (Bälter, 1997, p. 22; Whittaker & Sidner, 1996, p. 280). Even though email technology has moved on since the 1990s and people use multiple devices for their email, more recent studies show that these practices still occur (Balogh et al., 2022b; Henderson & Srinivasan, 2011; Oh & Belkin, 2011).

The use of the email in-box in lieu of a to-do list has been described as an organising strategy. Ducheneaut and Bellotti described email as ‘the killer application of the Internet’ (Ducheneaut & Bellotti, 2001) . Yet this strategy is often only effective for constant ‘filers’. In the case of ‘pilers’ and ‘periodic filers’, email in-boxes can become cluttered, making it difficult to recognise actionable items (Whittaker, 2011). More recent research has identified filer and piler behaviour occurring with personal electronic records at home (Balogh et al., 2022b). The question arises as to how these practices might impact on home personal records management practices, such as the overlooking of bill payments or the ability to re-find records when required. Affordances such as being able to search for and search within do, however, differentiate electronic records from hardcopy analogies.

The use of folders

A related question is how these records are structured. Research has explored how some people saved documents within a hierarchical folder structure (Henderson, 2004, 2005; Oh, 2012; Russell & Lawrence, 2009). Additional research investigated whether emails were retained in a similar folder structure (Gwizdka, 2004; Henderson, 2009a, 2009b; Whittaker & Hirschberg, 2001; Whittaker & Sidner, 1996). It was observed that systematic filing and organising practices were becoming increasingly important for users to be able to re-find the documents they required and to be reminded of the tasks relating to those records as they fell due (Bondarenko et al., 2010; Whittaker et al., 2006). It has been found that in using email as a tool for personal information management, people adopt similar behaviour that they use when managing their paper documents and files, such as whether they sort emails into folders or leave all their emails in the in-box (Bellotti et al., 2003; Grbovic, et al., 2014; Whittaker et al., 2011). Recent research found that tax documents, appliance receipts, and manuals were more likely to be retained in hardcopy, while payslips, bank statements and travel documents were more likely to be retained electronically (Balogh, 2022).

Behaviour and pain points in records management

It is at the moment when people try to re-find information that they ‘sometimes painfully’ come to the realisation of shortfalls in their personal information management practices (Jones, 2008, p. 81). Research continues to find that people are not satisfied with how they manage their personal information (Alon & Nachmias, 2020b), with this lack of satisfaction eliciting language such as ‘anxious’, ‘frustrated’ and even ‘desperation’ (Alon & Nachmias, 2020a). Alon and Nachmias underlined the importance of studying not just the behavioural facets of personal information management, but also people’s ‘perceptions and desires’ (2020b, p. 8) to understand the affective aspects. In 2022, Alon and Nachmias published further analysis of this survey data, concluding that feelings of anxiety and loss of control were predictors of organising behaviour; a feeling of self-efficacy was an even stronger motivator.

Karger (2007, p. 128) observed that users are frustrated by having to look in different applications for information because they may not recall how they had saved that item, and suggested that a more unified search and retrieval facility across multiple applications would make records less ‘hard to find’. Alon and Nachmias (2022, p. 8) suggested two ways of improving the experience: firstly, training to improve users' skills and secondly, that the designers of personal information management tools need to ‘reduce negative feelings by constructing affective-sensitive digital platforms…’. They concluded that ‘people’s feelings of enjoyment generated by interactions with personal information’ most effectively motivated actions to ‘improve its management’ (2022, p. 9). In a more recent paper, Alon and Nachmias (2020a) reported ‘high levels of anxiety’ with respect to possible loss of personal information or other digital failure.

Ineffective personal information management (where people cannot readily re-find information that they require) elicits negative feelings, and effective organised personal information management (where people can quickly and easily find the information they need) is rewarded by feelings of satisfaction that motivate enduring practices (Alon & Nachmias, 2022).

In summary, personal information management research has used the physical analogies of piling, periodic filing or filing on the fly to describe processes used for electronic documents. It has explored the decision making about how people decide when and if to delete records and where information is retained, such as within email, or saved in files and folders on a computer. It has been found that managing electronic records can be painful and elicit anxiety. The research reported in this paper addresses the prevalence of various behaviours and perceptions with regards to personal records, and how these may relate to pain points and self-assessed levels of satisfaction with how respondents managed their personal records relating to everyday life at home.

Method

This study is situated within a broader design science project, exploring how technology might better support people’s personal electronic records management practices. Design science is a research methodology in which researchers use the process of designing and evaluating an artefact with a stakeholder community to uncover insights about the problem domain (Weber, 2018). The whole project consisted of four stages. Stage one was a qualitative study and used a virtual guided tour method to understand how and why electronic personal records in the home are retained and managed (Balogh et al., 2022b). The study reported here is stage two, using an online survey with a larger group of respondents to refine our understanding of people’s practices and how they relate to pain points or self-reported dissatisfaction with their personal records management. The purpose of the online survey sample was to explore the topic in depth with more people rather than to enable strong claims about generalisability. The larger sample also enabled the use of analysis techniques such as Principal Component Analysis to refine our understanding of respondents’ practices and enabled the relationship between participants’ responses to be explored in more depth.

The findings from both stages will be used to inform stage three, the development of an instantiation, which is an application designed to support in-home personal electronic records management. Stage four will be a testing and evaluation of the application. This first test will lead to a cycle of development that progresses from discovery and description to refinement, as we progressively identify and define the factors relevant to the topic, in keeping with design research as described by Cash et al. (2022).

Sample

Our literature review identified that earlier personal information management research focused primarily on knowledge workers and students. Since our research focuses on every-day personal records management at home, we sought a broad sample that would not overly reflect knowledge workers and students. The chosen sampling technique reflects this stage of our theory building rather than a level of abstraction and generalizability (Cash et al., 2022). We also sought to include respondents from more than one country, although our intent is not to undertake a country-by-country analysis of people’s practices.

Respondents were recruited by means of posts placed in social media by the lead researcher, inviting contacts and their extended networks to complete the survey. Social media was selected as the sampling tool and because it can provide a potentially diverse sample (Vitak, 2016, p. 9), and pragmatically because it provides an effective low-cost tool to reach a sample that included international respondents and people who were not students or knowledge-workers. The survey was open to anyone aged 18 and over and anonymous. Contacts were encouraged to share the survey invitation, thereby establishing a snow-ball sampling effect to broaden the sample further (Baltar & Brunet, 2012; Kosinski et al., 2015).

The study sought to include participants from more than one country to achieve a diversity of experiences and opinions. In total, 333 useable responses were included in the analysis, comprising 69 males and 258 females. Responses came from a variety of countries: 245 lived in Australia and 88 lived in other countries, predominantly the UK (26), US (23) and Switzerland (13). The remainder lived in countries, including Bahamas, New Zealand, Laos, Lebanon, Monaco, Portugal, Singapore, Sweden and Zimbabwe. This variety, many with low response counts, precludes a country-by-country analysis of the data.

The survey instrument

The questionnaire design was informed by prior studies on personal information management in the workplace among academic communities (Alon & Nachmias, 2020a, 2020b; Bergman et al., 2020) and at home (Balogh et al., 2022a, 2022b). The survey consisted of questions about how people currently manage their personal records, as well as Likert scale questions relating to respondent’s behaviour and attitudes. The core questions were designed to determine how the respondents currently managed their personal records. Respondents were also asked to self-assess how satisfied they were with their personal records management, as well as being asked about negative (pain point) experiences, such as missing a bill payment, and their preferences, for example, whether they would prefer to keep records outside of their email.

The survey comprised three groups of questions, The first were a set of pre-coded nominal questions with a multiple-choice set of responses. The second group of questions elicited respondents’ attitudes towards their current personal records management, six behavioural experiences, and seven attitudes to personal records managements. These questions were Likert items using a five-point agreement scale (strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, strongly disagree). The scale was applied to a set of statements with varying degrees of positivity and negativity, allowing for logic checks between responses. A not applicable option was also provided. Cases where the respondent had not answered all the Likert scale questions were removed from the data set, as incomplete data would affect the results of the analysis approach described in the next section (Goretzko, 2022). In cases where respondents did not select a pre-coded response and only provided a text response, these were manually coded. In the data set, all such responses that were deemed relevant to the question (rather than, for example, apologies for not answering) were able to be coded against the pre-defined codes. Responses were only excluded from the analysis in cases that the analytical method could not be applied if there was missing data.

Analysis

The analysis comprised tabulations and factor analysis. Comparative percentages among sub-groups of the sample are only reported if statistically significant, calculated by means of a Z Test using the Bonferroni correction (Napierala, 2012; Weisstein, 2004). Reported correlations are derived from the SPSS Pearson correlation matrix.

Principal Component Analysis was chosen for its ability to consolidate the findings from a larger number of variables. Principal Component Analysis reduces the dimensionality of a dataset, identifying patterns in high-dimensional data, and uncovering latent variables that can explain the observed variation in the data. Principle Component Analysis consolidates standard deviation, covariance and eigenvectors into a graphical format comprising a correlation circle (Holland, 2008; Karamizadeh et al., 2013).

Limitations

A limitation of this study is that neither the sampling approach nor the sample size permit generalisation to the overall population, nor comparative analysis between different populations. Further research may find the results transferable to different situations.

Findings

In this section findings are reported as follows: firstly, in the first four sub-sections, we report the practices that respondents used in managing their personal records. Thereafter we compare these results to some of the literature on comparable workplace practices.

The last three sub-sections of the findings report a deeper level of analysis that was conducted on the self-reported levels of satisfaction with personal electronic records management and the statistical relationship between practices and levels of satisfaction to identify practices that appear to derive more satisfactory outcomes for the users.

Email and the cloud for personal electronic records

Figure 1 shows the locations in which respondents managed their personal electronic records, such as within email or in the cloud. Two-thirds (69%) of respondents reported using their email software to manage their personal records, compared with 29% who saved records to their computer, and 20% who saved their records to the cloud. Additionally, 8% of all respondents said that they saved items on a mobile phone or tablet. As the percentages indicate, some respondents used more than one location. Among respondents aged 18-39, 82% of respondents used their email to manage their personal electronic records.

Figure 1. Location of personal records

The use of folders to organise personal electronic records

Respondents were asked to specify how they organised their records such as whether the records were all within one folder (for example, the email inbox), or sorted into multiple folders on the computer or in the cloud. Figure 1 shows that of the 69% of respondents who used their email for their electronic records, approximately half of these respondents used email to manage their personal records and relied exclusively on one folder(i.e., their inbox), half (50%) used multiple folders within their email software and 7% used a combination of email in-box and folders (not shown in Figure 1).

Amongst the 29% of respondents who saved records on their computer, 77% sorted their records into folders (22% of all respondents), compared to 27% (8 % of all respondents) who saved everything into only one folder, leaving a small overlap of 4% who sometimes sorted items into folders. Similarly, approximately three quarters (71%) of respondents who managed their personal records in the cloud reported sorting records into folders, compared to 29% who left all their records in one folder in the cloud, such as Google Drive, Dropbox and iCloud.

Amongst respondents who used both email and another location to save their personal records, those who sorted records into folders on a computer or in the cloud also tended to sort their email records into folders. For instance, 56% of respondents who used email for personal records management sorted their email into folders. However, of respondents who saved and sorted their records on their computer into folders, 63% sorted their email records into folders and only 37% left all their records in the in-box. This compares to the finding that, among the 22% of respondents in this survey who only retained their personal records in email (meaning that they did not save records on a computer or in cloud storage), none sorted their email records into folders. Among the 20% of respondents who saved their personal records in the cloud, 72% sorted the records into folders, while the 28% of respondents who saved records on a computer, 77% sorted those records into folders. Across users of email, computers, and the cloud for personal records, 56% sorted items into folders and 47% said that they left all the records in one folder or in the in-box (with a slight overlap of practices).

Thus, respondents who saved personal records on a computer were more likely to sort the contents into folders (77%), followed by users of cloud storage (72%), while respondents who used only email for storing their personal records were much less likely to sort the records into folders (39%). From this we can see a clear spectrum of personal records management behaviour, ranging from the least active, (those who leave everything in their email in-box) through to respondents who save items in the cloud or on a computer; these latter who are significantly more likely to organise those items into folders, both outside and inside their email.

Interest in organising email into folders: filers and pilers

Respondents in the survey who left their personal records in their email in-box (n=131, 39%) were asked whether they would prefer to organise their email into folders, if there were an automated system to assist them with this task. Fifty-five percent responded that they would. A further 13% said that ‘it depends’, citing concerns as to how well this would be done. Two respondents expressed concern that they might overlook a task required in response to an email if emails were filed automatically:

I would be worried I missed something if they were automatically filed but apart from that would be good. Maybe to have a breakdown of new emails filed sent to my in-box weekly.

Happy to have emails sorted into files but concerned I would miss emails if they didn’t come through in-box.

Conversely, one respondent speculated that having their emails automatically sorted would help them prioritise tasks:

I prefer them sorted into categories, so I can choose which I deal with first.

These respondents’ comments reflect complex longstanding considerations of automation bias in human factors research. Kaber and Endsley (1997) and Parasumaran and Riley (1997) observed that although human monitoring of automated systems can be made very efficient and effective, situational awareness can sometimes be reduced and over-reliance on automation can appear. The balance of respondents, 32%, said they would prefer to leave all their email in the in-box.

Keeping personal records

Respondents were asked to nominate one preferred location where they would most like to keep their personal records (from the same list of possible locations). Respondents tended to nominate a more organised format than their current behaviour: only 8% would leave everything in their email in-box and 23% would choose email folders. A similar proportion, 23%, nominated the cloud sorted into folders, and 20% nominated saving their records on a computer.

Figure 2 shows the proportion of respondents selecting each location as their one preferred place to save their personal records. In total, 32% selected their email as the preferred place for their personal records, of whom only 26% (or 8% of all respondents) said that keeping their personal electronic records in their email was their preferred location.

Figure 2. Preferred location for personal records (n=333)

Of the respondents who only kept personal electronic records in email, 38% nominated a different medium as their preferred location; particularly in the cloud (22%) or on a computer (14%). Among respondents who currently do not sort their records into email, 69% preferred to sort their records, 33% preferred to keep their records in the cloud rather than email, and 22% preferred to save them on a computer.

Comparison of home and workplace records management

Respondents were also asked whether they managed their personal electronic records at home differently to the way they managed records at their workplace or place of study, given that workplaces, schools, and universities may influence how employees or students manage email and other records. Of the respondents in this research who also dealt with email in the course of work or study, 65% sorted their in-bound work or study email into folders once it was dealt with, while only 35% left all their email in the in-box compared with only 39% of respondents who sorted their home email into folders. Among respondents who did not sort any of their personal records into folders at home, approximately half (51%) did sort their email into folders at work or study. This suggests a complex relationship between personal electronic records management practices at home and personal information management practices in the workplace. Although many of the same themes and considerations appear in both, many respondents adopt different practices for each domain.

The reasons for this difference are not available from the study data, however we can speculate on three possible factors that might bear future investigation. The practices that someone adopts for their management of work and study records would be influenced by the needs and requirements of their employer, place of education, or colleagues, whereas their home records have fewer outside influences. A second possibility is that the choice of personal electronic records management practice might be influenced by factors of how frequently or soon they expect to use the record, i.e., that they are evaluating an effort-benefit trade-off in how much attention they invest in sorting their records.

Oversights in personal records management

Respondents were asked about five examples of potential negative outcomes from ineffective personal records management:

• Inability to find documents when needed

• Missing bill payments

• Failing to renew a vehicle registration

• Failing to cancel a subscription that is no longer required, and

• Failure to claim small tax deductions.

One-third (33%) of respondents said that they often could not find the documents they required, 38% said that they had missed a bill payment in the last three years, 14% had missed a vehicle registration, 60% had failed to cancel a subscription that they no longer required, and 49% missed claiming small tax deductions.

Overall, 81% of respondents agreed that they had experienced at least one of the prompted negative experiences. Respondents who only used their email to store personal records were marginally more likely to have had one of the negative experiences (87%), compared to respondents who saved their records in the cloud (79%) or on a computer (80%). Together, these suggest that negative experiences, or pain points, in personal electronic records management practices are prevalent at home.

Respondents were also asked if they downloaded statements and documents that were sent to them as a hyperlink to a URL within a portal (as opposed to email attachments). We have adopted the terms pull for items that the recipient has to actively retrieve from a link or within a portal, and push for documents sent as email attachments (Balogh et al., 2022b, p. 14). In response to this question, only 13% of respondents said that they always actively saved these documents by downloading pull records, despite 99% of respondents indicating that some of their personal records were only shared with them in this way. Conversely, 40% of respondents said they sometimes downloaded documents that required access through a portal and actively saved them, and 47% said they never downloaded these items. Respondents who saved personal records on a computer or in the cloud reported being slightly more likely (18%) to download pull documents, while respondents who relied exclusively on their email in-box were the least likely to save pull documents, at approximately 7%.

When asked if they had ever needed a document that they could no longer access online, 47% of respondents responded that they had had this experience and provided 113 examples of inaccessible items. The most commonly cited items that respondents were unable to access were bank statements, pay slips, and pay summaries.

Oversights and personal electronic records management behaviour

The incidence of oversights (for example missing a bill payment or being unable to find a document when required) varied among respondents with different personal records management behaviour patterns. In order to better understand the implications of this, respondents were also asked about their level of agreement with regards their practices and various aspects of their records management described below. This allowed for comparisons to be made between the satisfaction ratings provided by groups of respondents using various personal electronic records management practices. Significantly fewer respondents who managed their personal records exclusively in hardcopy format were satisfied with their current strategy (45%) in contrast to respondents using an exclusively electronic process, where 65% were satisfied, indicated by a chi-square χ²= 5.12 and p-value of 0.0236 (Preacher, 2021). Hardcopy record-keepers were also more likely to have missed a bill payment (52%) than the electronic systems users (39%) and 50% of hardcopy records users said that they often could not find a document when needed, compared to 22% of respondents who saved documents on a computer and 23% of respondents who saved records in the cloud. Respondents who saved their personal records on a computer or in the cloud were consistently less likely than those who relied on email to retain their personal records to have experienced negative events such as missing a bill payment, failing to cancel an unwanted subscription or not being able to find a required document. Respondents who sorted their records into folders were also consistently more likely to be satisfied with their personal records management than those who did not sort records into folders.

Table 1 reports the proportion of respondents who strongly or somewhat agreed with the statement that they were satisfied with their records management, amongst those with different records management practices. This table demonstrates that, regardless of where records were stored, respondents who sorted their records into folders were more likely to be satisfied with their current records management arrangements than respondents who did not sort into folders. Among respondents who kept their records in the cloud, overall satisfaction was relatively high (79%), irrespective of whether they were sorted into folders or not. The p-value of Pearson Chi-Square test shows that respondents who sort their personal records into folders were significantly more likely to be satisfied with their personal records management.

Where respondents saved records	Respondents who sorted records into folders	Respondents who did not sort records into folders
(Base = 221 who agree strongly/ somewhat that they are satisfied with their records management) n=	136	85	p value of Pearson Chi-Square <.001
In email	71%	52%
On a computer	82%	50%
In the cloud	83%	75%

Table 1. Incidence of satisfaction with personal records management

This brings us to the question as to whether certain keeping and organising behaviours can be associated with an improved ability to find documents when required. To address this, the behaviour patterns were grouped according to where respondents saved their records, such as in email, on a computer or in the cloud, and whether they sorted the records into folders in each location. The ‘net’ groups shown in Table 2 comprise the de-duplicated group of respondents in each category. For example, if a respondent sorted their email into folders and saved records on a computer sorted into folders, they are included in the group ‘net save sorted into folders’ only once.

Table 2 compares the proportion of respondents who used various methods for managing their personal records. Percentages in the first column are as a proportion of all respondents. The first and second column shows the proportion of respondents that saved records in each of the ways listed. In the second and third columns, the top row shows the 109 respondents (33%) who said that they often can’t find documents when needed. The second column of figures (V%) shows the proportion of respondents using each of the listed methods for managing their personal records as a percentage of respondents who often couldn’t find documents when needed. The third column (H%) shows the proportion of respondents who often could not find documents when needed, as a percentage of everyone using each method for managing their personal records. For example, the row labelled ‘Net save sorted into folders’ shows that 56% of all respondents save personal records sorted into folders, but amongst the subset of those that often can’t find documents when needed, only 45% save records sorted into folders and, conversely, only 27% of respondents who save their records into folders often have difficulty finding necessary documents. The fourth column on the far right uses the p-value of Pearson Chi-Square tests to determine whether there is a statistically significant relationship between respondents’ inability to find documents when needed and each of the behaviours listed. This shows a statistically significant correlation between respondents who saved personal records on a computer with reduced difficulty in re-finding documents when needed.

How respondents save records	All respondents	Proportion often can not find documents when needed V% H %		p value of Pearson Chi-Square
Everyone	n = 333	109 33%
Net save sorted into folders	56%	45%	27%	.007
Electronic Only	39%	39%	32%	.833
Net don’t sort into folders	47%	50%	35%	.398
Net save on computer	29%	18%	21%	.004
Net save in cloud	20%	14%	22%	.043
Email only	22%	28%	41%	.085
Hard copy only	13%	19%	50%	.011

Table 2. Behaviours by frequent inability to find required documents

Satisfaction with keeping and organising of records

When asked to what degree respondents agreed or disagreed with the statement indicating that were satisfied with how they managed their personal information and documents, 63% of respondents said they strongly agreed or somewhat agreed that they were satisfied (16% strongly agreed and 48% somewhat satisfied).

Table 3 compares the proportion of respondents who said they strongly agreed or somewhat agreed with the following statement: ‘Overall, I am satisfied with how I manage my personal information and documents’. In order to be able to compare positive and negative responses, we converted the ordinal Likert scale into a nominal response set by combining the strongly and somewhat agree categories and the strongly and somewhat disagree categories (Kitchenham & Pfleeger, 2003). We refer to the respondents who agreed strongly or somewhat agreed with this statement as being satisfied, and those respondents who disagreed strongly or somewhat disagreed with the statement as dissatisfied with their personal records management. The first column of figures shows the number of respondents to whom the labels on the left applied. The second column of figures shows the percentage of each group that were satisfied with their personal records management.

The rightmost column shows the p-value of Pearson Chi-Square tests, indicating whether there is a statistically significant relationship between respondents’ satisfaction with their personal records management practices and each of the behaviours listed on the left. Practices which were found to have a significant positive relationship with satisfaction are indicated with a ‘+’ symbol in the direction of correlation column, those where a significant negative relationship was found are indicated by ‘-‘ symbol, and those where no significant relationship was found are blank in the direction of correlation column. The observed correlation between practices and satisfaction levels should not be interpreted as evidence of a causal relationship; further studies are warranted to establish causation.

Respondents who saved their personal records in the cloud were the most likely to be satisfied (81%), followed closely by those who saved their records on a computer (75%). Respondents who sorted their records into folders (74%) were more likely to be satisfied than those who didn’t (54%). Respondents who relied on their email to store their personal records and respondents who only kept hardcopy records were the least likely to be satisfied, at 49% and 45% respectively.

How respondents save records	n=	% Satisfied	Direction of correlation	p value of Pearson Chi-Square
Everyone	333	63%
Net save in cloud	67	81%	+	.001
Net save on computer	95	75%	+	.006
Net save sorted into folders	185	74%	+	.000
Electronic only	131	65%		.642
Net don’t sort into folders	157	54%	-	.001
Email only	73	49%	-	.005
Hardcopy only	42	45%	-	.009

Table 3. Satisfaction by practice groups

Behaviours and attitudes related to personal records management

Table 4 shows the results of two sets of questions asked in the form of a Likert scale, from strongly agree through to strongly disagree. The first group of phrases related to respondent experiences, while the second group of phrases related to respondents’ attitudes regarding personal records management. The questions were all asked in this format in order to allow principal component analysis correlation scores, as shown in the first numerical column. Scores show the correlation between responses for each phrase compared with the responses given about the respondent’s overall satisfaction with their current personal records management strategy. The second numerical column shows the proportion of respondents who agreed strongly or somewhat with each phrase. The results in the “Satisfaction with current strategy” column are all negative, indicating that agreement with these phrases was inversely related to overall satisfaction. Each group of phrases, those relating to behaviours and those relating to attitudes, are sorted in descending order of inverse correlation with overall satisfaction.

Descending order of inverse correlation within each group of phrases Phrases are simplified in table for presentation purposes		Overall Satisfaction with current strategy	Agree strongly/ somewhat agree
Overall Satisfaction with current strategy		1.000	63%
Behaviours	Often can’t find document when needed	-0.564	33%
	Failed to cancel subscription	-0.315	59%
	Missed email bill in past 3 years	-0.303	37%
	Don’t claim small tax deductions	-0.294	48%
	Missed vehicle registration	-0.192	14%
	Keep all emails	-0.053	57%
Attitudes	Prefer to not think about where to file things	-0.344	74%
	Would benefit from tool for tax reporting	-0.261	42%
	Appreciate easy bill tracking	-0.216	88%
	Using email as to-do list would be better	-0.196	76%
	Would use less paper if more confident system	-0.160	46%
	Prefer to store outside email	-0.160	66%
	Want to reduce paper usage	-0.086	79%

Table 4. Correlation between behaviours and attitudes

These findings show that ‘often not being able to find records when needed’ was the phrase least correlated with satisfaction. ‘Not being able to find documents when needed’ is correlated with dissatisfaction with respondents’ personal electronic records management. Failure to cancel a regular subscription no longer required and missing payment of an email bill in the last three years are also correlated with dissatisfaction with personal records management.

Among the attitudinal phrases, the group of respondents who agreed that they preferred not to think about where to file things was the group most correlated with dissatisfaction. The results also show that some of the items that had a strong level of agreement were not necessarily highly correlated with dissatisfaction. For example, 88% of respondents agreed (strongly or somewhat) that they would appreciate a tool that helped them track their bills, but this was not as strongly correlated with dissatisfaction as some other phrases. Similarly, 79% of respondents said they wanted to reduce paper usage, but this was not significantly correlated with dissatisfaction with the management of one’s personal records.

The relationship between behaviour and attitudes to the management of personal records

Further analysis was conducted to determine whether there was a relationship between how respondents managed their personal records, and how satisfied they were with that process. These patterns are illustrated by the presentation of the PCA correlation analysis in the form of a correlation circle (Figure 5). This analysis identifies patterns in the responses across multiple variables, allowing for multiple factors to be summarised to their common themes (Smith, 2002).

Figure 3. Correlation between behaviours and attitudes related to the management of personal records

Figure 3 shows the correlations among the Likert scale phrases in the form of a circle. Each arrow shows the relativity of that item with respect to other items in the analysis. The items represented by arrows close together are more correlated than items represented by arrows that are further apart. Items with arrows pointing in opposite directions are negatively correlated. The group of items indicated by the arrows in the top right of the circle comprise phrases that respondents agreed with regarding missing payment of an email bill, failure to cancel an unwanted subscription, failure to claim small tax deductions and overlooking a motor vehicle registration renewal. These all reflect experiences, and collectively we label these as `personal records management oversights’. In total, one third (34%) of respondents had experienced at least three of these experiences.

The cluster in the lower right of the circle comprises items related to wishes that respondents had with regards to their personal electronic records management, such as that they would appreciate a tool that made it easier to track bills, they would benefit from a tool that helped with their tax, and they would like to sort their email such that they could use their email in-box as a to do list. This group of attitudes has been labelled ‘wish personal records management was easier’. Three quarters (76%) of respondents agreed with three or more of the phrases in this group. While the personal records management oversights cluster and the personal records management wish cluster overlap significantly in terms of incidence of respondents, the pattern of responses shows these as two distinct groups, both inversely correlated with satisfaction with one’s personal records management overall.

Discussion

There are clear correlations between patterns of personal electronic records management and certain negative experiences, as well as with the respondents’ self-reported level of satisfaction with their personal electronic records management practices. Confirming the recent findings of Alon and Nachmias (Alon & Nachmias, 2020b), the research found differences between the practices our respondents have and the practices they would like to have. Our research also found differences between how they manage work or study records versus how they manage their other records. For example, respondents were more likely to leave records in their email in-box at home than at work, and less likely to sort records into folders at home than at work.

From a design perspective, this suggests that a system that could conveniently support “better” practices might improve people’s feeling of satisfaction with the process and reduce anxieties (Alon & Nachmias, 2020a; Tugtekin, 2022). If people find that they miss fewer bills, avoid paying for subscriptions that they no longer need, avoid driving an un-registered car and generally avoid other oversights in their personal electronic records management, this may be of benefit. Similarly, if a system of personal records management were to make it easier to track bills, prepare information for taxation and be aware of what items are on one’s to-do list by using one’s email in-box as a to-do list, people may find utility in it. If such an automated system can help bring their personal electronic records management practices closer to what respondents say they would like them to be, they may perceive less of a gap between their desired practices and their actual practices (i.e., a lower self-reported level of dissatisfaction).

The data suggests some practical ways in which records management processes might be improved for respondents in order to achieve higher satisfaction with personal records management, specifically:

• By saving personal records outside of email, such as on a computer or in the cloud;

• By saving ‘pull’ documents locally. For example, if information about an item is sent to the user by means of a hyperlink, users or a system may be able to improve their overall satisfaction with record keeping by clicking on the hyperlink and retrieving and saving the document locally;

• By sorting personal records into folders, whether that be on a computer, in the cloud, within their email or in some other location. The results indicate that sorting into folders correlate with more effective personal records management; and,

• By making the process as effortless as possible. Despite the desire for improvement, such as by averting items being overlooked, there is little indication that people are keen to expend additional effort on their records management. Respondents agreed with items that provided improvements, provided there were tools to assist them in achieving these improvements. Correlation of the preference not to have to think about where to file things with the desire for improved personal records management outcomes is indicative of a general caveat: so long as the user does not need to do any more work.

Comparing respondents’ personal electronic records management practices at home with findings from the personal information management literature, we continue to observe similar themes of frustration (Alon et al., 2020; Alon & Nachmias, 2020b). For example, there is a tendency to use email for personal records management at home, just as in the workplace and by students (Dinneen & Krtalic, 2020; Whittaker et al., 2006, 2007b). Respondents who organise their records into folders, who resemble the ‘sorters’ described in the personal information management literature (Brackenbury et al., 2021; Whittaker & Hirschberg, 2001), tend to do so irrespective of the location, whether within email or on a computer. Sorters are more likely to save their records outside of email; this may indicate they are generally more diligent about their records management, whether that be within email or elsewhere. The majority of respondents who did save their personal records on a computer or in the cloud sorted them into folders (74% and 71% respectively). Respondents who sorted their records into folders were more likely to also sort their email into folders. Those who did not sort their email into folders equate to ‘pilers’ in the literature (Bälter, 1997; Whittaker & Sidner, 1996, p. 280). Thinking about participants who did not sort their email or records into folders as ‘personal records management minimalists’ may be a better descriptor than ‘non-sorters’ because the research does not suggest that they are opposed to sorting email or otherwise organising their personal records, merely that they do not. The differences in practice between work or study records and other records suggests that they may be evaluating an effort-benefit trade-off in terms of how much attention to invest in record-sorting, and therefore prioritising the sorting of particular kinds of records they expect to need to re-find again soon. Most minimalists (55%) responded positively to the idea of an automated system that would help them sort their email into folders. When asked which one way that they would like to save their records, only 8% said they would still keep their personal records in the in-box. Most respondents would prefer to use the in-box as a to-do list, rather than as a long-term record store.

Our findings support Jones’ (2008) much earlier observation that it is the practices of saving and sorting personal information that determines how effectively people will be able to re-find information when required, which in turn impacts on their overall level of satisfaction with their personal records management.

The survey suggests that some respondents would like to make their personal electronic records management practices at home more organised and less error-prone and that some people are non-sorters by practice but not by preference. Many respondents already sort their personal records to some extent, and express that they would prefer more organised practices. An automated system, such as an app developed with home personal records management in mind, may be able to alleviate some of the effort involved, making it easier, for example, to track bills and prepare tax information.

However, within respondents’ responses we also identified concerns about whether a fully automated system might reduce their control or increase risk of their overlooking items. This matches long-observed questions of automation bias, human monitoring of automated systems, and out-of-loop control, as suggested by Parasuraman & Riley (1997) and more recently by Merritt et al. (2012). Additionally, prior research has identified concern regarding data loss by automated and online systems (Alon & Nachmias, 2020a, p. 6), so the integrity of such systems for personal electronic records management would need to be extremely robust, and users would need to be reassured of the reliability and transparency of control that they would have over such a system.

Arising from the sampling limitations reported in the Method section, there is opportunity for further research into the differences between personal records management in different communities, for people in different age groups, people with different technological competencies, and in different circumstances. Such research may reveal different challenges for different groups of people, resulting in the requirement for alternative solutions to assist them in their records management. Another limitation is that the study did not address the specific affordances of electronic records, such as searchability, leaving this as an opportunity for further research.

Conclusion

This research on personal records management in the home brings into focus a number of commonalities with the findings of studies from the personal information management field in the workplace and at academic institutions.

The level of satisfaction respondents expressed with how they manage their personal records at home differs according to the practices they adopt. Overall, 63% of respondents were satisfied with their personal records management practice, compared to 74% satisfaction amongst those who adopted organised personal records management behaviours such as sorting items into files and folders. Respondents who sorted records into folders also experienced fewer oversights such as losing documents or failing to pay bills on time. Conversely, only 49% of respondents who left all their records in their email were satisfied with the records management practices. This research indicates that saving and sorting records into folders may result in improved satisfaction with the records management task, and reduced oversights. The findings also show that people may be motivated to adopt certain practices in their personal electronic records management (such as retrieving and saving records in a way that makes the records more easily re-found) if they find that the process is as easy or easier than their current practices and/or that the process provides a clear positive outcome for example easier tax reporting, improved budgeting, or reduced oversights. This means that prototypes and applications designed to assist with records management need to provide users with benefits such as easier budgeting or tax reporting without adding to the tasks that people need to perform in their records management.

About the authors

Matt Balogh is currently a PhD candidate at the University of New England, Armidale, NSW, Australia. Matt is a Fellow of The (Australian) Research Society. Matt worked in social and audience research for 36 years before commencing his PhD candidature. His particular interest is personal electronic records management. He can be contacted at matt.balogh@une.edu.au.

Dr William Billingsley is an Associate Professor in Computational Science at the University of New England. William’s research interests broadly include human-computer interaction, software design, and applied computer science: “smart useful systems”. He has particular interests in computing education, smart and social educational technology, and computer-supported cooperative work. He can be contacted at wbilling@une.edu.au.

Dr David Paul is an Associate Professor in Computational Science at the University of New England, Australia. David's research interests include computer networks and distributed systems, security and privacy, and applied computer science in areas such as health and agriculture. He can be contacted at dpaul4@une.edu.au.

Dr Mary Anne Kennan is an adjunct Associate Professor in the School of Information and Communication Studies at Charles Sturt University. Mary Anne’s research interests focus broadly on scholarly communication including open access and research data management; the education of, and roles for, librarians and information professionals; and the practices of information sharing and collaboration in various contexts. She is editor of the Journal of the Australian Library and Information Association (JALIA). She can be contacted at mkennan@csu.edu.au.

Dr Melanie Robertson-Dean is a Lecturer in mathematics and statistics at the University of Queensland. Her research interests include applied statistics across broad fields of science with an emphasis on linear mixed models as well as statistics and mathematics education, including help-seeking behaviour in first year university students. She can be contacted at m.robertsondean@uq.edu.au.

References

Alon, L., Forkosh-Baruch, A., & Nachmias, R. (2020, June 23-26). Gaps in personal information management behavior of higher education students: Implications for a training program. Paper presented at the EdMedia+ Innovate Learning, Online. http://www.learntechlib.org/p/217321/ (Internet Archive)

Alon, L., & Nachmias, R. (2020a). Anxious and frustrated but still competent: Affective aspects of interactions with personal information management. International Journal of Human-Computer Studies, 144, 102503. https://doi.org/10.1016/j.ijhcs.2020.102503

Alon, L., & Nachmias, R. (2020b). Gaps between actual and ideal personal information management behavior. Computers in Human Behavior, 107, 106292. https://doi.org/10.1016/j.chb.2020.106292

Alon, L., & Nachmias, R. (2022). The role of feelings in personal information management behavior: Deleting and organizing information. Journal of Librarianship and Information Science, 55(2), 313-322. https://doi.org/10.1177/09610006221079357

Balogh, M. (2022, April). Taming the paper pile at home: Adopting personal electronic records. ArXiv, 2204.13282. https://doi.org/10.48550/arXiv.2204.13282 (Internet Archive)

Balogh, M., Billingsley, W., Paul, D., & Kennan, M. A. (2022a). Attributes of personal electronic records at home. Archives and Manuscripts, 50(1), 47-70. https://doi.org/10.37683/asa.v50.10421 (Internet Archive)

Balogh, M., Billingsley, W., Paul, D., & Kennan, M. A. (2022b). Understanding the management of personal records at home: A virtual guided tour Information Research, 27(2), paper926. https://doi.org/10.47989/irpaper926 (Internet Archive)

Baltar, F., & Brunet, I. (2012). Social research 2.0: Virtual snowball sampling method using facebook. Internet Research, 22(1), 57-74. https://doi.org/10.1108/10662241211199960

Bälter, O. (1997). Strategies for organising email. Paper presented at the People and Computers XII Proceeding of HCI (Human-Computer Interface) '97, Bristol, UK. https://doi.org/10.1007/978-1-4471-3601-9_2

Bellotti, V., Ducheneaut, N., Howard, M., & Smith, I. (2003). Taking email to task: The design and evaluation of a task management centered email tool. Paper presented at the Proceedings of the SIGCHI (Special Interest Group in Computer Human Interaction) Conference on Human Factors in Computing Systems, Lauderdale, Florida, USA. https://doi.org/10.1145/642611.642672 (Internet Archive)

Bergman, O., Israeli, T., Whittaker, S., Yanai, N., & Amichai-Hamburger, Y. (2020). The effect of personality traits on file retrieval. Paper presented at the iConference 2020 Proceedings, Borås, Sweden. (Internet Archive)

Bergman, O., & Whittaker, S. (2016). The science of managing our digital stuff. The MIT Press. https://doi.org/10.17723/0360-9081-81.1.233

Bergman, O., Whittaker, S., & Gradovitch, N. (2022). Evidence for the merits of collecting streaming music. Personal and Ubiquitous Computing, 27, 331-342. https://doi.org/10.1007/s00779-022-01692-y

Boardman, R., & Sasse, A. (2004). “Stuff goes into the computer and doesn’t come out” a cross-tool study of personal information management. Paper presented at the Proceedings of the SIGCHI (Special Interest Group in Computer Huan Interaction) Conference on Human Factors in Computing Systems, Vienna, Austria. https://doi.org/10.1145/985692.985766 (Internet Archive)

Bondarenko, O., Janssen, R., & Driessen, S. (2010). Requirements for the design of a personal document-management system. Journal of the American Society for Information Science, 51(3), 468-482. https://doi.org/10.1002/asi.21280 (Internet Archive)

Brackenbury, W., Harrison, G., Chard, K., Elmore, A., & Ur, B. (2021). Files of a feather flock together? Measuring and modeling how users perceive file similarity in cloud storage. Paper presented at the Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event. https://doi.org/10.1145/3404835.3462845 (Internet Archive)

Bruce, H., William, J., & Susan, D. (2004). Information behaviour that keeps found things found. Information Research, 10(1), paper207. http://informationr.net/ir/10-1/paper207.html (Internet Archive)

Buttfield-Addison, P. (2014). Understanding and supporting personal information management across multiple platforms. [Unpublished doctoral dissertation]. University of Tasmania, Hobart, Tasmania, Australia. https://eprints.utas.edu.au/18564/ Cash, P., Isaksson, O., Maier, A., & Summers, J. (2022). Sampling in design research: Eight key considerations. Design Studies, 78, 101077. https://doi.org/10.1016/j.destud.2021.101077 (Internet Archive)

Dinneen, J., & Krtalić, M. (2020). E-mail as legacy: Managing and preserving e-mail as a collection. portal: Libraries and the Academy, 20(3), 413-424.

Dourish, P. (2001). The appropriation of interactive technologies: Some lessons from placeless documents. Computer Supported Cooperative Work (CSCW), 12. https://doi.org/10.1023/A:1026149119426 (Internet Archive)

Dourish, P., Edwards, K., LaMarca, A., Lamping, J., Petersen, K., Salisbury, M., Terry, D. B., & Thornton, J. (2000). Extending document management systems with user-specific active properties. ACM Transactions on Information Systems (TOIS), 18(2), 140-170. https://doi.org/10.1145/348751.348758 (Internet Archive)

Ducheneaut, N., & Bellotti, V. (2001). E-mail as habitat: An exploration of embedded personal information management. Interactions (Association for Computing Machinery), 8(5), 30-38. https://doi.org/10.1145/382899.383305 (Internet Archive)

Goretzko, D. (2022). Factor retention in exploratory factor analysis with missing data. Educational and Psychological Measurement, 82(3), 444-464. https://doi.org/10.1177/00131644211022031

Grbovic, M., Halawi, G., Karnin, Z., & Maarek, Y. (2014). How many folders do you really need? Classifying email into a handful of categories. Paper presented at the Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, Shanghai, China. https://doi.org/10.1145/2661829.2662018

Gwizdka, J. (2004). Email task management styles: The cleaners and the keepers. Paper presented at the CHI'04 extended abstracts on Human Factors in Computing Systems, Vienna, Austria. https://doi.org/10.1145/985921.986032 (Internet Archive)

Hellmich, H., & Dinneen, J. D. (2023). A guided tour study of the untidy but inspirational personal information management of visual artists. arXiv preprint, arXiv:2306.06026. (Internet Archive)

Henderson, S. (2004). How do people organize their desktops? Paper presented at the CHI'04 extended abstracts on Human Factors in Computing Systems. 2004, Vienna, Austria. https://doi.org/10.1145/985921.985972 (Internet Archive)

Henderson, S. (2005). Genre, task, topic and time: Facets of personal digital document management. Paper presented at the Proceedings of the 6th ACM SIGCHI (Association of Computing Machinery Special Interest Group on Computer-Human Interaction) New Zealand Chapter's International Conference on Computer-Human Interaction: Making CHI Natural, Auckland, New Zealand. https://doi.org/10.1145/1073943.1073957 (Internet Archive)

Henderson, S. (2009a). How do people manage their desktops. [Unpublished doctoral dissertation]. University of Auckland, Auckland, New Zealand. Retrieved on from https://a.storyblok.com/f/62420/x/c0de23330e/sarah-henderson-doctoral-thesis.pdf (Internet Archive)

Henderson, S. (2009b). Personal document management strategies. Paper presented at the Proceedings of the 10th International Conference NZ Chapter of the ACM’s Special Interest Group on Human-Computer Interaction, Auckland, New Zealand. https://doi.org/10.1145/1577782.1577795 (Internet Archive)

Henderson, S., & Srinivasan, A. (2011, January 4-7). Filing, piling & structuring: Strategies for personal document management. Paper presented at the 2011 44th Hawaii International Conference on System Sciences. https://doi.org/10.1109/HICSS.2011.205

(Internet Archive)

Holland, S. M. (2008). Principal components analysis (PCA [course materials]). Department of Geology, University of Georgia 30602-32501. (Internet Archive)

Jones, W. (2008). Keeping found things found: The study and practice of personal information management. Morgan Kaufmann.

Jones, W., Dumais, S., & Bruce, H. (2002). Once found, what then? A study of "keeping" behaviors in the personal use of web information. Proceedings of the ASIST Annual Meeting, 39, 391. https://doi.org/10.1002/meet.1450390143 (Internet Archive)

Kaber, D. B., & Endsley, M. R. (1997). Out‐of‐the‐loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety. Process Safety Progress, 16(3), 126-131. https://people.engr.ncsu.edu/dbkaber/papers/Kaber_Endsley_PSP_97.pdf (Internet Archive)

Karamizadeh, S., Abdullah, S. M., Manaf, A. A., Zamani, M., & Hooman, A. (2013). An overview of principal component analysis. Journal of Signal and Information Processing, 4(3B), 173-175.

Karger, D. (2007). Unify everything: It's all the same to me. In W. Jones & J. Teevam (Eds.), Personal information management. University of Washington Press.

Kitchenham, B., & Pfleeger, S. L. (2003). Part 6: Data analysis. In B. Kitchenham & S. L. Pfleeger (Eds.), Principles of survey research (Vol. 28): Association of Computer Machinery, SIGSPFT Software Engineering Notes

Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. (2015). Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. The American Psychologist, 70(6), 543-556. https://doi.org/10.1037/a0039210

Krishnan, A. (2010). Pervasive personal information spaces. [Unpublished doctoral dissertation]. University of Waikato, Hamilton, New Zealand. https://hdl.handle.net/10289/4590 (Internet Archive)

Mayoh, L. (2014, November 15). Drivers fined millions since new no vehicle registration sticker system introduced in NSW. Retrieved on December 31, 2019 from https://www.dailytelegraph.com.au/news/nsw/drivers-fined-millions-since-new-no-vehicle-registration-sticker-system-introduced-in-nsw/news-story/040a82526edc73eb8c23bce47fd1b8f9 (Internet Archive)

Merritt, S. M., Heimbaugh, H., LaChapell, J., & Lee, D. (2012). I trust it, but I don’t know why: Effects of implicit attitudes toward automation on trust in an automated system. Human Factors, 55(3), 520-534. https://doi.org/10.1177/0018720812465081

Napierala, M. A. (2012). What is the bonferroni correction. AAOS Now, 6(4), 40. (Internet Archive)

Oh, K. E. (2012). What happens once you categorize files into folders? Proceedings of the American Society for Information Science and Technology, 49(1), 1-4. https://doi.org/10.1002/meet.14504901253 (Internet Archive)

Oh, K. E. (2017). Types of personal information categorization: Rigid, fuzzy, and flexible. Journal of the Association for Information Science and Technology, 68(6), 1491-1504. https://doi.org/10.1002/asi.23787

Oh, K. E. (2020). Social aspects of personal information organization. Journal of Documentation, 77(2), 558-575. https://doi.org/10.1108/JD-06-2020-0104

Oh, K. E., & Belkin, N. (2011, February). Cross analysis of keeping personal information in different forms. iConference '11: Proceedings of the 2011 iConference, 732-733. https://doi.org/10.1145/1940761.1940888

Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230-253. (Internet Archive)

Potter, H., & Landauro, M. (2023, January 31). Unused subscriptions and forgotten outgoings could cost each aussie up to $1,261 a year. Retrieved on April 28, 2023 from https://newsroom.ing.com.au/unused-subscriptions-and-forgotten-outgoings-could-cost-each-aussie-up-to-1261-a-year/ (Internet Archive)

Preacher, K. (2021). Calculation for the chi-square test. Retrieved on August 18, 2021 from http://quantpsy.org/chisq/chisq.htm

Russell, D., & Lawrence, S. (2009). Search everything. In W. Jones & J. Teevan (Eds.), Personal information management. University of Washington.

Shepherd, E., & Yeo, G. (2003). Managing records: A handbook of principles and practice. Facet Publishing

Smith, L. I. (2002). A tutorial on principal components analysis. https://ourarchive.otago.ac.nz/bitstream/handle/10523/7534/OUCS-2002-12.pdf

Tugtekin, E. B. (2022). Investigation of the relationship between digital hoarding, information technologies self-efficacy, and anxiety. European Journal of Education Studies, 9(1). https://doi.org/10.46827/ejes.v9i1.4106

Vitak, J. (2016). Facebook as a research tool in the social and computer sciences. In The Sage handbook of social media research methods (pp. 627-644). SAGE Publications

Weber, R. (2018). Design-science research. In K. J. Williamson & G. Johanson, (Eds.), Research methods, information, systems and contexts (2nd edition, pp. 267-288). Elsevier Ltd.

Weisstein, E. W. (2004). Bonferroni correction. https://mathworld.wolfram.com/.

Whittaker, S. (2011). Personal information management from information consumption to curation. In B. Cronin (Ed.), in Annual Review of Information Science and Technology (Vol. 45, pp. 3-62). ASIS&T on behalf of the American Society for Information Science and Technology by Information Today Inc.

Whittaker, S., Bellotti, V., & Gwizdka, J. (2006). Email in personal information management: problems and possibilities. Communications of the ACM (Association for Computing Machinery), 49(1), 68-73. https://doi.org/10.1145/1107458.1107494

Whittaker, S., Bellotti, V., & Gwizdka, J. (2007b). Everything through email. In W. Jones & J. Teevan (Eds.), Personal information management. University of Washington.

Whittaker, S., & Hirschberg, J. (2001). The character, value, and management of personal paper archives. ACM Transactions on Computer-Human Interaction (TOCHI), 8(2), 150-170. https://doi.org/10.1145/376929.376932

Whittaker, S., Matthews, T., Cerruti, J., Badenes, H., & Tang, J. (2011). Am I wasting my time organizing email? A study of email refinding. Paper presented at the Computer Human Interaction (CHI) 2011, Vancouver, BC Canada. https://doi.org/10.1145/1978942.1979457

Whittaker, S., & Sidner, C. (1996). Email overload: Exploring personal information management of email. Paper presented at the CHI 96, Vancouver, Canada. https://doi.org/10.1145/238386.238530

Yeo, G. (2018). Records, information and data : Exploring the role of record-keeping in an information culture. Facet Publishing. https://doi.org/10.29085/9781783302284