Report on the Second Workshop on Collaborative Information Seeking: New Orleans, 12 October, 2011

Introduction

People working together for information seeking activities is not new, nor is researchers’ and practitioners’ interest in studying such situations of collaborative information seeking and creating solutions for supporting and enhancing them. In the 1990s, Grudin (1994) outlined factors that developers need to consider when creating collaboration solutions (groupware systems), and Twidale (Twidale and Nichols 1996) studied collaborative episodes in the context of virtual reference interactions in a library using his Ariadne interface. Other researchers have looked at aspects of the subject, but the majority of them remained limited to specific domains or situations. It was not until recently that researchers and practitioners from different disciplines started coming together for conversations and collaborations about collaborative information seeking and related topics. This is evident from a stream of recent publications at venues such as SIGIR, CSCW, Group, CHI, and ASIST conferences, as well as numerous workshops on this subject.

This technical report is culmination of the works presented and discussed at the 2nd workshop on collaborative information seeking, organized at the 74th annual conference of ASIST on October 12, 2011 in New Orleans. The workshop brought together participants from a wide range of domains and views, and this is reflected in their position papers available from the workshop website. The workshop brought together researchers and practitioners from different domains and organizations, allowing us to engage in discussing variety of important questions that relate to bridging the gap between theory and practice. Some examples are listed below.

1. How does collaboration fit into existing solitary information seeking activities?
2. How do we move from single-user information seeking models to those for multi-users (collaborative)?
3. The concepts of collaborative information handling and sharing are used in an unclear way. What is the distinction between them?
4. How to integrate collaborative seeking into everyday life and professional environments?
5. What should be required and desired characteristics of a collaborative information seeking system?

Position papers

Workshop participant Joseph Meloche from North Carolina Central University discussed collaborative information seeking’s role in teaching and research and argued that in five years, collaborative information seeking will be in all forms of activities. Nasser Saleh from McHill University described a study involving collaborative information seeking in education, using the concept of sense-making, and demonstrated how a learning task could impact students’ information seeking behaviour in collaboration. Rob Capra from the University of North Carolina at Chapel Hill, on the topic of tasks in collaborative information seeking, presented results from a study where the subjects carried out various types of tasks and were found to be more engaged in exploratory collaborative search tasks than in fact-finding or transactional ones. Gene Golovchinsky from FX Palo Alto Laboratory, Inc. argued for the need for algorithmic mediation in collaborative information seeking and showcased his Querium system. Miles Efron from the University of Illinois at Urbana-Champaign presented a collaborative search game incorporating fun and competition as two important aspects in collaborative information seeking. Preben Hansen from Swedish Institute of Computer Science presented his work with the CLEF evaluation campaign in order to show how one could extract requirements for collaborative information seeking situations. Roberto González-Ibáñez and Chirag Shah from Rutgers University argued for a need to have a comprehensive framework for evaluation in collaborative information seeking and showed how a number of aspects pertaining to user-driven processes could be measured.

Working group discussions

As part of the workshop, we had an afternoon work session to brainstorm and discuss topics important for future research on collaborative information seeking. In a plenary session, the following high-level categories were selected as being important: 1-system issues; 2-privacy, security and trust; 3-groups and roles; 4-future directions; 5-theory; 6-evaluation; and 7-data collection. Each of these categories was written on a blank poster sheet taped to the wall. Then, workshop participants were invited to individually brainstorm ideas for each category, write them on post-it notes, and stick them to corresponding poster sheets. After ideas had been collected, we broke into two groups to discuss the ideas in detail. The first group covered the categories 1-4, and the second group covered 4-7.

Report on discussion topics

System issues

This part of our discussion focused on how systems and tools impact collaborative information seeking.

• How can systems make collaboration more effective?
  The group is greater than the sum of individuals. We discussed a goal that a collaborative information seeking system should, by some measure, be more than the sum of the individuals collaborating. Possible quantities include the amount of information, the time spent, the diversity of information, and level of social interaction.
• What are important approaches and solutions to explore for collaborative information seeking systems?
  System-centered: these approaches could include techniques such as algorithmic mediation and using the system as a bridge (e.g., for catching up on collaborators’ prior actions or findings, linking collaborators, and moving information from personal spaces to group spaces and back).
  User-centred: here, we discussed how a system could help with selecting collaborators for a given task.
• How can a system help with division of labour?
  Division of labour is commonly employed by groups engaging in collaborative information seeking. We discussed three ideas for how systems can help support this practice (if desired by the collaborators): providing a communication channel as part of the system, allowing users to do a 'smart split' of the results, as available in the SearchTogether system (Morris and Horvitz 2007), where the search results are divided among the collaborators according different aspects (e.g., source, content or author), and support for managing policies about how the sub-tasks are divided, assigned, or taken on by the individual collaborators.

Privacy, security and trust

In this part of our discussion, we focused on how the related issues of privacy, security, and trust influence collaborative information seeking.

• Mapping trust to privacy awareness. How a system makes users aware of privacy issues can affect users’ trust in the system and willingness to share information.
• Masking and unmasking information. Systems could support methods for selective sharing of information among collaborators.
• Inconsistency of relevance judgments. Document relevance is known to depend highly among individuals, even trained judges. Collaborators may trust the judgments made by some collaborators more than others. Designers of collaborative information seeking systems should be aware of these issues and look for ways that it may have an impact on system design.
• Conflict management. In collaborative information seeking, conflicts may arise among collaborators and/or policies for privacy and sharing.

Groups and roles

We discussed how groups may conduct collaborative information seeking and what individual roles may be taken in the process. Based on the suggested ideas and our discussions, we list several areas for exploration.

• Symmetric vs. asymmetric roles—what implications are there for system design and evaluation?
• Situations with user-defined versus system-defined roles.
• "Learned roles"—in some situations, there may be roles that are learned, either by the collaborators, or, a system might try to learn roles based on the collaborators’ interactions and inputs
• How can systems support various types of roles, and what should role-specific interfaces do?

Future directions

In this part of our discussion, we examined several future directions for collaborative information seeking research.

• Smarter systems: features that provide facilitate collaborative information seeking through specific features
• Privacy and security: this is an area that needs more research to examine how these issues impact collaborative information seeking and system design
• Promoting and supporting asymmetric roles and diversity to encourage the whole activity being more than the sum of the parts—more work needs to be done to help understand how roles and diversity contribute in collaborative information seeking activities. Research in computer-supported collaborative work has examined many relevant dimensions, and more connections are needed between collaborative information seeking and such research.
• Integration of collaborative information seeking tools into existing systems and practices—more situations where collaborative information seeking occurs should be studied. There are connections here to research from the collaborative work and organizational knowledge communities.
• Transition from single-user situations and systems to collaborative information seeking and back—how can systems support the need for individuals to have personal information spaces in which they conduct searches and then share (perhaps selectively) their results with their collaborators, and then re-integrate group results into their personal space if desired.

The second part of the workshop included a work session in which the participants were divided into two groups. The second group debated theory, evaluation and data collection. These topics were selected as important in a plenary session before splitting up in groups. Based on the notes related to each of the topics, a discussion took place and the following issues were pointed out as important.

Theory

The following list of theoretical issues emerged, reflecting important issues we need to take into account in collaborative information seeking research.

• Growth of theory. Currently there are many different models and theoretical frameworks within human information behaviour, information seeking and information retrieval. The idea with theory growth is to expand existing theories with theoretical insights from the collaborative information seeking research. In such a way, the established models and frameworks could be expanded and enriched.
• Theoretical framework for collaborative information seeking. All participants agreed that it was very important to make a contribution towards developing a theoretical framework for collaborative information behaviour. In doing so, involving theories and frameworks from other disciplines and previous theories would intensify studies in collaborative information seeking.
• Differences between individuals and teams. We discussed the importance of investigating what the differences are between group behaviour and individual behaviour when performing collaborative information seeking activities in different domains and situations. What and how can we identify differences and how could these be supported?
• Activity theory. It was suggested that activity theory would be an interesting framework for research on collaborative information handling. The group reasoned that activity theory could be useful for more ethnographic research and that the theory is good for understanding and analysing phenomena such as finding patterns.
• Computer-supported collaborative work and computer-mediated commucation. Furthermore, it was recognized that these research areas are interesting as platforms for research and methods. Influences from these areas may contribute to research in the collaborative information seeking field.
• Play theory, social and positivistic psychology were also mentioned as possible contributors to the collaborative information seeking research area.

Evaluation

The second issue that emerged reflecting important issues we need to take into account concerned aspects of evaluation of related systems. What is needed is to re-visit the existing methods and measures for evaluating information systems. The methods and measures need to reflect issues from both a user's point of view as well as from a system point of view. From the user point of view, we may perform evaluation that is designed for naturalistic settings. At this level we have less control over the experimentation but, on the other hand, we the more real situation may include collection of a large amount of data. This level may include contextual aspects difficult to handle but it may reveal interesting aspects of the flow of interaction. On the other hand, at the laboratory level, we have more control over the experiment and the variables to be measures.

• Methods and methodologies
• Measures
• User Side
  • Naturalistic settings
  • Semi-controlled settings
  • Laboratory studies
• System side
  • Evaluation in the style of the Text Retrieval Conference (TREC)
• Lessons learned

The working group discussed a way to collect and display results from different experiments and studies to be able to share knowledge on research in this area.

Data collection

Finally, the group talked about aspects of data collection. During the discussion two things were considered interesting: different tools for collecting data on collaborative information seeking and that a document is more than just keywords. Based on the latter, the group developed a visualization of different aspects (or features) of a document. Document features that could be shared and that could be investigated from a collaborative information handling perspective. Features of interest and mentioned during the workshop were (see figure 1 below): time stamps of different kinds related to the document (created, changed etc); the document space the document resides in (in one or more collections); annotations; contextual features for each manifestation of the document; access and control of the document; who actually viewed the document; sharing the query and query sessions; and sharing the transaction logs for gaining knowledge.

Conclusion

The workshop was quite successful by several measures. First, it brought researchers and practitioners, students and teachers from several domains, which is essential for an interdisciplinary domain of collaborative information seeking. Second, the presentations and discussions were more focused than those done in some of the other such events the authors have attended, indicating focus and maturity that this field has achieved in the recent years. And finally, the lessons derived and the questions raised through these discussions (which are reported here) should provide pointers to further research and developmental works to those interested in this domain.

Acknowledgements

We are thankful to ASIST for helping us organize this workshop at their annual meeting in 2011. We are also very grateful to all the workshop participants who actively engaged and contributed to the workshop, making it highly successful.

About the authors

Dr. Chirag Shah is an assistant professor in the School of Communication & Information at Rutgers University. He received his PhD from the University of North Caroliona at Chapel Hill. He can be contacted at: chirags@rutgers.edu.
Dr. Preben Hansen is a senior researcher at the Swedish Institute of Computer Science. He recieved his Ph. D in Information Science from the University of Tampere, Finland in 2011. His contact address is preben@sics.se
Dr. Robert Capra is an Assistant Professor in the School of Information and Library Science at the University of North Carolina at Chapel Hill. He received his Ph.D. in Computer Science from Virginia Tech. (Virginia Polytechnic Institute and State University) He can be reached at: rcapra@unc.edu