Adventures in Winter Wonderland - observing user behaviour in a digital twin bookstore
Elke Greifeneder, and Maria Gäde.
Introduction. In casual leisure seeking contexts, visual information plays an important role for exploration or even serendipity affects. However, existing systems rarely support alternative search or browsing strategies. In the case study of this paper, we propose an approach to evaluate the potential and challenges of digital twins as digital replica of physical spaces.
Method. A mixed-method approach is applied combining observation data in the physical twin with web analytics in the digital twin.
Analysis. Both quantitative and qualitative analyses were applied to the data sources and comparisons between user behaviour in the physical and digital place are drawn.
Results. We report preliminary results and discuss the opportunities and limitations of current practices to observe digital and physical spaces. The data analyses have given a broad insight into behaviours in the physical twin. However, standard web analytics approaches could not reveal the same insight into online behaviour, showing the limitations of current research practices.
Conclusions. The analysis of digital twins requires the combination of offline and online practices. Experiences from the physical observation can inform the analysis of the digital space. It seems to be clear, that the analysis of log data alone is not sufficient but needs to be completed by other user behaviour methods.
Nowadays, a large part of book searches takes place online. Platforms like Amazon are widely used as a convenient alternative to the bookstore next door. Online book search is mainly realised via traditional search interfaces asking for query based information needs expressed through metadata elements such as authors, titles or publishers. However, in casual leisure contexts users often search for unknown books or even want to be inspired through the available offerings (Ross, 1999). Visual information seeking plays an important role for exploration or even serendipity affects but is rarely supported by existing systems.
Digital twins are 3D replications of physical spaces that aim at offering similar or extended services compared to their physical counterparts. Gardner (2019) predicted that ‘by 2022, over two-thirds of companies that have implemented [Internet of Things] will have deployed at least one digital twin in production.’ So far, most digital twins are developed and investigated for the manufacturing industry. In the case study of this exploratory paper, we are examining the digital twin of a local Berlin bookstore as an alternative to classical online systems.
In our work, we relate to previous research on social book search in causal leisure contexts. We consider digital twins as an opportunity to overcome current limitations of online search systems. This preliminary paper is part of a larger study to study methods and indicators to investigate user behaviour and experiences in digital twins. In particular, it aims at answering the question if observation data from the physical bookstore and web analytics data from the digital space are complementary and suitable to study the effect of digital twins?
The paper is structured as follows: First, we discuss previous work in the book search context, showing the need for augmented system design. We will then describe the concept of digital twins as well as our investigation of the case study Winter Wonderland. The paper concludes by discussing the current challenges and limitation for studying the use of and information seeking behaviour in digital twins.
Observations of behaviour and search strategies in physical libraries (Mikkonen and Vakkari, 2012) or bookstores (Buchanan and McKay, 2011) reveal that interpersonal and social contexts play an important role in selecting books (Buchanan and McKay, 2011; Gäde and Petras, 2016). Traditional online book search systems rarely support social and collaborative information seeking (SCIS) activities including conversational aspects among users and between users and librarians or bookstore staff (Evans and Chi, 2008). For a long time, research on online book search focused on queries in log file data and retrieval aspects in web-based systems (Agosti et al., 2012; Kim et al., 2012). More recent work on online forum discussion threads indicates that users often rely on book content, visual aspects such as covers and individual experiences when searching for new books (Boger et al., 2018). Considering that, several interactive information retrieval experiments were conducted testing alternative interface designs, supporting users during complex search tasks with user-generated and contextual information such as ratings, reviews and recommendations (Ullah and Khusro, 2019).
Taking into account that the mental concept of place has a strong influence on user behaviour and experience, researchers have been proposing designs for alternative and realistic systems for nearly 25 years (Harrison and Dourish, 1996; Pomerantz and Marchionini, 2007). To overcome current limitations and barriers, digital representations of realistic elements such as 3D objects, virtual bookshelves or interactive e-books have been introduced to support reality-based and even advanced interactions (Kleiner et al., 2013; Kim et al., 2017; Hinze et al., 2018). These efforts often include a combination of physical and virtual properties (Buchanan et al., 2019).
So far, the complete 3D representation of libraries or bookstores through the application of digital twins has not been investigated but could be a valuable data source for the challenging task of analysing and understanding interpersonal and contextual aspects of user behaviour particularly in online situations (Bruce and Partridge, 2011).
Digital twins are digital replica of physical spaces such as real-world buildings or shopping malls and are defined by three elements: there exists (1) a real space, called the physical twin or physical product, (2) the digital twin is a virtual copy of the physical space and there is (3) a connection between the physical and the digital twin e.g. defined by the real-time (sensor) data both generate (Tao et al., 2018; Sadik, 2018). In that regard, they extend users’ immersion into virtual worlds, compared to, for example, virtual tours like through the earthquake-damaged temples of Bagan in 3D (https://artsexperiments.withgoogle.com/bagan) or through the German (Technical) Museum in Munich (https://digital.deutsches-museum.de). The difference is that people can use the digital twin as a replacement of the physical twin and interact in the same way as with the physical twins. As with all emerging areas, there is a wealth of definitions, some of which allow digital twins to have developed extended capabilities beyond those of their physical twin (for example Madni et al., 2019).
Michael Grieves originally coined the term digital twin in 2002 at the Society of Manufacturing Engineers conference. In today’s digitised work, we expect the concept of digital twins to grow in importance, because they build an innovative bridge between offline and online life. It is important that information science, and in particular information behaviour research, contribute to the development of digital twins, so that the human factor is taken into account as early as possible in this emerging field. At the same time, digital twins allow information science research to directly compare online and offline behaviour.
For our case study Winter Wonderland we worked together with the German software company acameo which produces CUUUB, a digital twin application, which uses different laser scanning methods and User Experience (UX) software to develop curated digital worlds. In 2019, acameo joined forces with the German online bookstore genialokal (https://www.genialokal.de/) who delivers books and other media to over 650 local bookshops in Germany. In September 2019, acameo and team produced digital twins of two small bookshops in Berlin.
Both, the physical and digital place were examined using observational practices. In the physical bookstore, five research group members conducted non-participatory observations (Kromrey et al., 2016) during one week in December 2019. We received permission from the local bookstore to conduct the observations. The sessions included all work days across a range of all possible shopping times from very early in the morning up until closing of the shop. In total, we recorded 99 observations. Since the observations were non-participatory and the bookshop is not observable from one point, we had to follow customers unobtrusively, typing notes on smartphones which were later transferred to observation maps (see figures three and four as examples). We used an observation protocol and a written observation guideline to guarantee consistency and comparability.
Figure 3 illustrates the interactions that could be observed from the digital place: the selection of the drop-down menu for the navigation to a specific category of books (number 1 in Figure 5). Furthermore, users can navigate through the room either by elements on the ground or selecting a specific shelf (numbers 2 & 3 in Figure 3). The exhibited books can be selected for a detailed view by clicking on a shelf and a specific cover (number 3 in Figure 3).
For this preliminary analysis, it was decided to focus on page tagging data from December 2019 provided by Google Analytics. As stated above, the observable interactions are limited in contrast to physical observation, however, further user background can be taken into account such as location, systems used, languages and referring pages.
The preliminary results give insight into time spent in the physical twin, completed activities, and the role of in-person staff. The quickest shoppers needed only between one or two minutes; others spend their time trying to find particular things and stayed up to one hour and eight minutes. Thirty eight completed their activities within five minutes, twenty-five customers within 5-10 minutes, eleven customers between 11 and 20 minutes and seventeen stayed longer than 20 minutes. 68% were single female customers, 24% single male customers, 8% visited the bookstore together. A minor amount of 5% were accompanied children.
Customers showed a broad range of browsing behaviour with 12% who browsed under five minutes, 21% who spent between 5-10 minutes browsing the shelves and 29% who browsed more than ten minutes. 38% did not browse the shelves at all and instead went directly to the counter. The latter were customers who either had a book ordered online and came for pick-up (35% of all customers including those who also browsed showed this behaviour) or with a known item request (20% of all customers). A majority (63%) of all observed customers made a purchase.
Preliminary analysis of the created maps shows three distinct browsing behaviours: (1) a structured way where users only browsed a specific section of the bookstore; (2) a systematic browsing through the bookstore as a whole; and (3) where users meandered around the store in unpredictable ways (see figures three and four). More than half of all customers (60%) requested reference help from staff.
The analysis of online behaviour in the digital twin showed that, not surprisingly, most users are coming from Germany (90%), followed by the United States (2.5%). Only 19% of all sessions are from returning visitors, indicating that the majority of users are not coming back frequently. This is also confirmed through the vast amount of access (90%) coming from the referring bookstore portal geniallokal.de. Only a few users are directly or externally accessing the digital twin via other referral sources such as social media links.
In general, sessions were short in duration (30 sec.) with only 1.2 page views per session. The high bounce rate (users that do not perform any further action on the website) of nearly 85% needs to be treated with caution. Users do not necessarily need to perform any additional actions after entering the Winter Wonderland bookstore and could simply look around. Since the navigation is not captured by page-tags those interactions would not be taken into account.
The analysis shows that metrics from web analytics are not necessarily useful or sufficient to observe and interpret interactions in interactive digital places.
The observations in the physical twin show similar results as in previous studies, revealing the importance of interpersonal contacts and dialogues. In contrast, the web analytics data appears less appropriate to study user behaviour within the digital twin with very few observable interactions. While it is relatively easy to collect log data, the user information and actions that must be captured need to be specified in advance and carefully interpreted. So far, we do not have a clear understanding of relevant interactions within digital twins.
As a crucial difference between both versions the missing reference service plays an important role with respect to the available access points. For a future edition of Winter Wonderland, which is currently in preparation, a chat module following the trend of what is known as social commerce (Lam et al., 2019) is planned. If this implementation results in users treating digital twins as much as a social place as they use the physical twin, a follow-up study would need to analyse the logs of the chat modules to see if there are indications of social interactions. It is needless to add that this approach has high data privacy issues that may be difficult to overcome
The data analyses have given a broader insight into behaviour in the physical twin. However, standard web analytics approaches could not reveal the same insight into online behaviour, showing the limitations of current practices.
Digital twins need to be considered and evaluated as complex spaces that do not provide clear boundaries. Analysing information seeking behaviour in both places requires a variation of data collection methods as well as broadening the scope to alternative measures. What makes a good user experience in digital twins? Do both places influence each other and if, when or why are people using both places? As a follow-up step, experiences from the physical observation could inform the analysis of the digital space. It seems to be clear, that the analysis of log data alone is not sufficient but needs to be completed by user behaviour methods such as remote usability tests.
The authors wish to thank acameo and geniallokal for their openness to allow us to conduct this research, by offering us access to data and their physical premises.
About the author
Elke Greifeneder is a full professor for Information Behaviour at the Berlin School of Library and Information Science at Humboldt-Universität zu Berlin, where she also graduated. She previously held a position as assistant professor at the University of Copenhagen. She can be contacted at: email@example.com
Maria Gäde is a senior lecturer at the Berlin School of Library and Information Science at Humboldt-Universität zu Berlin. She graduated in 2014 on multi-lingual retrieval and has worked in particular on everyday life information seeking and information literacy. She can be contacted at firstname.lastname@example.org
- Agosti, M., Crivellari, F., Di Nunzio, G.M. (2012). Web log analysis: a review of a decade of studies about information acquisition, inspection and interpretation of user interaction. Data Mining and Knowledge Discovery, 24(3), 663-696. http://dx.doi.org/10.1007/s10618-011-0228-8
- Bogers, T., Gäde, M., Koolen, M., Petras, V. & Skov, M. (2018). What was this movie about this chick? In G. Chowdhury, J. McLeod, V. Gillet & P. Willett (Eds.), Proceedings of the International Conference on Information: transforming digital worlds (pp 323-334). Springer. (Lecture Notes in Computer Science, 10766). http://dx.doi.org/10.1007/978-3-319-78105-1_36
- Bruce, C.S. & Partridge, H.L. (2011). Identifying and delineating information experience as a research domain: a discussion paper. Paper presented at Social Media and Information Practices Workshop, University of Boras, 10-11 November 2011. http://eprints.qut.edu.au/47204/1.
- Buchanan, G. & McKay, D. (2011). In the bookshop: examining popular search strategies. Proceedings of the 11th Annual International ACM/IEEE joint conference on Digital Libraries (pp. 269-278). ACM Digital Library. http://dx.doi.org/10.1145/1998076.1998127
- Buchanan, G., Mckay, D. & Makri, S. (2019). Take me out: space and place in library interactions. Proceedings of the 2019 Conference on Human Information Interaction and Retrieval (pp. 45-53). ACM Press. http://dx.doi.org/10.1145/3295750.3298935
- Ceriani, M., Bernasconi, E. & Mecella, M. (2020). A streamlined pipeline to enable the semantic exploration of a bookstore. In M. Ceci, S. Ferilli & A. Poggi (Eds.), Proceedings of Italian Research Conference on Digital Libraries: the era of big data and data Science (pp. 75-81). Springer. (Communications in Computer and Information Science, n° 1177). http://dx.doi.org/10.1007/978-3-030-39905-4_8
- Evans, B.M. & Chi, E.H. (2008). Towards a model of understanding social search. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (pp. 485-494). ACM Digital Library. http://dx.doi.org/10.1145/1460563.1460641
- Gäde, M. & Petras, V. (2016). Hood or hypertext: a comparison of offline and online book search sessions. In K. Balog, L. Cappellato, N. Ferro & C. Macdonald (Eds.), Proceedings of theConference and Labs of the Evaluation Forum, 1609, 1097-1105. http://ceur-ws.org/Vol-1609/16091097.pdf. (Archived by the Internet Archive at https://web.archive.org/web/20200515103456/http://ceur-ws.org/Vol-1609/)
- Gardner (2019, February 20). Gartner survey reveals digital twins are entering mainstream use.https://www.gartner.com/en/newsroom/press-releases/2019-02-20-gartner-survey-reveals-digital-twins-are-entering-mai (Archived by the Internet Archive at https://web.archive.org/web/20200618133724/https://www.gartner.com/en/newsroom/press-releases/2019-02-20-gartner-survey-reveals-digital-twins-are-entering-mai)
- Harrison, S. & Dourish, P. (1996). Re-place-ing space: the roles of place and space in collaborative systems. In Proceedings of the 1996 ACM conference on Computer supported cooperative work (pp. 67-76). ACM Digital Library. http://dx.doi.org/10.1145/240080.240193
- Hienert, D. & Mutschke, P. (2018). Challenges for measuring usefulness of interactive IR Systems with log-based approaches. https://arxiv.org/pdf/1809.02413.pdf (Archived by the Internet Archive at https://web.archive.org/web/20200714090439/https://arxiv.org/ftp/arxiv/papers/1809/1809.02413.pdf)
- Hinze, A., Timpany, C., Vanderschantz, N. & Thomson, S. (2018). Who was that? Enriched eReading through in-book visualisation. In proceedings of the 32nd international BCS Human Computer Interaction Conference (pp. 1-11). ScienceOpen. http://dx.doi.org/10.14236/ewic/HCI2018.23
- Jansen, B.J. (2009). The methodology of search log analysis. In B.J. Jansen, A. Spink & I. Taksai (Eds.), Handbook of Research on Web Log Analysis (pp. 100-123). IGI Global Publisher.
- Kelly, D. (2009). Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval,3(1-2), 1-224. http://dx.doi.org/10.1561/1500000012
- Kim, J.Y., Feild, H. & Cartright, M. (2012). Understanding book search behavior on the web. Proceedings of the 21st ACM international conference on information and knowledge management (pp. 744-753). ACM Digital Library. http://dx.doi.org/10.1145/2396761.2396856
- Kim, M., Greenspan, B. & Quan-Haase, A. (2017). STAK - Serendipitous tool for augmenting knowledge: a conceptual tool for bridging digital and physical resources. Digital Studies, 9. http://doi.org/10.16995/dscn.265
- Kleiner, E., Rädle, R. & Reiterer, H. (2013). Blended shelf: reality-based presentation and exploration of library collections. Proceedings of the 13th Conference on Human factors in Computing Systems (CHI 13) (pp. 577-582). ACM Digital Library. http://dx.doi.org/10.1145/2468356.2468458
- Kromrey, H., Roose, J. & Strübing, J. (2016). Empirische sozialforschung: modelle und methoden der standardisierten datenerhebung und datenauswertung mit annotationen aus qualitativ-interpretativer perspektive (13th. ed.). Konstanz, München: UVK; UVK/Lucius.
- Lam, H.K.S., Yeung, A.C.L., Lo, C.K.Y., Cheng, T.C.E. (2019). Should firms invest in social commerce? An integrative perspective. Information & Management, 56(8), 103164. http://dx.doi.org/10.1016/j.im.2019.04.007
- Madni, A.M., Erwin, D. & Madni, A. (2019). Exploiting digital twin technology to teach engineering fundamentals and afford real-world learning opportunities. Atlanta: American Society for Engineering Education-ASEE.
- Mikkonen, A. & Vakkari, P. (2012). Readers’ search strategies for accessing books in public libraries. Proceedings of the 4th Information Interaction in Context Symposium (IIIX’12) (pp. 214-223). ACM Digital Library. http://dx.doi.org/10.1145/2362724.2362760
- Pomerantz, J. & Marchionini, G. (2007). The digital library as place. Journal of Documentation, 63(4), 505-533. http://dx.doi.org/10.1108/00220410710758995
- Ross, C.S. (1999): Finding without seeking: the information encounter in the context of reading for pleasure. Information Processing & Management, 35(6), 783-799. http://dx.doi.org/10.1016/S0306-4573(99)00026-6
- Saddik, A.E. (2018). Digital twins: the convergence of multimedia technologies. IEEE MultiMedia, 25(2), 87-92. http://dx.doi.org/10.1109/MMUL.2018.023121167
- Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H. & Sui, F. (2018). Digital twin-driven product design, manufacturing and service with big data. The International Journal of Advanced Manufacturing Technology, 94(9-12), 3563-3576. http://dx.doi.org/10.1007/s00170-017-0233-1
- Ullah, I. & Khusro, S. (2019). On the search behaviour of users in the context of interactive social book search. Behaviour and Information Technology, 39(4), 443-462. http://dx.doi.org/10.1080/0144929X.2019.1599069