The interplay between literacy and digital technology: a fuzzy-set qualitative comparative analysis approach

Introduction

The ever-increasing span of digitalisation and the pace with which new digital technologies reach educational environments have resulted in students at the universities having to put additional effort in to keep up with the digital technology development. The adoption of digital technology and the decision to use, depend on students’ positive perception of digital technology. Moreover, despite the growing interest in using digital tools for teaching and learning, the need for information and digital literacy skills has become prominent. Some authors state that as digital technology is widely present within the contemporary activities, the need for information literacy and digital literacy skills is prominent for the effective execution of such activities, e.g., learning (e.g., Lodge, et al., 2020).

According to Kurbanoglu, et al. (2006, p. 730), in the information-based society, information literacy incorporates not only the abilities to recognise when information is needed but also the abilities to initiate appropriate search strategies to locate the needed information. In the digital era, an information literate person is a person who has required abilities to evaluate, synthesise, and use information appropriately, ethically, and legally once it is accessed regardless of its format (digital or non-digital) and the source where it was retrieved. With regard to digital literacy, although literature presents no consensus in what the concept exactly stands for, digital literacy refers here to the ability to understand and use information in multiple formats from an array of digitally available sources (Glister, 1997) as well as the ability to use information and communications technology (ICT) effectively (Bawden, 2008).

In this paper, we propose that students’ literacy skills (both information and digital) determine their intention to use digital technology for learning. In other words, the interplay between the literacy skills and digital technology influences students’ intention to use digital technology for learning. Many scholars stated that digital technology has become increasingly common for learning and teaching and is even expected to be a part of formal learning environments (Lea and Jones, 2011; Meyers, et al., 2013). Therefore, understanding students’ perceptions of digital technology for learning and their information and digital literacy skills not only contributes to research in this domain, but also provides insights to policymakers to identify implications for future investment in the use of digital technology for learning at the universities.

The main objective of this paper is to conduct an empirical research and provide insights and knowledge on factors influencing students’ intention to use digital technology for learning. To make this paper as comprehensive as possible, we conducted a thorough literature review, and based on the review results, we designed a questionnaire to collect data from students in a Finnish and an Italian university. The data comprises of 49 Finnish and 37 Italian students of different master’s programmes. Collected data has been analysed through a novel methodological approach, i.e., fuzzy-set comparative analysis (fsQCA) Ragin, 1987) which has seldom been used in this line of research.

Literature review

Bruce (1999) stated that in an information society, an information literate person is expected to have the ability and competence to operate effectively when confronted with information and be a critical thinker. In a similar vein, Ng (2012) defined an information literate person as a person who is expected to have the ability to effectively locate, use, store, retrieve, share, evaluate, and communicate web-based information. In the educational context, Maybee (2006) argued that an information literate student is expected to have the ability to determine the nature and extent of the information needed to accomplish a specific task and the ability to access needed information effectively and efficiently. On that account and based on empirical studies, some authors pointed out that such abilities and skills influence individual’s perception of ICT and the intention to use digital technology, specifically in learning environments (e.g., Lai, et al., 2012; Ng, 2012; Teo et al., 2019; Nikou, et al., 2019).

We argue that an information literate student should be able to recognise his or her information needs and possesses an array of abilities, e.g., to identify the credibility and usefulness of information, and recognises the originality of information sources. Such abilities and skills can be enhanced under specific conditions; for instance, by using digital tools more frequently or/and becoming proficient in using them. As such, it could be speculated that the more they [students] use digital technology, the more competent they become in assessing the credibility of information. In other words, frequency of use as well as proficiency in using digital tools could be considered as potential factors enhancing students’ information literacy skills and enabling them to critically appraise information. Yu, et al. (2017) showed that information literacy effects the relationship between individual digital skills and ICT adoption behaviour. However, it should be noted that use of technology does not necessarily cause proficiency or increase one’s ability to critically assess the value and the quality of information (Guess, et al., 2019).

While information literacy is generally a concept that defines a persons’ ability to manage and use information, digital literacy includes abilities and skills to find and use information with the help of ICT in digital environments. According to Ng (2012), a digital literate person should have the needed technical and operational skills to use technology and ICT to locate, find and use information. Moreover, Eshet (2004, p. 103) argued that individuals can use their digital literacy skills to improve their performance and overcome the problems they might encounter in digital environments. In addition, Nikou, et al.(2018) in their effort to examine the impact of digital literacy among university students and university staff (teachers and researchers) found that digital literacy positively and directly impacts the intention to use digital technology in an educational environment.

Still, in addition to information literacy and digital literacy, literature shows that there are other factors that have been frequently mentioned as the predictors of intention to use digital technology among university students, e.g., social influence and self-efficacy. From a social psychology viewpoint, social influence is defined as ‘perceived social pressure to perform or not to perform a behaviour’ (Ajzen, 1991). For example, Gu, et al. (2013), Thompson, et al. (2006) and Chang and Suttikun (2017) argue that the perceived social influence from others significantly impacts on student’s perceptions and beliefs about the usefulness of technology. The current generation of students who have grown up in a digital environment where the Internet has always been part of their lives and that has shaped how they think, behave, interact, and communicate with their peers and friends. Thus, it can be assumed that they are more knowledgeable than their counterpart generation (parents and teachers) who have had to learn into a digital environment (Maybee, 2006). Thus, in our paper, social influence is conceptualised as one of the determinants of intention to use ICT and technology in the educational environment.

In addition to the impact that social influence has in an educational environment, technology self-efficacy is also important for several additional reasons. According to Bandura (2010) self-efficacy is people’s beliefs about their capabilities to exercise control over their own level of functioning. In other words, people’s beliefs in their ability to influence events that affect their lives. There have been many studies that have investigated the perceived technological ability (e.g., Peng, et al., 2006) and the findings show conflicting results. For instance, one reason frequently given for a student’s negative perceptions of technological ability in the classroom is that they suffer from low technology self-efficacy (Huffman, et al., 2013, p. 1779; Igbaria and Iivari, 1995). Bates and Khasawneh (2004) posit that students who fear computer technologies may experience confusion, anxiety, a loss of personal control, in other words suffering from low computer self-efficacy. However, on a positive side, Wang and Newlin (2002) found that technology self-efficacy is positively correlated with online learning performance. Thus, in our paper, self-efficacy is conceptualised as one of the determinants of intention to use ICT and technology in educational environment.

Finally, our dependent variable is intention to use digital technology in an educational environment among university students. Gu et al. (2013) have found that the gap between teachers and students’ use of technology is associated with how important they perceived it to be. Moreover, Wang, et al. (2012), observed that intention to use is a strong predictor of technology acceptance in educational environment among students. Therefore, intention to use digital technology for learning purposes is conceptualised as an outcome. We assume intention to use is influenced by students’ perceptions towards their digital and information literacy skills, social influence, and self-efficacy.

Research methodology

The target population of this research is students from a Finnish university (comprised of international master’s degree students in, e.g., governance of digitalisation and information technology) as well as students from a university in Italy (comprised of engineering students from two master’s degrees, mechatronic and materials, and master students in business studies). The goal is to investigate Finnish and Italian university students’ intentions to use digital technology for learning purposes. These two countries are known for their use of ICT in educational environments (Ferri et al., 2009; Pöntinen, et al., 2017). Data were collected through two sets of surveys; Finnish data was collected via an online questionnaire in April 2019 and data from Italy was collected via a paper-based questionnaire in November 2019. The questionnaires used for data gathering were identical in both countries and English was the main language.

In this paper, the survey items were all adapted from previously validated studies and if needed were slightly modified to fit the context of this study. For measuring digital literacy (comprising of three dimensions: technical, cognitive, and social-emotional literacy), we used ten items from Ng (2012). In addition, inspired by literature, we used two self-developed items ‘digital technology enables me to collaborate better with my peers on project work and other learning activities’ and ‘I use my cognitive skills to find, evaluate, create, and communicate information’. For information literacy, we used seven items from (Ahmad, et al. 2020; Kurbanoglu et al., 2006). The items for social influence, self-efficacy and intention to use were derived from (Ayeh, et al., 2013; Venkatesh, et al., 2012) with four, eight and six items, respectively. All items were measured using a 7-point Likert scale ranging from 1 being strongly disagree to 7 being strongly agree. Moreover, additional questions related to proficiency in using digital tools and frequency of use of digital devices were asked. To measure proficiency, we asked, ‘pleaseindicate how proficient you are while using the following digital technologies’ and was measured on a scale of 1-7 with 1 being not proficient at all and 7 being very proficient). For frequency of use we asked, ‘please indicate how frequently do you use the following digital devices’ and was measured from I do not use to several times each day. Survey items were analysed through factor analysis to verify the constructs structures using IBM SPSS v.24. All factor loadings were assessed, and results showed that all loadings, except for two items, were above the recommended threshold of 0.70. The reliability of the items was also estimated using Cronbach’s coefficient alpha and alpha values were all above recommended threshold of 0.70.

Configuration theory

Regarding the data analysis, we used a variant of qualitative comparative analysis, namely, fuzzy-set qualitative comparative analysis (Ragin, 1987). The core methodological objective for applying fuzzy-set qualitative comparative analysis was the novelty of this approach in the field of our study as well as its ability to provide complementary insights which would be difficult to obtain if conventional statistical analysis such as regression analysis was applied. Fuzzy-set qualitative comparative analysis enables researchers to examine conjunctural causation and find alternative paths leading to the outcome. This method allows to identify different patterns of causal conditions that produce the desired outcome, rather than how each individual independent variable relates to the outcome, a sort of outcome we often obtain when, e.g., structural equation modelling is applied. Moreover, instead of postulating hypothesis, with this approach, a researcher can formulate proposition(s). Thus, our proposition is: ‘digital literacy, information literacy, social influence and self-efficacy together are necessary conditions for predicting students’ intention to use digital technology for learning’.

However, one can speculate that other factors (e.g., accessibility to the internet) which were not included in this study might equally have potential to impact the students’ intention to use digital technology for learning. While, this is a valid argument, we believed, considering the context of the research and insights gained from the literature review, factors included in this paper are as comprehensive as possible to predict the students’ intention to use digital technology.

As mentioned, we used fuzzy-set qualitative comparative analysis to assess the possible effect of different combinations of conditions (variables) on the outcome of interest (intention to use). The necessary conditions analysis, in terms of fuzzy-set Qualitative Comparative Analysis, emphasises the cases that influence the outcome (intention). Fuzzy-set qualitative comparative analysis uses an algorithm to simplify the combinations and minimises the solutions. Moreover, the core theoretical assumption when applying fuzzy-set qualitative comparative analysis is that there may be more than one combination of conditions that leads to the outcome of interest (Mas-Tur et al., 2015), and it is referred to asequifinality. Equifinality indicates that a single combination (configuration) of causal conditions cannot explain the outcome, rather there might be several causal paths to the phenomenon being examined (Fiss, 2011). Thus, the fuzzy-set qualitative comparative analysis results will be often in the form of multiple configurations.

The fuzzy-set qualitative comparative analysis must be conducted in a sequence starting from calibration, necessity analysis, truth table construction and ending with sufficiency analysis which is the final step. Calibration is the first step to transform the scores respondents given to each survey items into fuzzy sets ranging from zero to one (Ragin, 2008) and we used the fuzzy-set qualitative comparative analysis v.3 software to calibrate the values. There are two types of calibration, a direct and indirect calibration, in direct form, we identified three qualitative breakpoints of the fuzzy sets and in indirect calibration we rescaled the original scores based on the qualitative assessments. When calibrating Likert scales, Ragin and Davey (2016) recommend using three qualitative anchors (0.95, 0.5 and 0.05), representing a (i) full-membership, (ii) the cross-over point (most ambiguous membership), and (iii) a full non-membership. So, as we used seven Likert scales. We took this approach and assigned full membership (0.95) for the value of 7 given to survey items, for 4 (the cross-over point) we assigned 0.51, and we assigned 0.05 for 1 (non-full membership) (Fiss, 2011; Ragin, 2000). The other values, that is, 2, 3, 5, and 6, were calibrated based on a linear transformation function.

Necessity analysis

In the next step, we assessed the necessity analysis to examine if there were any conditions that could be identified as necessary for the outcome (intention to use) to occur. The values above 0.90 indicated that the condition is necessary for the outcome to occur; however, our analysis showed that none of the conditions (variables) had value above 0.90. Thus, we moved forward and constructed the truth table which is a data matrix comprising a list of all combinations of causal conditions with respect to the cases (observations) within each combination (Muñoz and Dimov, 2015, p. 644). Ragin (2008) suggested to set the frequency threshold to one and delete the combinations from the truth-table that do not comply to this threshold. When fuzzy-set qualitative comparative analysis is used, we also account for consistency value which is used to assess the extent to which a combination of causal conditions in the truth-table is sufficient for the outcome of interest to occur. Ragin (2008) recommended to use 0.75 for the consistency. In this paper, we took a more conservative approach and set the frequency cut-off to 3 and consistency threshold was set to 0.75. Finally, the output of the sufficiency analysis, which is the final step of fuzzy-set qualitative comparative analysis, presents three types of solutions (complex, parsimonious, and intermediate) and it is recommended to use the intermediate solutions for the interpretation of the results. So, we follow this recommendation and when we present the fuzzy-set qualitative comparative analysis result, the intermediate solutions will be interpreted and elaborated.

The following notations are used to demonstrate the fuzzy-set qualitative comparative analysis results: black circles (⚫) indicate the presence of a condition and blank circles (⚪) indicate its absence. Blank spaces indicate do not care (Ragin and Fiss, 2008).

Results

Descriptive statistics and measurement results

In the Finnish sample, 49 students participated, of whom 28 were female and 21 were male. The average age of the respondents was 24.2 years with 4.18 Standard deviation. In the Italian sample, 37 students participated, of whom 16 were female and 21 were male. The average age of the respondents was 25.3 years with 2.18 Standard deviation. The result of self-reported proficiency in using digital tools is shown in Table 1 (mean values are reported). As it can be seen, almost in all items (except social media) Finnish students in average scored higher than Italian students in rating their proficiency.

Table 2 shows the frequency of use of digital devices by the students. The result shows that in average, Finnish students scored higher than Italian students in rating their frequency of use of digital tools. For example, the frequency of use of mobile Internet, while the average value for Finnish students is 6.32, this value for Italian students is 4.8. This indicate that Finnish students use the handset devices more often to connect to the Internet. The plausible reason could be the low Internet access fee throughout Finland or because Finland has one of the best mobile internet coverage worldwide.

Fuzzy-set Qualitative Comparative Analysis results

For the main part of the fuzzy-set qualitative comparative analysis and to understand better students’ intention to use digital technology, we examined the combined effects of information literacy, digital literacy, social influence and self-efficacy. The relationships between these conditions (variables) and the outcome, intention to use are shown in Table 3 for the Finnish sample and in Table 4 for the Italian sample. The fuzzy-set qualitative comparative analysis results for the Finnish students generated four configurations (solutions). Table 3 shows that except for one configuration (solution 1), the presence of digital literacy dominates all configurations, it means digital literacy is a strong predictor for the Finnish students’ intention to use digital technology for learning.

In solution one, the presence of social influence (the effect of others) and the presence of information literacy lead to the outcome. Interestingly, this is the only solution where digital literacy does not play a role. In solution two, we see that the negation of self-efficacy and the presence of digital literacy lead to the outcome of interest. In other words, this configuration indicates that the absence of self-efficacy plays a role in the decision of Finnish students to use digital technology for learning. In solution three, the presence of both information literacy and digital literacy lead to the outcome and based on the consistency value and the raw coverage value, this configuration is the strongest among all the other configurations. Finally, solution four indicates that the presence of social influence, self-efficacy, and digital literacy lead to the outcome of interest. This solution is an interesting configuration, because it indicates the pressure of others (social influence), and own beliefs about the ability to use ICT (self-efficacy) are decisive factors in addition to digital literacy for the Finnish students. Based on the fuzzy-set qualitative comparative analysis results, the overall solution consistency is 0.996 and the solution coverage is 0.707. In other words, these four solutions cover (predict) approximately 71% of the cases’ (respondents) intention to use digital technology for learning (see Table 3).

Regarding the Italian sample, the fuzzy-set qualitative comparative analysis generated three solutions (see Table 4). The results show that the presence of information literacy dominates all three configurations, which is an interesting observation compared to the Finnish results. In solution one, the presence of self-efficacy and information literacy lead to the outcome. Interestingly, in this solution digital literacy does not play a role. It can be assumed that for students in this configuration their beliefs in their own abilities to deal with various situations as well as their perceptions of their information literacy skills are sufficient conditions (factors) influencing their decision to use digital technology for learning. In solution two, the presence of information literacy and digital literacy lead to the outcome of interest. In solution three, the presence of both information literacy and digital literacy in addition to presence of social influence lead to the outcome of interest. Based on the consistency value, solution three is the strongest among all configurations. The overall solution consistency for Italian sample is 0.994 and the solution coverage is 0.736. In other words, these three solutions cover approximately 74% of the cases (see Table 4).

According to the results, the main difference between Finnish and Italian students in their intention to use digital technology is their perceptions of digital literacy and information literacy. While, it is clear that both dimensions of literacy skills have been evaluated as being important abilities, nevertheless how these two concepts impact students’ intention is different among Finnish and Italian students. According to the results shown in Table 3 and Table 4, we can observe that the four factors included in this research (i.e., social influence, self-efficacy, information literacy and digital literacy) influence intention to use digital technology. However, their influence, in terms of fuzzy-set qualitative comparative analysis, is observable through different combination sof conditions. Indicating that there are multiple paths to the outcome of interest.

Discussions and conclusion

In the digital age where people are exposed to information overload, digital skills have increasingly become one of the core competences required from individuals, and especially from university students. In this paper we investigate what factors impact university students’ intention to use digital technology for learning. Based on the fuzzy-set qualitative comparative analysis results, we argue that the research on literacy should focus on the combined effect of factors, rather than only focusing on information literacy or digital literacy or technological skills. Our findings reveal that a combination of both information and digital literacy, in addition to social influence and self-efficacy influence intention to use digital technology for learning. However, the relationships between these factors are realised through different configurations of conditions. Taken together, these results would predict that university students who have a positive perception about their literacy skills and are proficient in using ICT, would be more likely to have high intention to use digital tools and devices in their learning.

By applying fuzzy-set qualitative comparative analysis on a sample of 49 Finnish and 37 Italian students, this method enabled us to understand the contribution of different conditions leading to the occurrence of the outcome. In other words, based on the equifinality concept, fuzzy-set qualitative comparative analysis allows us to see how variables combined lead to outcome to occur. Moreover, fuzzy-set qualitative comparative analysis as a novel method in this line of research, enabled us to perform an in-depth investigation on the complexity of the role that digital literacy and information literacy play in the educational environments. As such, we were able to identify several configurations where these two dimensions of literacy play a central role. In other words, our findings showed there are multiple significant pathways that may predict students’ intention to use digital technology, rather than a single linear approach that could be applied to all. Regarding to literacy skills, the results show that for Finnish students, digital literacy plays a major role (see Table 3), while for Italian students, information literacy is the dominant factor appearing in all three configurations (see Table 4). However, in addition to these two dimensions of literacy, social influence and self-efficacy also influence the formation of decision to use digital technology for learning purposes.

Moreover, we assess students’ proficiency (self-reported) in using digital tools and their frequency of use (self-reported) of digital devices. We found that Finnish students, in average, scored higher than Italian students in rating their proficiency in using and frequency of use of digital tools and devices. These differences are explained in Table 1 and 2.

Our findings have some practical implications for university staff as well as decision-makers in educational environments. For example, university teachers and course designers must pay a close attention to the students’ digital competence, especially those students with low information and digital literacy skills and take the necessary actions. They could do so, e.g., by developing some courses and workshops that are explicitly designed for such students. In addition to literacy skills, of equal importance is students’ perceptions of their self-efficacy. Thus, policymakers in educational environments must be careful in investing sufficient resources to enhance and improve students’ self-efficacy, especially for those who suffer from a low self-efficacy. For instance, by providing training sessions aiming at enhancing students’ ICT and computer self-efficacy. Considering how new generations of university students interact with each other through available digital technology (e.g., internet, social media applications), it is expected that university teachers and course planners pay attention to the impact that students receive (social influence) from their peers and fellow students for committing a particular behaviour or not. In summary, we argue that the use of digital technology in educational environments does not alone improve students’ performance, and that decision-makers need to place pedagogical reforms in the driving seat as much as they put and invest in digital technology.

Our study has some limitations. For instance, we only used a handful of factors (i.e., social influence and self-efficacy) in addition to digital and information literacy to examine the intention to use digital technology. Future research can examine the same phenomenon by incorporating other factors (e.g., media literacy, habitual behaviour, perceived enjoyment) in their studies. Sample size and the limited number of participants in both countries might also be considered as a limitation in this research. However, as we applied a method (i.e., fuzzy-set qualitative comparative analysis) which is less sensitive to sample size, we argue that the sample size is not an issue in our research (Fiss, 2011).

Acknowledgements

This work was supported by Academy of Finland, project The Impact of Information Literacy in the Digital Workplace [grant number 295743]. This research was also supported in part by the Foundation for Economic Education (LSR) grant.

About the authors

Shahrokh Nikou is a Docent of Information Systems and a senior lecture at the Department of Information Studies at Åbo Akademi University in Finland. His primary areas of research include information and knowledge management, literacy, digitalisation, digital platforms, and information practices. The contexts of his research ranges from higher education, creative economy, social media, and corporate and public organisations. He can be contacted at shahrokh.nikou@abo.fi
Andrea Molinari, graduated in Economics with a specialisation in Computer Science, since 1990 is contract Professor at the Universities of Trento (Italy), at the University of Bolzano (Italy) and at Åbo Akademi University in Finland. His research interests include e-learning, virtual communities and information systems. Since 1998, he is coordinating the distance learning research team currently part of the newly created Laboratory of Sport, Technology and Research at the University of Trento. He can be contacted at andrea.molinari@unitn.it
Gunilla Widén, is Professor of Information Studies, Åbo Akademi University, Finland. Her research interests are within information and knowledge management, and information behaviour. She has led several large research projects focusing information culture, social capital theory, youth information, and information literacy. She can be contacted at gunilla.widen@abo.fi

How to cite this paper

Appendices

List of Items

Social influence:

1. Most people who are important in my life approve of my using of digital technologies.
2. Most people who are important in my life think it is desirable that I use digital technologies.
3. Most people who are important to me think I should not use digital technologies.

Self-Efficacy

1. I will be able to achieve most of the goals that I have set for myself by using digital technologies.
2. When facing difficult tasks, I am certain that I will accomplish them by using digital technologies.
3. In general, I think that I can obtain outcomes that are important to me by using digital technologies.
4. I believe I can succeed at most any endeavour to which I set my mind by using digital technologies.
5. I will be able to successfully overcome many challenges by using digital technologies
6. I am confident that I can perform effectively on many different tasks by using digital technologies.
7. Compared to other people, I can do most tasks very well by using digital technologies.
8. Even when things are tough, I can perform quite well by using digital technologies.

Dimensions of Digital Literacy (Technical)

1. I know how to solve my own technical (ICT related) problems.
2. I can learn new digital technologies easily.
3. I keep up with important new digital technologies.
4. I know about a lot of different digital technologies.
5. I have the technical skills I need to use digital technologies for working/learning and to create artefacts (e.g. presentations, digital stories, wikis, blogs) that demonstrate my understanding of what I have learnt.
6. I have good digital technology skills.

Dimensions of Digital Literacy (Cognitive)

1. I am confident with my search and evaluate skills in regards to obtaining information from the Web.
2. I am familiar with issues related to web-based activities e.g. cyber safety, search issues, plagiarism.
3. I use my cognitive skills to find, evaluate, create, and communicate information.

Dimensions of Digital Literacy (Social-emotional)

1. Digital technology enables me to collaborate better with my peers on project work and other learning activities.
2. I frequently obtain help with my university work from my friends over the Internet e.g. through Skype, Facebook, Blogs.
3. ICT enables me to collaborate better with my peers on project work and other learning activities.

Information Literacy

1. When given a work task, I feel confident determining what information I need to search.
2. I am sometimes unsure of how much information I need for solving work related problems. (reverse)
3. I can easily get my hands-on right information when needed.
4. I understand the organization of information in my company.
5. When looking for information I can easily identify the right information sources (e.g. colleagues, intranet/database, clients, and partner organizations).
6. I can determine the authoritativeness, correctness and reliability of the information.
7. I am not confident that the information I get is accurate. (reverse)

Intention to Use

1. I will not hesitate to use digital technologies to access information.
2. I plan to use digital technologies to seek information.
3. I intend to use digital technologies to seek for information.
4. I am very likely to use digital technologies to gain information.
5. I will continue using digital technologies in the future.
6. I will recommend my friends to use digital technologies.