Research methods and techniques in Spanish library and information science journals (2012-2014)

Introduction

There is a long tradition of studies that evaluate the research methods and techniques used in library and information science (Delgado, 2002). Jarvelin and Vakkari (1990; 1993; Tuomaala et al., 2014) are probably the most authoritative authors in this field, having constantly evaluated methodology throughout their academic careers. Their work, which has provided the basis for many other writers, offers four analytical categories to order and broadly characterize the current set of existing studies on library and information science scholarship: the type of research publication the study examines (papers, theses, conference proceedings, etc.); the period of time it considers (e.g., the study examines research publication over a three-year period or across a decade); the scope of its analysis (it considers which topics researchers have addressed and/or the methods and techniques they have used); and finally, its focus on either national or international publication.

The type of research publication favoured by most studies is the journal paper, although some studies consider doctoral theses (Blake, 1994) and conference proceedings (Ríos, 1998). Studies tend to select their papers from the highest quality journals (indexed in Web of Science or Scopus) and analyse only a few papers (usually between five and ten). The broadest study is Koufoginnakis (2004), which covered 97 journals, followed by Davarpanah and Aslekia (2008) on 56 journals, and Hider and Pymm (2008) on 20 journals. Some studies analysed just one journal, such as the Revista Española de Documentación Científica (Ríos, 2001).

Standard studies focus on time periods of just a few years, although some analyses cover longer periods. For example, Atkins (1999) analysed a decade and Morena de Diago (2013) covered 30 years (1981-2010), although this was a only a qualitative study. Very few studies examine trends in different time periods. The most comprehensive is that of Tuomaala et al. (2014), which compared data from 2005 with those from 1965 and 1985. Jarvelin et al. (1993) were the first to analyse trends and this kind of analysis is also found in Hider and Pymm (2008).

About the scope of analysis, most studies consider the choice of topic in the publication and the research methods and techniques that were used. However, some studies only analyse a specific methodological approach. For example, qualitative research was the exclusive focus of studies by authors such as Àngel Borrego (1999), who analysed the methods, topics and techniques found in three international journals, or Morena de Diago (2013), who analysed qualitative research papers published in the period 1981-2010. In some cases, authorship was also analysed. Davarpanah and Aslekia (2008) examined the authorship and citations of 894 papers published in fifty-six library and information science journals indexed in the Social Sciences Citation Index (SSCI) in 2000-2004. Aharony (2012) undertook a descriptive analysis of the authors (number, geographic distribution and affiliation) of 415 papers published in 2007-2008 in the ten library and information science journals with the highest impact factor in the Journal Citation Reports (JCR). Penta and McKenzie (2006) analysed the presence of public library professionals as authors of journal papers in 1999-2003.

Finally, about the focus on either national or international publication, most studies have centred on international journals although some have examined one country in particular. These include studies on Denmark (Kajsberg, 1991), Taiwan (Lin, 2011) and Malaysia (Thavamani, 2014). In Spain, Virginia Cano (1999) analysed 354 papers published in the Revista Española de Documentación Científica and in Documentación de las Ciencias de la Información in 1977-1994. Subsequently, there were similar studies by Rís (1998), Guallar (2003), who focused on the topic of journalistic documentation, and Kawalec (2013), who analysed the subjects covered in 1051 papers from the period 2000-2010.

Spanish papers account for 5% of the library and information science literature worldwide, which is the same percentage as China. Therefore, Spain occupies third place after the United States, which produces 36% of the total papers, and the United Kingdom, which generates 9% (Walter and Wilder, 2015).

In this context, the present study focuses on papers published in Spanish journals in 2012-2014 and analyses the research methods and techniques used by the authors, the topics they addressed and their authorship affiliation.

Objectives and method

The general objective of our study was to analyse the research methods and techniques used in the top Spanish scientific journals on library and information science, and to compare the results with those of similar studies.

We formulated the following questions:
- What is the percentage of research papers in Spanish journals?
- What research topics are the most prevalent?
- What is the profile of the authors, in terms of country of origin, sex and professional status?
- What are the most common research methods?
- What are the most common research techniques?
- Is there a correlation between the profile of the authors and the publication of research papers?

We analysed papers published in Spanish library and information science journals that are indexed in the Web of Science (WoS, Thomson Reuters) and Scopus (Elsevier). The following journals included in the analysis can be seen in Table 1 and descriptive data on these journals can be found in Appendix 1.

Our study covered a period of three years (2012-2014), which we considered sufficiently long to reflect the normal publication activity of the journals in question and comfortably contain any changes in publication pattern caused by specific calls for papers or proceedings. The total number of papers analysed was 580. The distribution of papers per journal is shown in Table 1

As indicated in Table 1, there was a notable difference in each journal's contribution to the output of high-profile Spanish library and information science papers. One journal (EPI) published a substantial number of papers (232 out of 580 or 40% of the total). This was followed by 111 papers in the second largest publisher (REDC, with almost 20% of the total). Together, the number of papers in these first two journals comprised 60% of the total sample. The gap narrowed between the remaining journals, which published between 48 and 78 papers, with the exception of one journal (Cybermetrics), which only published 5 papers (under 1% of the total).

Various indicators were analysed for each paper:

The proposed research methods and techniques were based on an analysis of the categories used by specialists in library and information science research methodology. They were drawn from guides on research methods such as Busha (1990) or Powell (1997), or from studies on the use of methods in publications (e.g., Feehan, 1987; Järvelin-Vakkari, 1990; Blake, 1994; Dimitroff, 1995; and, Hider and Pymm, (2008). The paper by Järvelin and Vakkari is probably the source of most agreement between authors who have carried out similar studies. Appendices 2 and 3 list our research methods and techniques.

The subjects we used were based on the Tesauro de Biblioteconomía y Documentación (Thesaurus of Information Science and Librarianship) (Mochón and Sorli, 2002), produced by the Institute of Documentary Studies on Science and Technology (CINDOC). The thesaurus was designed to address the lack of Spanish lexicons covering all the semantic fields represented in scientific and technical texts on library and information science published in Spain.

In all cases, data from previous studies are given when possible to provide contexts for comparing the results. However, we are aware of the difficulties in making such comparisons, due to the lack of common categories.

Results and discussion

Percentage of research papers

Research had a strong presence in the main Spanish library and information science journals (68% considered research). The percentage of research papers was much higher than that found in previous studies in Spain, such as Ríos Hilario (2001), in which such papers represented about half the total number of published documents (45.5%), Delgado (2002) at 44.6%, Delgado (2002) at 39%, and Guallar (2003) at 34.7%. In the long decade between the compilation of these data and those presented here, there appears to have been a considerable increase in research activity in specialized library and information science literature in Spain, even when we take into account the differences in the samples analysed in each study. Furthermore, the percentage of research papers found in this study is only four points below the value (72%) stated in a study by Tuomaala et al. (2014), using data on international journals from 2005.

Research topics

Table 4 shows the results in terms of the main topics of the papers. Twelve topic categories were created grouping descriptors from CINDOC's thesaurus. Appendix 4 shows the equivalences.

The three main research topics were information sources (15%), metric studies (14%) and technologies (12%). In addition, a considerable number of papers examined communication (9%). This figure is directly related to the expansion of the topics covered by the EPI to communication, and the publication of various monographs on this topic. Therefore, EPI contained most of the papers on this topic.

Although it is difficult to draw parallels, the main topics identified in the study by Cano (1999) were library and information science services, information retrieval and scientific and professional communication, which indicates that there has been a shift in interest. A more recent study by Kawalec (2013) found that the main topic in Spanish library and information science publications was information sources, support and channels (24% of the total), followed by three topics at the same level (13%): information treatment for information services; industry, profession and education; and the sociology of information. In this case, the first place was also occupied by information sources. The main topics in international papers (Tuomaala et al., 2014) were information storage and retrieval (30%), scientific and professional communication (24.3%) and library and information science services (17%).

Surprisingly, our data reveal a low number of papers on archival science (3%) and on users and technical process (both at 5%). Clearly, papers on archival science are mainly found in journals that are specialized in this field (Lligall, ISSN 1130-5398, and Boletín ANABAD, ISSN 0210-4164) and do not appear in more general journals such as those analysed in our study.

Another research topic that is not found in the Spanish literature is library and information science methodology. Tuomalaa et al. 2014) only found 3 articles in their study, and none were found in the present analysis.

An analysis of the difference in topics found in research papers and non-research papers shows that some topic categories are closely linked to research. This is the case of metric studies (19% vs. 2%) and theoretical principles (10% vs. 4%). In contrast, two topics are clearly associated with non-research papers: information units and services (15% vs. 3%) and information and communication technologies (19% vs. 9%). This is because papers on metric studies and theoretical principles tend to be by academic authors (from universities and research centres) who are closely associated with research, whilst papers on information units and services and ICT tend to be by authors with a professional profile.

Authorship

Authors by country

The data in Table 5 show a clear majority of Spanish authors, at 78% of the total. In other words, just over three quarters of the papers were written by Spanish authors, without the collaboration of writers of other nationalities. The percentage of papers published in Spanish library and information science journals by non-Spanish authors was only 16%. Papers co-written by Spanish and non-Spanish authors made up only 5.17% of the total, which is a low percentage of collaboration between countries. When the data were broken down into research or non-research papers, similar percentages were found.

Table 6 shows an analysis of international authorship. There was a considerable difference between the number of contributions by Latin American authors and those by authors of other nationalities. Over half of the papers (64.2%) were by authors from this region, including 13.5% of research papers and almost 6% of non-research articles. Quite far behind in second place were combinations of Latin American and Spanish authors of research papers (almost 4%). In contrast, the second largest group of authors of non-research articles was European authors from countries other than Spain (almost 5%).

A detailed examination shows that authors were from a wide range of countries (over 20) mainly in Latin America, followed by countries in Europe and the United States. The number of international authorships overall was not very high.The country of origin for most of the non-Spanish authors was Brazil, with 26 papers. This represents only 4% of the total number of papers published in the seven Spanish journals. The next countries, which accounted for 2.5% of the total papers, were Mexico, Cuba, Colombia and the United States. Nationalities were counted as follows: if there were two Spanish authors and one Latin American author, the authorship was considered Spanish-Latin American. Row data can be found at the Appendix.

Table 7 shows authors grouped according to whether they were Spanish, from other countries, or a combination of Spanish authors and authors of other nationalities (see also Table 5). The proportion of Spanish authors was very high (81.2%), whilst the proportion of authors from Spain combined with authors of other nationalities was negligible (0.7%). Authors from countries other than Spain accounted for 17.3% of the cases.

Finally, to complete this section, Table 8 shows 20 papers that were co-authored by writers from Spain and from other countries. Country combinations are only included when the number of co-authored papers is over two. The countries with the largest number of co-authors of papers were Colombia and Portugal, followed by Brazil, Cuba, Argentina and France. These are all Latin American and European countries.

Academic and professional authors

Table 9 shows the authors grouped according to whether they were academics (including teachers, researchers and postgraduate students), professionals or both. The data show that 61.89% of all papers were by academic authors, 21.72% by professionals and 15.68% by a combination of academics and professionals.

This number of academic authors is low in comparison with high profile journals such as the Annual Review of Information Science (ARIS), Information Processing and Management (IPM), Journal of the American Society for Information Science and Technology (JASIS&T), Journal of Documentation (JDOC), Journal of Information Science, and Scientometrics, in which the proportion of academic authors is at least 80% (Schlölg et al., 2008). However, the low number of professional authors in academic publications (21.72%) follows the trend revealed in other studies (Zemon and Bahr, 1998).

Clear differences were revealed when we broke down the data on authorship according to whether the paper was a research work. As could be expected, research papers were mainly written by academics and researchers (almost 70% of cases). A further 18% of papers were authored by a combination of academics and professionals. Only 11.7% of research papers were by professionals.

This was not reflected in the non-research papers. In these, the proportion of academic and professional authors was almost equal (45.70% were academic authors, 43% were professionals and 10.75% were a combination of both). Naturally, the percentage of professional authors of non-research papers was much higher than the absolute value for professional authors in all the papers together.

The data in Table 9 indicate that collaboration between academics and professionals could be fruitful in terms of scientific output. A total of 18% of papers co-authored by academics and professionals were the result of research. This is higher than the percentage of research papers by professionals without academic collaboration (12%) and the percentage found in international studies (10%) (Chang, 2016).

Number of authors per paper

The data on co-authorship in Spanish library and information science publications show that, among all the papers under study, there was a clear predominance of works by just one author (41.89%).The numbers of papers by 2, 3 and 4 authors were also considerable, although the percentages decreased progressively as the number of authors increased (29.82%, 16.55% and 8.96% for 2, 3 and 4 authors, respectively). The occurrence of over 4 authors was very low, and there were no papers with over 6 authors.

As in the section above, the stratified data show significant differences in the number of authors per paper between research and non-research articles. In research papers, the main form of co-authorship was that of two authors (33.76%). However, this percentage was very similar to that of papers by just one author (32.74%). The percentages of papers by 3 or 4 authors were above 10%. Only seven papers were by five authors, along with another seven by six authors (only just over 1% of the total). In non-research studies, the main category was very clearly that of papers by just one author (61.29%). Co-authorship would thus appear to be more common among academic authors (who are predominant in research papers), and less common among professional authors (who are predominant in non-research papers).

In the study by Cano (1999), papers by just one author represented 68% of the total. Hence, the percentage of 41.89% obtained in this study indicates a clear drop in single-author papers and an increase in co-authorship.

Authors by sex

Data on authorship by sex indicate that out of a total of 1287 authors, just over half were men: 715 (55.55%) men, compared to 572 (44.44%) women. There were similar percentages of male and female authors of research and non-research papers.

As a reference, data on the sex distribution of university library and information science lecturers provided by the Spanish Ministry of Education, Culture and Sport show a slightly higher proportion of women than men (56% versus 44%). A sex bias is also revealed in publications, although the higher proportion there is of men.

In the professional field, women make up the majority of the members of the Catalan Association of Librarians and Documentalists (COBDC) (82.3%) and of the Spanish Association for Documentation and Information (SEDIC) (74.96%). Therefore, the sex bias in publications is even greater among professional authors.

Research methods

Mention of the method in the abstract

The following table shows whether the research method was clearly described in the abstract or could be easily deduced from it, or whether there was no explanation of the research method in this section of the paper.

Mention of the research method in the abstract is an indicator of quality. Most journals allow authors to choose the style of the abstract and only indicate the maximum number of words. One exception is the Catalan journal BiD, whose abstract format requires the description of different categories, including a Methods section.

If we consider all of the papers analysed, there are almost equal proportions at around 50% of those that describe the method in the abstract and those that do not (289 and 291 papers, respectively). However, if we distinguish between research and non-research papers, the pattern is very different. A total of 274 research papers (almost 70% of the total) contain information on the method in the abstract, compared to 120 that do not. In contrast, in the non-research articles, 171 (over 90%) do not mention the method at all in the abstract, compared to just 15 (8%) that do.

Therefore, we could consider that a description of the method in the abstract of a published paper is an indication of a research work.

However, the fact that an abstract is divided into some established categories does not increase the presence of the method in the abstract.

Qualitative/quantitative research

In this section, we analysed research papers only. As Table 13 shows, quantitative and qualitative methodologies predominated (89% of the total) and both were used in similar proportions (45% of the papers adopted a quantitative approach and 44% used qualitative methods). In 9.6% of the papers a combination of both approaches were used.

These data show a lack of triangulation between methods in research published in library and information science journals. This may be due partly to the fact that bigger studies may be published in more than one research paper. In this case, even if the research adopts both quantitative and qualitative approaches, a specific paper is likely to use just one of these methods.

The difference between the Spanish results and international data are considerable. Tuomaala et al. (2014) reported 58.4% of quantitative methodology and only 14% of qualitative approaches. (The study also included 24% in the category “Inapplicable”.) Furthermore, they found that only 3.5% of papers used a combined methodological approach, in contrast to 9.6% in the present study. As Järvelin and Vakkari (1993) noted, this may be due to the choice of research problems, which has an impact on the choice of methods.

Methods

The most commonly used research method in papers published in Spanish library and information science journals was descriptive research (40.1%) followed by bibliometrics (19%), which together accounted for almost 60% of the total papers. Some way behind in third place was theoretical and conceptual research (11.68%). The remaining methods were found in similar proportions, always below 10%. Thus, evaluative research and information systems design were found in a similar number of papers, around 9%, followed by historical research and literature reviews (5% each) and, at a very low level, experimental and bibliographic research (with only 4 papers each or 1% of the total).

There was little agreement with the study by Cano (1999), in which the main three methods were empirical research (33.6%), descriptive research (20.33%) and discussion (15.5%). The results also differ from the distribution found in journalistic documentation (Guallar, 2003), where a similar classification was used and systems design (40%) was in first place, followed by evaluative research (24%) and descriptive research (20%). This distribution is logical if we consider the professional bias of the study sample.

However, those data put survey methods (26.5%) in first place, followed by evaluation method or experiment (20.9%) and conceptual research strategy (13.9%).

Data on the use of research methods according to the number of authors indicate that two authors was the most common kind of co-authorship for most methods. This was true of descriptive research, evaluative research, information systems design, historical research, literature reviews and bibliographic and experimental research, with percentages between 33% and 66%. Exceptions to this trend were bibliometrics and cybermetrics, in which papers by three authors or more were predominant (38% and 31%, respectively) and theoretical research, in which papers by just one author were the most prevalent (45%).

Techniques

Out of the 394 research papers, 327 used just one technique. Only 62 papers described more than one research technique. Therefore, the average number of techniques per paper is just over one (1.19).

An analysis of research techniques (Table 17) revealed a first group of four techniques that were very frequently used: content analysis, which was associated with descriptive, evaluative and systems analysis research, and applied in almost a quarter of all papers (23.61%); text interpretation, which was found in descriptive and theoretical studies and literature reviews (19.10%); citation analysis, which was associated exclusively with informetrics (17.25%); and survey, which was very common in descriptive research (11.91% of the papers).

A second group of techniques was used with intermediate frequency between the first and third groups. This group includes analysis and design of information systems associated with systems design research; case study, which was always used along with one or more than one other technique and was found in less than 10% of papers; and interview, almost always used in descriptive research and in many cases as a complement to other techniques, in 4.93% of papers.

Finally, a third group of techniques was used much less frequently, with percentages between 2.26% and 0.62% of the papers. From the most to the least common, the techniques in this group were log analysis, observation, secondary analysis, experiment, discussion group and Delphi method. This group also contains techniques whose use is fairly common in other disciplines but negligible in library and information science: experiment, which is very important in some science and technology disciplines, and observation, which is very important in communication. At the bottom of the table are two qualitative research techniques with very specific characteristics: the discussion group and the Delphi method.

In the international arena, the most frequently used techniques (Tuomaala et al., 2014) were information retrieval experiment (16.9%) and questionnaire (15%). However, according to Hider and Pymm (2008), the data collection methods employed most often in library and information science literature in 2005 were either a questionnaire or an interview and previously collected data.

An analysis of techniques according to the type of author (academic, professional or a combination) showed that content analysis was the preferred technique in all three cases. However, in the professional arena the difference from the other techniques was greater. In general, all techniques except the survey were used in similar proportions in both fields and in cases of mixed authorship.

In addition, a smaller range of techniques was used by professional authors: log analysis, secondary analysis, Delphi method, experiment, discussion group and observation were not used, or were only used occasionally by professionals.

The main research techniques used more in papers with only one author were content analysis and text interpretation, which were also the most common techniques in the set of papers taken together. Out of the least frequently used techniques, the Delphi method was more common in papers with a single author (in two out of the three papers that applied this technique). In content analysis and text interpretation, as the number of authors increased the number of papers employing these techniques decreased. For example, in content analysis, there were 42 papers by just one author, 31 by two authors, 26 by three authors and 16 by over three authors (which correspond to 42%, 27%, 22% and 14%, respectively). A similar situation was observed for text interpretation, with figures of 31, 29, 23 and 10 papers (32%, 31%, 24% and 10%, respectively).

For citation analysis, which was the third most frequently used technique in all the papers taken together, the situation was almost the opposite. This technique tended to require larger teams (as we saw in the discussion of research methods). The number of papers by two, three and over three authors that used this technique was the same: 22 papers in each case (26%). On the other hand, the number of single-author papers that applied citation analysis was 18 (21%). Among the least frequently used techniques, half of the discussion group papers had over three authors, which is the opposite of the situation found for the Delphi method.

In general, most of the techniques were applied in papers by two authors. This was the case for survey, systems analysis, case study, interview, log analysis and observation (which was found in the same number of single-author papers) and secondary analysis. There was only one case (experiment) in which most of the papers were written by three authors. Therefore, two authors is the most common number of authors in the majority of library and information science techniques, as was the case with the research methods.

If we analyse these data together with those in Table 10 (number of authors per paper), we find a majority of papers by just one author (in the two most common research techniques) or by two (in the greatest number of different techniques). The exception is citation analysis, which tended to be used in papers by higher numbers of authors.

Conclusions

Our study of seven Spanish library and information science journals over the three-year period 2012-2014 reveals a high degree of consolidation in the our country’s literature. This is evidenced by the high standard of qualitative research, the clear signs of progress in bibliometrics and the fact that the number of research papers published in these journals is comparable to the number published internationally.

The conclusions reached in this study reflect that library and information science research in Spain is in a constant process of change and that its progress remains closely tied to history. In Spain, professional library and information science training dates back to the creation in 1915 of the Barcelona school of librarians, called the Escola de Bibliotecàries. However, Spanish research in library and information science was not undertaken until the mid-1980s (Abadal, 1999) when the subject’s higher educational status was recognized and when schools, faculties and doctoral programmes were established. Consequently, library and information science in this country is still in its early years and lacks experience in the use and development of research methods and techniques.

This also explains the predominance of descriptive research. As a relatively recently-formed discipline which lacks the support of studies, Spanish library and information science scholarship tends to be characterised by state of the art analyses which lead to a second and more detailed stage of research in specific sub-fields (where experimental studies are particularly common). Likewise, the presence of theoretical and historical research reflects the fact that many lecturers who joined library and information science schools and faculties in the early 1990s had received their academic training in the humanities. Added to that, the Spanish case is not entirely comparable with the findings made in international studies that library and information science literature relies heavily on contributions from librarian authors (Walter and Wilder, 2015), given that in our study only 22% of papers were written by professionals.

Library and information sciene research basically started as an individual activity. This explains why, in one study at the end of 1990s, single author papers accounted for 68% of the total (Walter and Wilder, 1999) but have now dropped to 41% as the number of research groups increases and research itself becomes more commonly perceived as a collaborative activity. Furthermore, almost all of the research published in Spain is written by Spanish authors, which illustrates the absence of internationalisation, both of our researchers and our journals.

We consider that training in research methodology will be critical for the future of Spanish library and information science research. We also propose that it may provide the singlemost important means of ensuring a qualitative improvement in research and in the exploration and development of other areas. The increasingly cross-cutting and interdisciplinary nature of many Spanish research groups will also make a major contribution to broadening the range of methods used.

Acknowledgements

This study is a part of the activity of "Culture and Digital Contents" Research Group (2014 SGR 760) funded by AGAUR, research agency of Generalitat de Catalunya.

About the authors

Núria Ferran-Ferrer is Associate Professor in the Department of Information and Communication Sciences, Universitat Oberta de Catalunya (www.uoc.edu). She received her Ph.D. from Universitat de Barcelona. She is currently involved in three research projects: on youth engagement on politics through the analysis of social networks (H2020), citizen sience (H2020) and open access to science (national project). She can be contacted at nferranf@uoc.edu
Javier Guallar holds a PhD in Information and Documentation and is Professor at the Faculty of Library and Information Science of the University of Barcelona. He collaborates with the Culture and Digital Contents Research Group. His main research interests are documentation in the media, digital journalism, scientific publishing, and content curation. He has authored numerous articles and several books, such as El content curator (UOC publishing house, 2013). E-mail: jguallar@ub.edu
Ernest Abadal is Professor at the Faculty of Library and Information Science of the University of Barcelona. Author of several books and papers on digital publications and open access to science. Director of the and Culture and Digital Contents Research Group (http://bd.ub.edu/grups/ccd) and a research project on open access in Spain (www.accesoabierto.net). E-mail: abadal@ub.edu
Adan Server works as a librarian at the Biblioteca de l'Ateneu Barcelonès;. He has been collaborating with Culture and Digital Contents Research Group and also with other research groups driven by University of Barcelona, focussed on the study of taxonomies and metadata applied to the organization and retrieval of journal articles.