The potential for altmetrics to measure other types of impact in scientific production: academic and social impact dynamics in social media and networks

Introduction

The inclusion of tools from the so-called social Web into scientists’ everyday life has brought about several changes in the contemporary scientific communication system. One of these changes is the creation of new metrics targeting the dissemination of research products in social media, which can be used as science impact assessment mechanisms, complementing consolidated studies on bibliometrics and scientometrics. Thus, altmetrics arise as a new area of studies on science impacts.

The new metrics are promising the understanding of the impact of scientific communication in the digital age and its various social medias. They indicate a regard for a diversity of scientific works, reflecting on their use, access, interest and scientific community acceptance rate, as well as other audiences not connected to academia. It can be considered a new area, strongly linked to the scientific community and to the field of information science, with great proximity (and also differences) with the areas of bibliometrics and scientometrics.

Some advantages or benefits commonly attributed to altmetrics concern the new possibilities of measuring academic and scientific production impact, of which the most important are: broadness (possibility of measuring beyond scientific context); diversity (potential of measuring other types of production in addition to scientific papers); quickness (capacity of measuring impacts more quickly when compared to consolidated indicators); and openness (altmetric data are free and relatively easy to collect) (Bornmann, 2014a).

The questions guiding this research come from the need to improve the understanding of broadness or, more specifically, the relation between science and society, in theory, provided by altmetrics. It is often stated that altmetrics have the advantage, when compared to other consolidated metrics, of amplifying impacts between academic production and society in general. For several authors, the new indicators can help draw together different aspects of visibility and scientific impact, in addition to academic impact (Piwowar, 2015; Bornmann, 2014a; Thelwall and Wilson, 2015; Bornmann, 2014b), expanding the audience of these products to other professionals, government and public in general (Adie, 2014).

The ways to calculate indices and indicators from social media data, such as those extracted from Facebook and Twitter, will present a given impact, but what this impact actually is, is still unknown. There are doubts on whether current measurements calibrated by altmetric tools are not reproducing those provided by bibliometrics and scientometrics; i.e., it is not clearly known whether altmetrics are measuring relations among scientists themselves who engage in social media, rather than the relation between science and society.

It seems altmetrics capture an ample, or at least different, aspect of visibility and impact of the investigation comparing to counting citations. “Non-publishing” or “pure” readers are estimated to constitute one third of the scientific community, as they can tweet or post papers on blogs never quoting other papers (Thelwall, Tsou, Weingart, Holmberg and Haustein, 2013). Nevertheless, there are doubts on whether the types of impacts provided by altmetrics still persist to a certain measure and many studies are necessary to comprehend this phenomenon.

A theoretical vision that altmetrics value other types of impact is defended. There certainly is an interaction between individuals and academic production; nevertheless, who compounds such audiences and in what context they are inserted is not clearly known. Analysing aspects and characteristics of social impact and academic impact provided by altmetrics in social media, especially Facebook and Twitter, becomes relevant for the development of the area of altmetrics and the comprehension of new dynamics of science assessment and communication.

This study aims to comprehend how the scientific community, and above all, the community outside academia, is acting and interacting, causing and suffering impact, somehow, by scientific knowledge disseminated in the environment of social media and networks. A possible line of investigation arises from the identification of the characteristics of subjects or audiences behind the actions and interactions in the context of academic publications circulating in social networks.

Thus, some questions arise and are investigated with this research under the perspective of altmetrics:

• Is the impact that altmetrics assess academic (among academics) or social (among people not directly connected with the academic environment)?
• What characteristics do academic impact and social impact have in social media such as Facebook and Twitter in the view of altmetrics?
• What dynamics do academic impact and social impact have in different areas of knowledge?

Hence, this research aims mainly to help building the answers for these questions and concerning such questions, so as to comprehend the dynamics of academic impact and social impact, provided by altmetrics, from the actions and interactions of social actors on Twitter and Facebook.

Altmetrics and their potentialities for the comprehension of academic impact and social impact

The changing and dynamic character of knowledge is an indispensable factor for scientific investigation, since science is built with renovations of its truths and the aggregation of new discoveries in a cyclic manner (Arbesman, 2012). One of the most important characteristics of science is that it is only performed when disclosing its results, and it depends viscerally on communication to be realized (Fausto, 2013). This renovation and disclosure is influenced by new technologies and tools inserted in the scientific process, such as those based on Web 2.0, or the social Web, which has been bringing impact in the development of new methodologies for communication, assessment, promotion and certification of scientific activities.

It is notorious that the means to assess the quality of a publication consists in verifying the rate of interest others have in a given research (Meadows, 1999). Therefore, methods, techniques and tools were developed to help in the activity of assessment. Among several developed techniques in the ambit of bibliometrics, scientometrics, informetrics and webometrics, the analysis of citations received for a given research is highlighted.

One of the best-known measurements in the context of bibliometric and scientometric studies is the impact factor, an indicator based on citations. Despite its limitations, it remains widely used to assess scientific production. This index has been criticized for being insufficient in showing the interest in the scientific research from other audiences; for taking too long to become apparent (a paper may take years to be quoted), and for not having the potential to assess papers individually (the index is focused on the journal and not on the individual paper).

Traditionally, scientific production is assessed observing peers’ proofreading and the indicators produced by means of counting citations. Nevertheless, it is noticeable that these methods are limited when it comes to certain factors, such as: peers’ proofreading being a slow process, which ends up discouraging the innovation; and counting citations being insufficient, as it disregards context and reasons for quoting, limiting its use generally to formal publications, unable to assess the influence scientific works may have outside academia (Souza and Almeida, 2013).

From these criticisms, come altmetrics and their group of new indicators claiming to reduce the limitations of measurements traditionally performed by the areas of bibliometrics and scientometrics. There may be a potential to amplify the comprehension of the impacts of scientific and academic production in society in a broader manner. Although, the term altmetrics points to the idea of an alternative to the consolidated indicators, altmetrics may complement, rather than replace such measurements, as the phenomenon drawn together and measured are relatively different.

The impact factor (an index based on citation) has been used to assess the individual influence of papers and scientists, starting from the assumption that a paper published in a journal of high impact will necessarily have greater influence than a paper published in a journal with a smaller impact factor (Priem, Taraborelli, Groth and Neylon 2010). Altmetrics come with the promise of analysing more fully the impact of papers in journals, patents and such, as well as making it possible to follow up research products out of the scope of traditional filters (Souza and Almeida, 2013).

The term altmetrics was first used on September 28^th 2010 by Jason Priem in a post on Twitter highlighting the preference in using the term altmetrics (alternative metrics) to replace the expression 'article level metrics'. Priem, Piwowar and Hemminger, (2012) define it as 'the study and use of metrics of academic impact based on online activities, tools and environments'. It can also be defined as 'the creation and study of new metrics for the analysis of scientific communication out of the traditional channels of scientific communication systems, such as social networks, blogs, forums, etc.' (Galyavieva, 2013, p. 94).

The practices of altmetrics as a tool to explore academic impact show that none of the approaches, be they alternative metrics or the analysis of citations, makes it possible to fully describe scientific communication. Such techniques show different impacts compared among themselves (Priem et al., 2012). However, very little is known about these impacts, given the complexity of the variables, audiences, actions and reactions involved in the ambit of social media. The very concept of the term “impact” must be rethought and reconsidered, taking into account the context where it is used, as there is no conceptual consensus between its meaning in the academic or social context.

Before deciding whether altmetrics can be accepted to assess scientific impact, it is necessary to understand the meaning of research impact (Sankar, 2015). In the context of altmetrics, differently from bibliometric and scientometric contexts, the concept must be broader, by indicating the idea of contribution to scientific activities in general society, in its various scenarios (institutional, political, cultural, social, economic, etc.). A possibility would be to adopt the concept proposed by Allen (2013), understanding research impact as any academic contribution altering thoughts or practices of individuals or organizations, bringing some improvements.

A recurring point in literature is the advantages of altmetrics, among others, concerning the possibility of producing indicators of research papers and products out of the traditional ambit, of which blogs and computer programs are examples. In addition, they can simplify the verification process of these impacts in relation with audiences, such as research and the public in general (Piwowar, 2013).

Current studies on alternative metrics concentrate on validating the new method (Bornmann, 2014a). It is noticeable that these studies focus on empirical studies applying altmetric indicators and methods to demonstrate in practice the use and viability of altmetrics (Souza 2014). Altmetrics is an area still in construction, which is immersed in much more uncertainties than certainties. Since 2010, when discussions on alternative metrics began arising, until now, many studies have been developed in the area, but the panorama on this topic is still not perfectly clear or consensual (Barros, 2015).

Recently, data citations have gained thrust, reflecting in the development of data-level metrics and correlations studies between altmetric indicators and citations (Peters, Kraker, Lex, Gumpenberger and Gorraiz, 2015). On the other hand, we believed that there is a lack of research to understand political, cultural, economic, social, behavioural and infocommunicational aspects of altmetrics. Since there is a need for theoretical investigation for the nascent area, the central theme of relevance here is to comprehend what the academic and social impacts provided by altmetrics are.

The altmetrics concept accompanies the diversity of web based tools to produce new indicators. The scholars have also led to some studies to understand their relation or the association with established indicators such as citation analysis. Most of these studies have found low, medium and high correlations (among altmetrics and citation scores). But the altmetrics might capture other types of impact, being necessary to develop more large-scale studies with quantitative and qualitative approaches. (Costas, Zahedi and Wouters, 2015).

It is not clear how the impact of the investigation in other areas of society should be assessed, as opposed to the impact of the research on itself. While peers’ proofreading and bibliometrics became standardized methods to assess the impact of research in other research, there is still no accepted structured framework to measure social impact (Bornmann, 2014b). Furthermore, one must observe that the roles and audiences in scientific communication change according to the context, where scientists can also be considered part of the population in general in matters that go beyond their specializations, making it even more difficult to assess the phenomenon.

Some authors have asked whether altmetrics should be accepted to assess the impacts of academic researches and, if so, how. (Sankar, 2015). Altmetrics, when thoroughly adapted, have a fundamental role in performing the assessment of the social impact of the investigation, while the academic impact is, in great part, covered by traditional metrics (Sankar 2015).

Despite the discourses and discussions on the social impact of academic production provided by altmetrics, little is known empirically about its real existence and at what rate. Questions persist into the attempt to clarify the types of impact (social and academic) provided by altmetrics, the central element of this research. This is a fundamental research question, capable of helping to understand potentialities and limits of altmetrics. Hence, it can promote the consolidation of the area and the development of its own theories and concepts, currently depending somewhat on those developed by the areas of bibliometrics and scientometrics.

It is important to mention that there is a growing interest in understanding the disciplinary differences with altmetrics applications. Many studies have been conducted using different media, tools, documents, for various purposes. One paper investigates disciplinary differences in how researchers use the microblogging site Twitter in ten disciplines (Holmberg and Thelwall, 2014). Another study tries to contribute to the understanding of altmetrics in different disciplines of social science (Htoo and Na, 2017). Hammarfelt (2014) analyses the altmetric coverage and impact of humanities-oriented articles and books.

To gain a deeper understanding of the disciplinary differences, Vaughan et al. (2017) investigates the issue in five disciplines of science examining relationships between citation and download data. The research found that social sciences and humanities are different from science, engineering, and medicine and that the pattern of differences are consistent across all measures studied. In general, these studies show pattern of differences across the disciplines. The findings are relevant for further understanding the value of altmetrics, to do coherent comparisons, evaluations and to make correlations at different disciplines. In a similar way, contributions to understanding correlations between altmetric and bibliometric indicators at disciplinary level is also a subject that has been investigated (Costas, Zahedi, and Wouters 2015).

Methods

This study can be considered quantitative. It presents altmetric indicators, focusing on two major and more widely used social media in terms of active users: Facebook and Twitter. The categories analysed therein were: posts, likes, and shares on Facebook; and tweets and retweets on Twitter. The use of the altmetric indicators provide by altmetric.com, across the scientific field, is used successfully by other researchers (Costas, Zahedi and Wouters, 2015).

For the collection of data, the adopted tool was altmetric.com, using a login for tests offered to visitors (explorer login). 100 papers with higher scores were selected (paper Altmetric Score), categorized according to the criteria of the altmetric.com tool, belonging to Scientific Electronic Library Online (SciELO). The 100 papers were selected according to the altmetric score in April 2016. For that, no filters were used for areas of knowledge or selection of journals. Altmetric Score is calculated considering several actions occurring in social media worth points, especially scientific papers, such as: News: 8; Blogs: 5; Questions and Answers Forums: 2.5; Twitter: 1; Google+: 1; Facebook: 0.25 points.

The 100 highest-scoring papers were categorized for the analysis according to the classification of the journals by areas of knowledge proposed by SciELO: agricultural sciences; biological sciences; health sciences; humanities; applied social sciences.

The profiles of those who posted on Twitter or Facebook were identified, as well as those who retweeted (Twitter), liked or shared (Facebook). Therefore the profiles was classified in two categories: Academic Impact (academic impact) or Social Impact (social impact). The individuals categorized in academic impact were those who, in some manner (work, study, etc.) were in the date when data were collected, inserted in the academic universe, such as professors, researches, university students and workers in teaching/research institutions.

The category social impact corresponds to individuals who were not part of the academic community and were not connected with teaching/research institutions on the date when the data were collected. The profiles of individuals from various occupations were identified, such as: community leaders, associations of mothers of victims of illnesses, and activists (environmental, political). Other profiles of institutional character (such as hospitals and pharmaceutical companies) were categorized in social impact or academic impact, considering the content of the posts on the webpage.

The profiles of 947 Twitter and 682 Facebook accounts were identified throughout the month of April 2016, totalling 1629 accounts or profiles. Actions and reactions in these accounts concerning the 100 selected papers were analysed. Inactive or unidentifiable Facebook (23) or Twitter (12) accounts were discarded. A limitation of the study is the fact that Facebook and Twitter individuals did not disclose their real occupation. Thus, individuals categorized as academics might act as part of general public.

Despite the caution at the moment of manually categorizing the user profiles from Twitter and Facebook, considering they are not evident in these social media, one must observe that there may be mistakes. In a similar research (Sanka, 2015), such difficulties were indicated when reporting that there was no absolute evidence of the results found, considering the tenuous classification of the types of users on Twitter, hence there are failures in the categories, lack of information on the location, uncertainties about the credentials of the scientists, societies and professionals, among other limitations. The author considers also the classification of the profiles used in the study (public, scientists, professionals and scientific communicators) is not effective in understanding conclusively the academic and social impacts of the academic productions assessed by altmetrics. Despite the care with data collection, in this research we assume (as well as this author), limitations imposed by the data of Twitter and Facebook. However, at the moment, it seems that there is no solution to such limitation.

The objects, tools, methods sand phenomena we intend to measure in this new area denominated Altmetrics are still little known and explored. The altmetric indicators produced, are still a challenge and lack theoretical, conceptual and methodological developments. If one really wishes these indicators to surpass mere experimentation and academic studies, and to be employed to assess academic activity, it is necessary to resolve theoretical issues (of meaning and conceptualization), as well as methodological (validity of the sources) and technical (normalization) ones (Torres-Salinas, Cabezas-Clavijo and Jiménezes-Contreras, 2013). Thus, this research hopes to contribute by making methodological experiments in altmetric indicators field.

Results and discussions

The graphics produced represent, in percentage, characteristics of the altmetric indicators in both media (Twitter and Facebook). At the first moment, in a field of 100 papers, the division was made by areas and categorized according to SciELO. Thus, it was thus possible to check the quantity of publications in each area, according to the impact and the Altmetric Score, helping to understand the differences of impact by area of knowledge.

The obtained results can be checked in Figure 1. Note that the concentration of papers is in the area of health sciences (57%), followed by applied social sciences (14%), biological sciences (13%), humanities (11%) and agricultural sciences (5%). There were no occurrences of papers in the other areas of knowledge.

The Altmetric Score averages by area of knowledge are also presented in Figure 1. One can observe that the area of agricultural sciences obtained the greatest average and median value compared to the other areas, obtaining an average value of 24.4 and a median of 25 Altmetric Score for each paper (the score does not vary a lot between the five papers). Next are found the areas of Health and applied social sciences with the values 12.2 and 8 and 7.1 and 6, respectively. The lowest averages and median are found in the altmetric scores of the area of humanities, with the values 5.5 and 2, respectively.

Although the area of agricultural sciences has the highest average and median values, it has the lowest frequency of papers found (five papers), making it impossible to infer that this area has the highest altmetric impact. There was nothing special that could explain the phenomenon of greater circulation in social media. There are no news articles and apparently there are not a lot of blog posts written about the papers. In general, the area of health sciences stands out strongly when compared to the others, quite probably because of its strong social appeal as it affects most people’s lives.

The surprise is in the area of applied social sciences, highlighted in altmetric impact, demonstrated by its relatively high median score in social media. Hence, the data can present indications that the area of Social Sciences may have gains when altmetrics are used to better comprehend its structuring and impacts, currently not captured by consolidated bibliometric indices. This corroborates the view of some researchers who see in altmetrics the possibility to improve the comprehension of audiences in the areas of social sciences and humanities, as traditional bibliometric indicators possess limitations in capturing the particularities of these areas of knowledge (Hammarfelt, 2014). However, it should be noted that there are only 14 papers in the data from applied social sciences, referring to the need for more in-depth studies.

In general, a certain proximity is not noticed among the median scores of the papers. There are some papers influencing more strongly than others in the altmetric score, i.e. they had greater audiences than others. However, one cannot compare the quantity of data and observed results and state that there is a similarity of altmetric impact with the bibliometric laws, which profess the principle few with many and many with few.

The social impact and academic impact total classification, distributed in Twitter and Facebook media, can be visualized in Figure 2. The results show that, in the field of 100 analysed papers there are 958 tweets and retweets, where 651 (68%) correspond to the academic impact and 307 (32%) to social impact. Analysing Facebook, a total of 682 posts, likes and shares was found, where 394 (58%) correspond to academic impact and 288 (42%) to social impact.

Note that the results present a significantly higher academic impact in comparison to the social impact for both media. Considering a total of action and interactions in both media, the total reached 595 (36%) corresponding to social impact and 1045 (64%) to academic impact. The conclusion from the data is that academic impact, considering both social media combined, has a difference of 450 actions and interactions, indicating an impact with a 76% difference of academic impact in relation to social impact.

Not much researches was found in literature pointing to understanding who the social actors behind the actions and interactions in social media are. Sankar (2015) conducted a study from the data of Tweets on four Nature journals (Nature Biotechnology, Nature Nanotechnology, Nature Physics, Nature Chemistry). The Twitter profiles were studied considering these categories: public, scientists, professionals and scientific communicators, identifying approximately 61% (general public profiles), 33% (scientists), 1% (professionals) and 5% of the tweets from science communicators, indicating social impact higher than academic impact.

Differently from this research, Sankar (2015) identified, analogously, greater social impact than academic impact. A number of hypotheses could explain the divergences in the results of the researches. One of them is the fact that the data collection performed by the researcher, was in publications from Nature Publishing Group, a commercial group characterized by publishing papers with a strong social appeal whose mission is, explicitly, to present to the more general public the great scientific discoveries and results, stimulating their more generalized use in education and everyday life. It is naturally believed that Nature journals possess higher social impact when compared to other scientific journals with a smaller social appeal.

Another hypothesis that can help to explain the difference between social impact and academic impact identified in the studies might concern the methods used. One may note that, due to the methods used, there were differences in the areas approached by the studies. The selection of journals Sankar (2015) made was in exact and biological areas, while this research did not attempt to direct areas when collecting data, resulting in the inclusion of data from humanities and social areas.

Another aspect that could have influenced the result concerns the methodological choice for selecting journals. As, in this research, the data was extracted from SciELO database, it is possible to have a local or regional bias (database indexes mainly journals of Latin American countries) typical of journals indexed in the base. Thus, it is believed that the results could be different from an analysis made with Web of Science data, for example. But this hypothesis must be better investigated. Alperin (2013), in his discussion on what altmetrics can do for developing countries, reminds us that their publications are underrepresented in international databases, making it more difficult to draw together funds for researchers, journals, projects and institutions in these nations. This author points out that altmetrics can be a more inclusive and democratic alternative than those offered by editors and databases, being able to revert decades of marginalization in the current system. Nevertheless, Alperin (2013) emphasizes that social media are more prominent in the North than in the South, making caution necessary when interpreting the results.

Another noteworthy fact is that Twitter behaves similarly regarding posts and retweets, both in AI and social impact. In percentage, social impact and academic impact in Twitter were identical in those questions, i.e. 32% (posts) and 68% (retweets). A different behaviour is found for the actions and interactions on Facebook. academic impact had greater representativeness than social impact, with percentages of 87% against 13%, respectively. Regarding likes and shares, even though academic impact had the highest values, the distance between them is smaller, with an average of 47% (social impact) and 53% (academic impact).

Facing the data, one observes that the posts on both, Facebook and Twitter tend to be, predominantly of academic origin. The posts (tweets) from Twitter presented higher social impact potential when compared to Facebook (posts). However, the reverberations after the initial post (likes, shares or retweets) are greater on Facebook, with higher social impact when compared to Twitter. Despite not being an object of analysis, it is believed that the announcement of the academic work may have come from the authors themselves or from a social network from an academic organization and only after it reached society outside academia. Thelwall, Haustein, Larivière and Sugimoto (2013), studying tweet links to academic papers, concluded that the most tweeted ones did not directly concern the author of the paper, but some made reference and others were clearly self-citations.

Facebook is probably the best known of all social media tools, more so than Twitter. Both are used by individuals, groups, businesses, and other organizations to connect and share information including photos and videos. One important difference between these two tools is the size (number of characters) that can be published in each one of them. Besides that, Twitter also seems to be used more often for academic purposes, by people and organizations from publishers to individual journals to editors, researchers, and other academic individuals and entities widely represented. (Roemer and Borchardt, 2015). The results obtained confirm this statement (Figure 2).

From the posts point of view, Twitter starts with greater social impact comparing to Facebook (academic impact 113% higher than social impact). However, the reverberation (retweet) of those interactions impact socially in a similar way (academic impact 108% higher than social). With different dynamics, Facebook starts with a lower social impact (academic impact 578% higher than social impact) compared to Twitter. Yet, the reverberations (likes and shares) of these interactions impacted socially much more broadly when compared to Twitter; i.e. the academic impact of the likes and shares grow much smaller, reaching 1.4% and 17.6%, respectively, in relation to the social impact.

In order to better understand this phenomenon, academic impact and social impact were separately analysed in each area of knowledge. This made it possible to check if there are differences in impact dynamics among different areas. Table 1 presents the data segmented by areas of knowledge and the actions and reactions observed in each social network (Facebook and Twitter). The last two columns in Table 1 present, respectively, the percentage difference between academic impact and social impact (academic impact versus social impact) and the difference between the social impact regarding the academic impact (social impact versus academic impact).

Some data presented in Table 1 call for special attention. The areas with higher actions and interactions in the analysed social networks were, firstly, the area of health sciences (892; 54%), followed by the areas of applied social sciences (275; 17%), agricultural sciences (258; 16%); biological sciences (161; 10%), and humanities (54; 3%).

When analysing social impact and academic impact in each area individually, various discrepancies are observed. Although it is necessary to ponder that the number of papers for some areas is low (especially the area of agricultural sciences, with five papers), some analyses are pertinent.

The area of humanities, combining the data from Facebook and Twitter, presented the lowest social impact (11%) in relation with the academic impact (89%). The academic impact of this area is much higher than its social impact, with a percentage difference of 700%. Next emerges the area of biological sciences with a relatively low social impact: 19% social impact and 81% academic impact (percentage difference of 377%), and, thirdly, the area of agricultural sciences, counting on 24% social impact and 76% academic impact (percentage difference of 223%).

On the other hand, the area of health sciences, despite presenting higher academic impact than social impact, is proportionately higher than that presented by other areas. The area of health sciences presented 35% social impact, while its academic impact was 54%, with a percentage difference of 86%. Although there are still great differences between the impacts, they are strong indications that health sciences are the area with the highest potential for social impact when compared to some other areas of knowledge.

The area of applied social sciences, once again, presents surprising data. It is the only area herein studied presenting higher social impact than academic impact, with an identified relation of 68% social impact and 32% academic impact. Hence, the percentage difference between the academic impact and social impact is negative (-52%). In other words, the percentage difference between the social impact and the academic impact is of 109%. One must consider, nevertheless, that, differently from other areas, applied social sciences had a quantity of retweets in dissonance with the data conjunct. In spite of this and the limitations assumed in this research, these are believed to be important indications that the area of applied social sciences has a higher social impact potential when compared to other areas of knowledge.

Despite it all, facing this scenario, the conclusion reached is that there are strong indications that the impact assessed by altmetrics reproduces, in great scale, a scientist-versus-scientist relation, quite as with bibliometrics and scientometrics. Taking into consideration, among the sources of information used to construct the altmetric indices, Facebook and Twitter can be considered with greater social adhesion (when compared to other sources of altmetric data), making possible to infer that the potentiality of altmetrics is considerably higher to assess the academic impact than the social impact of science.

Conclusions

The area of altmetrics and the validity of its indicators are discussed. This research draws on the analysis of the academic impacts and social impacts assessed by altmetrics from actions and interactions of social actors on Twitter and Facebook. The central questions sought to understand:

• If the impact assessed by altmetrics is academic (among academics) or social (among people not directly connected with the academic environment).
• The characteristics of academic impact and social impact in social media such as Facebook and Twitter under the perspective of altmetrics.
• The dynamics of academic impact and social impact in different areas of knowledge.

Despite the limitations made explicit throughout this paper, some preliminary conclusions and findings stand out concerning these questions.

The retrieved papers demonstrated that the ones with highest altmetric scores are those from the areas of health sciences (57%), followed by applied social sciences (14%), biological sciences (13%), humanities (11%) and agricultural sciences (5%), with no occurrences of papers in the other areas of knowledge. A certain proximity was observed among median scores of the papers in general. This research confirms that the papers with the most important altmetric median scores were in the areas of health sciences (8) and of applied social sciences (6); apparently, these areas can be better favoured with altmetric indicators. Surprisingly, applied social sciences take a less visible position when compared with indices of bibliometric citation, especially in the case of peripheral countries. Although the area of agricultural sciences has the highest average and median values (26), it has the lowest frequency of papers found (five papers), making it impossible to infer that this area has the highest altmetric impact.

The results presented a significantly higher academic impact in comparison with social impact for both Facebook and Twitter. Considering the total of actions and interactions in both media, the total reached is 595 (36%) corresponding to social impact and 1045 (64%) to academic impact. The conclusion from the collected data is, hence, that the academic impact, considering both social media combined, presents a difference of 450 actions and interactions, which brings about an impact with a 76% difference of academic impact in relation with social impact.

The tweets and retweets combined totalled 651 (68%) regarding academic impact and 307 (32%) concerning social impact. Analysing Facebook, the total found was 394 (58%) regarding academic impact and 299 (42%) social impact. The posts (tweets) from Twitter presented a greater potential of social impact when compared to the posts on Facebook. However, since reverberations after the initial post (likes, shares or retweets) are greater on Facebook, there is more social impact on that medium than on Twitter. Nevertheless, there are indications that Facebook possesses a higher potential to encourage social impact than Twitter in the context of altmetric indicators.

Regarding the analysis of academic impact and social impact among different areas of knowledge, the highest absolute amount of actions and interactions found in the analysed social media were firstly in the area of health sciences (892; 54%), followed by the areas of applied social sciences (275; 17%), agricultural sciences (258; 16%); biological sciences (161; 10%); and humanities (54; 3%). When analysing social impact and academic impact in each area individually, various discrepancies were found among the areas. Health sciences presented one of the highest social impacts (35%), while the academic impact was 54% (a difference of 86%). However, it is possible that the numbers correlate with the number of papers in the study.

The area of applied social sciences presents surprising data. It is the only one among the analysed areas showing an social impact higher than the academic impact, being the identified relation 68% social impact and 32% academic impact. Thus, the percentage difference between the academic impact and social impact is negative (-52%). In other words, the percentage difference of the social impact regarding the academic impact is of 109%.

Despite this and the assumed limitations in this research, these are indications that the area of applied social sciences and the areas of health sciences possess a higher potential for social impact, when compared to other areas of knowledge. Regardless of the need for further advances in the thematic knowledge, it is certain that altmetrics can assess a greater sector of scientific impact. The data presented by the area of applied social sciences point at this direction, possibly being most useful for areas lacking strongly indexed journals in international databases, especially from developing countries.

The results show clear variances between the subjects. It is important to highlight that the dynamics of academic and scientific production and communication have differences between disciplines and areas of knowledge. These differences between the sciences have been known for a long time, with dangers inherent in ignoring subject matter characteristics and practices of disciplines. In other words, the social structure and output of scholars are different (Biglan, 1973). There are distinct degrees between the sciences (hard vs. soft; pure vs. applied; life system vs. nonlife system) in terms of the various characteristics: socially connected to others; commitment to teaching, research, and service; number of journal articles, monographs, and technical reports that they published, and the number of sponsored dissertations (Biglan, 1973). These differences can provide clues to the understanding of the structuring and impact of the distinct areas of knowledge, which can not be disregarded, since they can help to explain the observed results. In the scope of the altmetric indicators differences between the disciplines are also reported (Zahedi, Costas and Wouters, 2014). Recent searches using altmetrics shows divergences between the distributions across subject fields. For example, Medical and Life and Natural Sciences received the highest proportion of altmetrics in all data sources (more than 30% from Medical and Life Sciences and more than 23% from the fields of Natural Sciences), and other fields, each received less than 10% of total altmetrics. (Zahedi, Costas and Wouters, 2014).

Facing this scenario, strong indications are found that the impact assessed by altmetrics reproduces, in great measure, the scientist-to-scientist relationship, such as with bibliometrics and scientometrics. Considering that, among the information sources used to build the altmetric indices, Facebook and Twitter can both be considered sources of greater social adhesion, it is possible to infer that the potentiality of altmetrics is considerably higher to assess academic impact than social impact in scientific activity. However, altmetrics undeniably provide difficultly assessable social impacts, going beyond the competences of bibliometric and scientometric indicators. It is important to reinforce that the social impact actions and interactions on Facebook and Twitter reached a significant 37%. Before the emergence of altmetria, the collection and analysis of data to understand the impacts of science on science were more complex, there was a strong dependence on perception studies based on questionnaires.

Neylon, Willmers and King (2014) considered that, from altmetrics, it is possible to proactively identify the audiences from a demographic, disciplinary or geographic point of view, investigating which are being reached and using strategies to maximise the impact of academic production. Yet, a restriction of altmetrics is that there is no standard for interpretation available for a large quantity of data, making it more difficult to study certain aspects (Galligan and Dyas-Correia, 2013). More research is still needed to understand the uses, meanings and possibilities of altmetric indicators. The understanding of the quality and reliability of the altmetric data retrieved is necessary before any interpretation and actual uses of these data and indicators. (Zahedi, Costas and Wouters, 2014)

Thus, it is found that the identification of information regarding the understanding of academic impact and social impact of academic researches on social networks (Facebook and Twitter) still presents limitations at the moment of collecting and interpreting data, rendering it impossible to automatically raise precise data on the audiences reached. Other complementary approaches can be tested, as more doubts than certainties still persist concerning the significance of altmetrics and potential impacts.

About the authors

João de Melo Maricato, is professor at University of Brasília (UnB), School of Information Science (FCI), Information Science Graduate Studies Program (PPGCINF). Member of Scientific Communication Research Group, Brazil. His e-mail address is:jmmaricato@gmail.com.
Jayme Leiro Vilan Filho, is professor at University of Brasília (UnB), School of Information Science (FCI), Information Science Graduate Studies Program (PPGCINF). Member of Scientific Communication Research Group, Brazil. His e-mail address is:jleiro@unb.br.

How to cite this paper