Which academic papers do researchers tend to feature on ResearchGate?

Introduction

Research publications are often evaluated with bibliometric indicators from an outside perspective, such as institutions or other researchers. One indicator is the number of citations a paper receives, which is a measure of a paper's impact on the scientific community. Though researchers dispute the association of citation numbers with importance, creativity, quality, eminence and persuasiveness (Shadish,Tolliver, Gray and Gupta 1995), a paper's citation number is generally regarded as providing credit to its authors (e.g., Gazni and Ghaseminik, 2016; Sahoo, 2016). When a paper is written by more than one author, credit assignment methods aim to fairly allocate credit to each author according to their importance to the paper. As the proportion of papers with alphabetical authorship decreases (Waltman, 2012), credit assignment has changed from even assignment, such as normal (or standard) counting (Lindsey, 1980) and fractional counting (de Solla Price, 1981), to uneven assignment (e.g., Egghe, Rousseau and Van Hooydonk, 2000; Trueba and Guerrero, 2004; Liu and Fang, 2012; Stallings et al., 2013). Another simple indicator to assess a paper is the reputation of the journal in which it is published. Not all papers published in high-reputation journals are of a high quality (Haunschild and Bornmann, 2015) and prominent journals have rejected high-quality papers, even those of Nobel-class (Campanario, 2009). The scientific community values papers published in prominent journals because authors are most likely to be proud of the associated prestige. Another possible reason is that papers in these journals have a high possibility of being highly cited (Wang et al., 2012).

Contrary to evaluation from an outside perspective, how researchers perceive their own work is seldom considered. A potential reason for this is that there have been few channels for obtaining such information. The academic social network website ResearchGate, www.researchgate.net (Thelwall and Kousha, 2014, 2015a; Jeng et al., 2017) provides a chance to address this problem. ResearchGate has been used in the study of altmetrics (Haustein et al., 2014; Ortega, 2015; Thelwall and Kousha, 2015b; Hoffmann, Lutz and Meckel, 2016) and found to have a self-quantification function for its members (Hammarfelt, de Rijcke and Rushforth, 2016), although research evaluations based on indicators provided by ResearchGate or other social media are still in dispute (Jordan, 2015; Kraker and Lex, 2015; Jamali, Nicholas and Herman, 2016; Memon, 2016; Nicholas, Clark and Herman, 2016; Orduna-Malea et al., 2017; Thelwall and Kousha, 2017). Here we try to mine other useful information from its contents. ResearchGate members are mainly researchers or working in the sciences and their research products (including papers, datasets and chapters) are listed on ResearchGate. Up to five research products can be selected by each member to be listed in the 'Featured research' section of their ResearchGate home page. Most research products listed as featured research are members' academic publications. A research product has higher visibility when chosen as featured research than it would have when not featured.

To the best of our knowledge, there has yet to be an investigation of featured research in ResearchGate. There are some reports about featured articles on other platforms, such as Wikipedia, where editors award the distinction of featured article to those they have determined to have the highest quality (Ransbotham and Kane, 2011; Ingawale et al., 2013). Additionally, journals feature papers on their cover as the cover paper. Some journals feature the best paper of an issue (such as a future Nobel Prize contender) while others choose those with eye-catching or striking images (Wang, Liu and Mao, 2015). For featured articles both on Wikipedia and in journals, the selection is made by people other than the authors. In contrast, ResearchGate enables its members to select their own research products to feature. According to the criteria for featuring articles in Wikipedia and journals, we supposed that if the featured products are selected by members themselves, ResearchGate members would feature their best or most important publications, or those to which they want to draw the most attention. Under the current academic research evaluation system, citations, source reputation of a paper, and roles played in a paper, can bring credit to researchers. Recent publications represent an author's research advancements and capability.

In this paper, we investigated how researchers view their own publications by comparing ResearchGate members' featured research and other publications. Our research questions are: (a) What kinds of publications are featured by their authors? (b) What do researchers consider when choosing publications to feature on ResearchGate? (c) Are there papers that the author values but others have not recognised? Suggestions for researchers and the scientific community regarding these points were provided according to the answers.

Data collection

Information was collected about ResearchGate members' publications and featured research from universities in the US. These universities were selected from the 2016 National University Rankings by US News & World Report. Collecting data from ResearchGate is time consuming and requires considerable resources, so only one of every five universities out of the first 231 listed (those with a rank provided) were selected (See Table A1). The selected universities had their ranks schemed as : 1, 6, 11, 16, ..., 221, 226, 231. Some universities may have the same rank. For example, Columbia University and Stanford University tie for 5th (they cover rank 6). For such cases, the university with the most ResearchGate members was selected. Following this approach, for the twelve-university tie for 220th (covering ranks 221, 226 and 231), New Mexico State University, University of Massachusetts Boston and Utah State University were selected based on the number of ResearchGate members.

If a member has more than three research products, ResearchGate lists three of them as the default, featured research according to the recency of the publication, its citation and view counts, among other criteria. Members can change this listing and select up to five featured research products. As many members might not notice or care to use this function and do not change the default, we needed to identify those who did select their featured research. Members with more than three featured research products set the featured research by themselves. Therefore, for each university selected, we investigated only the ResearchGate members with four or five featured research products. Table A1 lists the numbers of such members from each university.

In this study, we compared the reputation of sources (academic journals and conferences) publishing the featured research and other publications of the selected members. We only considered papers whose sources are listed in ResearchGate. Some ResearchGate members feature all their publications; thus, information about such members is meaningless for the statistics in this study. Therefore, we only counted ResearchGate members with at least 10 publications whose sources are listed in ResearchGate. A total of 2,708 ResearchGate members were investigated. We collected information about their cumulative 95,424 publications and 11,821 of those chosen as featured research. Note, featured research is designated in ResearchGate, while all the featured research products we selected for investigation were papers published in academic journals and conferences that were denoted as featured publications.

The reputation of the publication source is represented by SCImago Journal Rank (SJR) from Elsevier-Scopus, a size-independent indicator of journals' and conferences' influence or prestige (Gonzalez-Pereira, Guerrero-Bote and Moya-Anegon, 2010; Olmeda-Gomez and de Moya-Anegon, 2016). 2015 SJR was used to measure the sources. For the renamed journals, we used the SJR value of the journal under its new title. If a journal or conference had no SJR, then we set it as 0.

Besides the source, we also obtained the publications' ages, citations received (collected by ResearchGate), the author rank of the ResearchGate member, number of authors per publication and the academic positions of selected ResearchGate members who listed their academic positions in ResearchGate. Data collection was completed on 1 November 2016.

Some readers might not know how to edit their own featured research in ResearchGate. The steps for ResearchGate members to set their featured research are as follows: click on their avatar in the upper-right corner of the ResearchGate page and find the Featured research section in the Overview tag page. Next, scroll to the Edit link in the upper-right corner of the section (the link is an inconspicuous pen icon when the mouse is outside the Featured research section). Click this link and a pop-up dialog box will appear, in which ResearchGate members can select or delete their own featured research products on their ResearchGate home page.

Methodology

Estimation of authorship tendency of featured research

ResearchGate only lists up to four authors for each publication: i.e., the first, second, third and last author. If a publication has more than four authors, ResearchGate provides a link indicating the number of remaining authors. When the link is clicked, up to 50 additional authors are visible. If the target author is not in the expanded list, the link can be clicked again to reveal the next 50 authors.

We used the author rank information of the inspected ResearchGate member based on the following criteria: whether he or she is the first, second, third or last author. This is because those positions are usually used to represent author importance. Our investigation is to check whether the member is important to his or her publications and thus there is little need for the time-consuming and tedious work required to check author position by expanding the link.

To differentiate the importance of a ResearchGate member in each paper with his or her various positions in the by-line among his or her publications, we used the axiomatic approach (Stallings et al., 2013). This assigns credit of a given paper to the co-authors to estimate the proportion of the contribution made by the ResearchGate member to the paper. We used the axiomatic approach with equal contribution groups. We tied the first and last author for the most important author, because many last authors are leaders of the research (Zhao and Strotmann, 2011) and are the corresponding or senior authors (Riesenberg and Lundberg, 1990; Liu and Fang, 2014). This created the possibility for error because some last authors are the least important; this kind of error will be discussed in the following sections. We tied the authors other than the first, second, third and last (denoted as middle authors for short) for the fourth and least important authors. This also presented the potential for introduced error and will be discussed in the following sections. Then the credit assigned to an author was estimated as (Stallings et al., 2013):

            (1)

where m is the number of author groups based on author importance. If the paper has one or two authors, then m = 1. If the author number is three or four, then m is 2 or 3, respectively. If the authors are more than four, then m is 4. r is the rank of author group. If the ResearchGate member is the last author, then r is 1, tied for the first author. If there are more than four authors of the paper and the ResearchGate member is a middle author, then r is 4. Otherwise, r is the author rank of the member in the by-line. n_k is the number of authors in the k-th author group. If the author number is 1, then n₁ = 1, otherwise n₁ = 2. If the paper has more than two or three authors, then n₂ = n₃ = 1. If there are more than four authors, then n₄ equals the number of authors minus 4.

We then ranked each selected ResearchGate member's papers using the member's estimated importance to the papers, according to Equation 1. The higher the c(r, the greater the importance of the member to the paper. Each member's papers are ranked in descending order of c(r). We gave each rank in this queue a normalised score from 0 to 1. Supposing a member has N papers in such a queue, the score of the k-th (1 ≤ k ≤ N) paper is:

            (2)

If several papers have the same c(r), we used the average of their scores to replace the individual scores. The average and median value of all the paper scores in the queue is 0.5.

Suppose a member has N_f featured publications inspected in this study. We denoted the score given to the i-th (1 ≤ i ≤ N_i) featured publication as s_{ac_f}(k_i), where k_i is the rank of the i-th featured publication in the queue. If the member featured publications in which he or she played an important role, then the average value of all s_{ac_f}(k_i) is low.

Estimation of recency tendency of featured research

To investigate the recency tendency of featured publications, we ranked each ResearchGate member's papers in descending order of publication time, which is when a paper was published by the journal that accepted it. The first paper (rank 1) in this queue is the most recently published. We gave each rank in this queue a normalised score, similar to Equation 2. The only difference is that we used the rank of recency of the papers to replace the rank of author's importance. We denoted the score of the i-th featured publication in this queue as s_{rec_f}(k_i).

Estimation of citation tendency of featured research

To investigate the impact tendency of featured publications, we ranked each selected ResearchGate member's papers in descending order of their number of citations. Because citations require time to accumulate, we only compared papers published before 2015; thus, most of them had at least two years to accumulate citations. We gave each rank in this queue a normalised score, similar to Equation 2. The only difference is that we used the rank of paper citation numbers to replace the rank of author's importance. We denoted the score of the i-th featured publication in this queue as s_{cit_f}(k_i). In this comparison, we chose the ResearchGate members with more than ten publications published before 2015. We used 67,397 selected papers, 6,200 of which are ResearchGate members' featured publications.

The number of citations increases with a paper's age. A newer publication may have lower citations at the time of investigation than an older one, not because its potential total impact is low, but because it is more recent and thus has not had enough time to accumulate a high number of citations. To correct this bias in comparing papers with different ages, we also ranked the papers of each selected member according to their average citations a year, or citation rate (Cano and Lind, 1991), in descending order. The score of the i-th featured publication in this queue is denoted as s_{citr_f}(k_i). The samples used in this comparison are same as those in the previous paragraph.

Estimation of source reputation tendency of featured research

To investigate the source reputation tendency of featured publications, we ranked each selected ResearchGate member's papers in descending order of the publication sources' SJR. The first paper (rank 1) in this queue was published by the most reputable source. We gave each rank in this queue a normalised score, similar to Equation 2. The only difference is the use of the rank of source SJR in place of the author's importance. We denoted the score of the i-th featured publication in this queue as s_{sr_f}(k_i).

Results and discussion

Authorship tendency of featured research

Figure 1 shows the cumulative distribution of s_{ac_f} of all the selected ResearchGate members' featured publications in their own paper queues ranked in the descending order of the proportion of paper credit assigned to the member. The average and median value of s_{ac_f} is 0.418 and 0.381. If there is no preference in the selection of featured publications by ResearchGate members, s_{ac_f} obeys uniform distribution and its mathematical expectation is 0.5. Thus, it denies that ResearchGate members did not consider their rank in by-lines when they selected publications to feature, with p < 0.001. Members tended to select papers in which they were important authors as featured publications.

When we estimated the importance of ResearchGate members to their publications, we regarded the last author as the most important; however, there are situations in which the last author is the least important. Additionally, middle authors have differing levels of importance and might have a higher status in some papers (e.g., they may be corresponding authors or co-senior author). Thus, this could introduce errors in the estimations. Here, we use the data in Table 1 to support the above conclusion. In more than 60% of the featured publications, ResearchGate members are the first author, who is of high importance for papers other than those with alphabetical authorship. When all ResearchGate publications are considered, the proportion of papers for which the ResearchGate member is the first author is lower (36.1%). The proportion of papers naming the ResearchGate member first is comparatively higher among featured publications, which holds true for all the ResearchGate member groups that we identified. In addition, for all identified member groups, the proportions of papers in which the ResearchGate member is the second, third, middle and last author are smaller for featured publications than among the complete list. The only exception is that for the group of ResearchGate members identified as research leaders (defined in Table 1 and the following paragraph), the proportion of featured publications with the ResearchGate member as last author is higher than that of all the papers inspected. In particular, papers in which the ResearchGate members are the middle and last authors only account for 7.2% and 14.9%, respectively, of featured publications; both proportions are distinctly lower than the corresponding proportions of all inspected publications. Additionally, among the featured publications in which the ResearchGate members are the last authors, about one-third of papers (572 out of 1762) have only two authors, the second of whom may have also played an important role even if he or she is not the corresponding or senior author.

Table 1 also shows that for nearly 15% of the featured publications with ResearchGate members as last authors, the ResearchGate members are research leaders (including professors, associate professors and deans). Some ResearchGate members were marked only as deans (or department chairmen) without other academic titles. As deans usually are of high academic reputation, we grouped them together with professors and associate professors as research leaders. Nearly one-third of research leaders' featured publications are those in which they are the last authors, indicating senior authorship. This proportion is much higher than that of all publications inspected (21.9%), all featured publications inspected (14.9%), featured publications of assistant professors and lecturers (11.5%) and those of junior researchers (3%). Especially, the much lower value of this proportion for junior researchers demonstrates that junior researchers are less likely to be the last author (which is mainly occupied by senior authors) or they avoided featuring papers in which they did not have an important role.

Table 1 indicates that the importance of the ResearchGate members for some of their featured research is lower than their importance for most of their publications. One cause might be the errors introduced from the treatment of the middle and last authors, as aforementioned. The other cause is most likely, as we will show that ResearchGate members considered other aspects when they selected publications to feature.

Recency tendency of featured research

The cumulative distribution of s_{rec_f} of all the selected ResearchGate members' featured publications in their own paper queues, ranked in the order of the recency of their papers, is similar to the curve in Figure 1. The average and median value of s_{rec_f} is 0.381 and 0.328, respectively. Similarly, we reached the conclusion that ResearchGate members tend to select recently published papers to feature (p < 0.001), which is consistent with the long academic tradition that researchers list their most recent publications first.

Citation tendency of featured research

Figure 2 shows the cumulative distribution of s_{cit_f} and s_{citr_f} of all the selected ResearchGate members' featured publications published before 2015, in their own paper queues ranked in descending order of citations and citation rate. The average and median value of s_{cit_f} is 0.409 and 0.387, respectively. The average and median value of s_{citr_f} is 0.340 and 0.283, respectively. Thus, we concluded that ResearchGate members tend to select highly cited papers to feature (p < 0.001).

Figure 2 shows that the cumulative distribution of s_{citr_f} is higher than that of s_{cit_f}, and the average and median values of s_{citr_f} are distinctly lower than those of s_{cit_f}. The figure demonstrates that the selected ResearchGate members considered citation number and recency in their selection of publications to feature, or they valued papers with a high citation rate.

Figure 1 shows the cumulative distribution of s_{ac_f} of all the selected ResearchGate members' featured publications in their own paper queues ranked in the descending order of the proportion of paper credit assigned to the member. The average and median value of s_{ac_f} is 0.418 and 0.381. If there is no preference in the selection of featured publications by ResearchGate members, s_{ac_f} obeys uniform distribution and its mathematical expectation is 0.5. Thus, it denies that ResearchGate members did not consider their rank in by-lines when they selected publications to feature, with p < 0.001. Members tended to select papers in which they were important authors as featured publications.

Source reputation tendency of featured research

The cumulative distribution of s_{sr_f} of all of the selected ResearchGate members' featured publications in their own paper queues, ranked in descending order of the reputation of the publication source, is similar to the curve in Figure 1. The average and median value of s_{sr_f} is 0.386 and 0.348, respectively. Similarly, we concluded that ResearchGate members tend to select papers published by high-reputation sources to feature (p < 0.001)

Multi-aspect considerations in selection of featured research

Figure 3 shows a scatter plot of s_{rec_f} (x axis) versus s_{sr_f} (y axis) for the featured publications inspected. The samples ranged nearly over all possible positions in the space, but were not distributed evenly with a higher density close to the origin. The closer a point is to the co-ordinate origin, the denser are the samples at that point. The scatter plots of other combinations of featured publications rank scores (among s_{ac_f}, s_{rec_f}, s_{citr_f} and s_{sr_f}) are similar. The distribution of points in the scatter plots demonstrates that the investigated ResearchGate members tend to feature publications in which they are important authors, those that are published recently, have a high impact (reflected in a high citation number) and are published by high-reputation sources. If a featured publication does not meet all the above conditions, it usually meets at least one of the above conditions.

Some ResearchGate members value certain conditions (such as the number of citations) more than other conditions; thus, they neglect the latter in their selection of publications to feature. To show their academic standing, some ResearchGate members may prefer to feature highly cited publications, which are usually their old work. Conversely, some members may intend to contribute to the scientific community by featuring their publications and further their careers by seeking co-operation and financial support for their research and/or further development that could increase their citation numbers and thus credit their research, in which case they tend to use recent publications.

ResearchGate members can change the order of their listed featured publications. We ranked the top publication as 1 because visitors usually see it first among the featured publications and the bottom publication is ranked last. We performed multivariate linear regression on the rank of featured publications with s_{rec_f}, s_{cit_f}, s_{sr_f} and s_{ac_f}. We obtained: R_FR = 1.433 x s_{rec_f} + 0.555 x s_{cit_f} + 1.241 x s_{sr_f} + 3.558 x s_{ac_f}, where R_FR is the rank of each featured publication on its author's list of featured publications. When the multivariate linear regression was performed on the rank of featured publications with s_{rec_f}, s_{citr_f}, s_{sr_f} and s_{ac_f}, we obtained R_FR = 1.743 x s_{rec_f} + 0.893 x s_{citr_f} + 0.837 x s_{sr_f} + 3.406 x s_{ac_f}. This demonstrates that ResearchGate members regard their importance to the paper as the most important factor in featuring their publications.

The second important factor is publication recency. Citations (or citation rate) and source reputation are less important than the above two factors. Whether the intention of a ResearchGate member in featuring their publication is to show academic standing or to contribute to the community, he or she needs to choose publications in which he or she played an important role. The coefficient of s_{rec_f} in the regression equation is larger than that of s_{cit_f} (or s_{citr_f}) and s_{sr_f}, which is reasonable because featuring recent publications can both highlight the author's high-level research and create the potential to contribute to the wider scientific community, whereas featuring publications with high citations or those published in high-reputation sources can only show an author's academic standing.

Finally, ResearchGate members may have considered other factors when choosing publications to feature, which we will discuss in the next subsection.

Other considerations in cases of choosing publications to feature

To find whether publications are valued differently by their authors and the scientific community, we conducted a survey of ResearchGate members with a featured publication satisfying all of the following conditions: the member is an important author, the paper was published before 2010 in a low-reputation source and it has few citations. Additionally, the surveyed member should have more than 20 papers used in this study.

We selected featured publications in which the ResearchGate member had an important role because that implies familiarity with the paper. We selected featured publications published before 2010 because these papers have had enough time to accumulate citations. We surveyed the ResearchGate members with more than 20 papers to avoid surveying members with a narrow choice in their selection of featured publications. Eight hundred sixty-eight featured publications authored by 497 members satisfied the above three conditions.

We chose researchers with featured publications with few citations and low source reputation to survey because these factors indicate that the research has not yet been recognised by the research community. A low number of citations of the selected featured publications shows that other researchers have not used the research presented since publication. Publication by a low-reputation source implies that they may have been underestimated by more reputable sources before their publication. These conclusions are based on general experience and may not apply to all ResearchGate members because authors might not have submitted to high-reputation sources. Five of the 868 featured publications selected in the last paragraph were published in low-reputation source and had few citations.

Finally, we surveyed the five ResearchGate members who featured the above five publications that had few citations and were published in low-reputation source. Our question to them is shown below.

We are investigating academic publications using www.researchgate.net. We found that you set several publications as featured publications. Compared with your other publications, the following featured publication is not a highly cited paper and it was not published in a journal with a high reputation. This featured article is: [Featured publication - anonymised].
Would you please tell us why you set this paper as a featured publication?

Three of the five ResearchGate members responded. Response 1 provides an alternative research evaluation criterion, social impact rather than citations in scientific community.

The essay appears originally in the New York Times. Not certain what this cite is.

Response 2 shows that, as first reports of a method or concept may not be referenced in later studies on the same topic, the indication of worth based on citation is potentially flawed (Waltman, van Eck and Wouters, 2013).

Hello! This paper was the first (to my knowledge) of a key new process-full waveform inversion-in borehole seismic work. So, I wanted to establish the technique's lineage.

Response 3 shows the member is confident with the quality of this featured publication. She attributes its low citation to the low popularity (reputation) of the journal in which it appeared. She wants to expand its impact.

Interesting research. My highly cited articles are known, read and cited already. I try to publicize my less cited articles using this feature. These are lesser-known journals, so the awareness of these articles is low. I am trying to increase the awareness on these in order to improve their impact through reading and citation by the academic audience.

Based on the results of a questionnaire on researchers' perceptions of their papers, Aksnes and Rip (2009) concluded that for most of the investigated publications, citation counts reflected well the degree of contribution by articles. However, they found that there are some unimportant papers that were highly cited for a variety of reasons, such as the bandwagon effect or "friend" citations within some groups. Likewise, papers that were considered important by their authors had few citations for a variety of reasons, including the low visibility of the publishing journal, others citing follow-up studies of the paper, papers without follow-up studies because they completely solved a problem, relatively narrow topic, or work considered ahead of its time. The method used here provides a way to automatically search lowly cited papers that were valued by at least one author. However, the performance of the method depends on how many ResearchGate members featured their valued but lowly-cited publications.

Limitations

There are two possible sources of error in our investigation. One is that ResearchGate data have potential errors, such as incorrect paper publication times. In addition, there is uneven coverage on ResearchGate among scholars with different ages because younger researchers are more active on social media than older researchers (Nicholas et al., 2015; Nicholas, Clark and Herman, 2016). Therefore, there are fewer "old" publications in ResearchGate. Consequently, only a small number of featured publications were searchable that were valued by their author(s) but otherwise ignored by the scientific community. The other source of error is our method, such as our estimation of the relative importance of ResearchGate members to their papers. However, we believe these errors do not lead to systematic deviation in the evaluation. Therefore, they have little effect on our conclusions.

One limitation of this work is that the proportion of ResearchGate members investigated in this study to all ResearchGate members at the selected universities is low. Many ResearchGate members may possibly have trouble locating the function to edit their featured research. Thus, the Featured research section on the ResearchGate members' home page is not yet fully effective.

Another limitation of this work is that it does not explore whether ResearchGate members feature their most characteristic research, which is a natural assumption to make. Future work should conduct a content analysis comparing ResearchGate members' self-featured publications and his or her other publications, as well as with the featured work and publications of their peers.

The last limitation is that this study does not validate whether featuring publications in ResearchGate increases their citations. Despite the effect of featuring publications on citation counts, this work found featured articles with none or few citations.

Conclusion

Using the collected ResearchGate data, we found that researchers tend to feature their own publications in which they are important authors, or were recently published, highly cited or published in high-reputation sources. We also identified research publications valued by the author, but which have not been recognised by others.

ResearchGate provides opportunities for researchers to highlight their academic level. Therefore, it is unsurprising that researchers prefer to feature publications in which they are important authors. When researchers evaluate their peers' publications, they consider the citation number and the reputation of the publication source. Our findings show that most researchers treat their own achievements and those of others similarly, or they cater to such evaluation criteria in the selection of featured publications. However, this is not always the case. In some cases, quality is not indicated by citations or source reputation. Pioneering work may be neglected in citations.

ResearchGate's Featured research function also facilitates its members to contribute to the greater scientific community. From this perspective, some ResearchGate members prefer to feature recent published papers in which they played an important role to show what they can do for others.

We did not find variations in trends of authorship, recency, citation and source reputation tendency of the featured publications based on the rank of the universities affiliated with the ResearchGate members or the members' academic position (this was not listed for brevity's sake).

We did find academic papers that were valued differently by the authors versus the scientific community. We believe there must be other similar papers because our data did not cover all universities and many ResearchGate members may not be aware of the option to select their own publications to feature. Some ResearchGate members may have featured their self-archived papers that were not published in journals or conferences. The survey results show that featuring lowly cited papers published in less-reputable journals implies the member's insistence that such featured publications are important, although their peers have failed to recognise them.

The ability of our investigation method to find articles that were valued differently by the authors versus the scientific community can be applied to find potentially important published studies that have not been recognised by the research community, such as sleeping beauties or delayed recognition (Glanzel and Garfield, 2004; van Raan, 2004) and resisted discoveries (Barber, 1961; Fang, 2015). Such studies are usually innovative and important to scientific progress (Barber, 1961) because they are ahead of their time (van Raan, 2004). The authors of the papers of two such cases valued these papers, despite the lack of recognition or acceptance from others. If a researcher values their low-citation paper, s/he deems that there are some important findings or methods in it and it might be recognised by others someday in the future, and then it will be a future sleeping beauty. The method in this investigation may provide opportunities for finding and identifying such undervalued work. However, this does not mean that all unrecognised papers that are valued by their authors are important, because their authors may insist on wrong ideas. Detailed inspection is needed to determine whether such authors are correct. Seriously and carefully studying others' lowly cited featured publications that were published in low-reputation sources may possibly facilitate promising breakthrough research directions. This method can benefit individual researchers and the whole scientific community. We believe that our method will have higher efficiency if the Featured Research function on ResearchGate is utilised fully. With time elapsing, now young members will provide a wealth of featured publications for this article's approach to identify more unrecognised important papers.

Acknowledgements

This study was supported by Humanities and Social Sciences Foundation of the Ministry of Education of China (16YJE870002); Philosophy and Social Sciences Foundation of Jiangsu High School (2016SJB870005).

About the authors

Xuan Zhen Liu is an associate senior librarian at Library of Nanjing Medical University, No. 140, Hanzhong Road, Nanjing, China. Her research interests include bibliometrics, research evaluation, scientometrics. She can be contacted at: lxz@njmu.edu.cn.
Hui Fang is an associate professor at School of Electronic Science and Engineering, Nanjing University, No. 163, Xianlin Avenue, Qixia District, Nanjing, China. His research interests include research evaluation, research policy, scientometrics, semantic analysis, Data Mining. He can be contacted at: fanghui@nju.edu.cn.

How to cite this paper

Appendix