Current findings from research on structured abstracts

By James Hartley, Ph.D., Research Professor
Department of Psychology, Keele University
Staffordshire, United Kingdom

Background: Structured abstracts were introduced into medical research journals in the mid 1980s. Since then they have been widely used in this and other contexts.
Aim: The aim of this paper is to summarise the main findings from research on structured abstracts, and to discuss the limitations of some aspects of this research.
Method: Narrative literature review of all of the relevant papers known to the author.
Results: Structured abstracts are typically longer than traditional ones but they are also judged to be more informative and accessible. Authors and readers also judge them to be more useful than traditional abstracts. However, not all studies use 'real-life' published examples from different authors in their work, and there are areas where more work needs to be done.
Conclusions: The findings generally support the notion that structured abstracts can be profitably introduced into research journals. Some arguments for this, however, have more research support than others.

INTRODUCTION

Readers of this article will have noted that the abstract is set in a structured format. Such abstracts typically contain sub-headings and sub-sections - such as background, aim(s), method(s), results and conclusions - and provide rather more detail than do traditional ones. Furthermore, these features are clarified by the typographic layout. Structured abstracts are more common in articles describing experimental research but, as the example above indicates, they can also be used with reviews.

Structured abstracts were introduced into medical journals in the mid 1980s [ 1], and since then, their growth has been phenomenal [2]. Indeed, they are now commonplace in all serious medical research journals. Furthermore, their use has been recommended, and indeed is growing, in other scientific areas [see 3-8]. Structured abstracts can be found in American, European, Australian, Japanese and Chinese journals [9]. In addition, some academic societies now require contributors to send potential conference submissions in this format. The British Psychological Society, for instance, has dispensed with the need for the three-four page summaries previously required from potential participants and now publishes the accepted structured abstracts in their Conference Proceedings [e.g., see 10].

KEY FINDINGS

The case for using structured abstracts in scientific journals has been bolstered by research, most of which has taken place in a medical or a psychological context. Table 1 lists all of the research studies on the topic known to the author at the time of writing.

The results from these studies suggest that, compared with traditional ones, structured abstracts:

• contain more information (studies 1-11), but not always so (study 12);
• are easier to read (studies 2, 3, 13);
• are easier to search (studies 3, 14) - although some authors have queried this (studies 15-17);
• are possibly easier to recall (study 18);
• facilitate peer-review for conference proceedings (studies 5, 6, 19); and
• are generally welcomed by readers and by authors (studies 1-3, 11, 19-20).

However, there have been some qualifications. Structured abstracts:

• usually take up more space (studies 1-4, 7, 11, 13, 21-24);
• sometimes have confusing typographic layouts (studies 25-26); and
• may be prone to the same sorts of omissions and distortions that occur in traditional abstracts
(studies 8-10, 27-31).

SOME OBSERVATIONS

Although these findings seem clear-cut there are some problems with the research in general that need to be taken into account. There are two main issues here. Firstly, but possibly not too important, is that many studies have used undergraduate students as participants. Such undergraduates, of course, do not have the experience of full-time academics and post-graduates in reading journal abstracts, and thus the requirements of the studies may be rather different for them. Secondly, and much more important, is that not all of the studies have compared actual, previously published, traditional and structured abstracts, thus reducing the validity of the comparisons.

For a study to be properly valid in this context one needs, in effect, to compare, from a particular journal, sets of published traditional abstracts with sets of (different/later) published structured abstracts. What has often happened in practice, however, is that either (i) the structured versions of published traditional abstracts have been written independently by the researchers for the studies in question (e.g., studies 1, 2, 11), or (ii) the published traditional abstracts have been shortened and simplified for experimental purposes, and then structured versions of these simplified versions have been written by the researchers (e.g., studies 13,14, 18). The main reason for this state of affairs stems from the fact that the researchers in question are often advocating the use of structured abstracts in a particular discipline where none (or very few) are available at the time of writing, and thus they have had to create their own. Such procedures do not destroy the validity of the findings, but they do limit their generality. To be properly valid the abstracts need to be written by the authors of the articles and not by the researchers.

If we apply these considerations to the findings listed above, then:

• It is not surprising that most studies find that structured abstracts contain more information than traditional ones. If researchers/authors create structured abstracts from traditional ones, then it is likely that they will increase the amount of information that they provide when they are doing this. The best evidence to support the argument that structured abstracts contain more information than traditional ones comes from studies comparing published abstracts in both formats from the same journal(s) written by independent authors. Of the twelve studies that meet these requirements (studies 4-5, 8-10, 12, 21) all support the view (with the exception of study 12) that structured abstracts are more informative than are traditional ones.


• It is also perhaps not so surprising that structured abstracts appear easier to read than traditional ones if the structured abstracts are revisions of the traditional ones (e.g., studies 1, 2, 11). Again the best test of this readability hypothesis would come from studies comparing separately published abstracts in both formats. But no such studies have been reported on this issue.


• Similarly, if the structured abstracts are revisions of simplified traditional ones, then it is perhaps not surprising that they are easier to search (study 14). Studies with the more complicated 'real-life' abstracts presented in MEDLINE have not shown an advantage for search speed (studies15, 17).


• There have been virtually no studies on the recall of traditional and structured abstracts to find out whether or not structured abstracts are remembered any better than traditional ones. Six 'miniature' studies outlined by Hartley and Sydes (reported in study 18) suggested that this was the case when the structured abstracts were longer and were more readable than the traditional ones. However, this was not the case when the abstracts were equated for length and readability. Furthermore, these six studies were done with simplified versions of published abstracts, using students as participants. Research in other contexts does suggest that structured text might be better recalled than text in traditional formats [11], although some people have argued that because less well-structured text requires more processing, it might subsequently be remembered better.


• Suggestions about the benefits of using structured abstracts for selecting conference papers are rarely evidence-based. They simply suggest their usefulness in this respect. One exception is the study by McIntosh, Duc and Sedin (study 6). These authors reported that referees were less frustrated reviewing the information content and deciding on the suitability of a conference submission when they judged re-submitted structured abstracts compared with traditional ones.


• Most studies of readers' reactions to structured versus traditional abstracts have in fact relied on the judgements of academics rather than students (e.g., study 2, 3 and 6) but few have been asked to judge independently published versions of both sets.


• If structured abstracts do present more information, it is not surprising that they are usually longer than traditional ones. These two measures are correlated. The weighted average increase in length for the structured abstracts in the eleven studies listed in Table 1 is 21%. However, eight of these studies used simplified and/or especially written abstracts for the purpose of illustration. In the three studies that compared independently published abstracts, and provided data on this measure, the structured abstracts were 40%, 30% and 29% (weighted average 35%) longer than the traditional ones, respectively (studies 3, 21, 23).


• There is, as yet, no hard information on whether or not structured abstracts contain less - or more - omissions, distortions and errors than do traditional ones. Nonetheless. several authors have reported omissions in the published structured abstracts in the medical journals that they studied (studies 8-10, 27-28). Hayes et al (study 19) argued that there might be fewer errors in structured abstracts than traditional ones, but Pitkin and his colleagues (studies 29-30) argued the reverse of this because of their extra length. But none of these authors have made direct comparisons. Hartley (study 31) found no differences in the degrees of distortion or errors between traditional and structured abstracts, but his structured abstracts were re-written versions of traditional ones so. To date, no one to my knowledge has compared the accuracy of separately published traditional and structured abstracts.

CONCLUSIONS

The literature reviewed in the first part of this paper suggests that structured abstracts are an improvement over traditional ones. However, as argued in the second part, the evidence used to support these claims is sometimes not as good as one might wish. In particular, we have to judge the applicability of the findings from this research to the 'real world'. Nonetheless, the explosion in the use of structured abstracts, particularly in the medical context, suggests that these judgements have already been made.

Some editors have complained that structured abstracts 'take up too much space'. Indeed, the data reviewed in this paper from the more valid studies suggest that the extra space required by introducing structured abstracts may be quite considerable. But we have to remember here that we are only talking about the extra line-space required by the abstract, and not the article as a whole. Indeed, setting a word limit (such as 200 words as in the JMLA) could control this amount. Whatever the case, introducing structured abstracts into a journal is unlikely to require changes in the overall pagination. There is often 'free' space at the ends of articles, and typesetters are skilled at fitting text to appropriate page dimensions [12]. Such concerns, of course, do not arise with electronic journals and databases.

Some authors - and some editors too - have also complained that the formats for structured abstracts are too rigid, and that they present them with a straightjacket that is inappropriate for all journal articles. Undoubtedly this may be true in some circumstances but it is remarkable how in fact the sub-headings used in this present article can cover a variety of research styles. Bayley, Wallace and Bryce [13], for example, provide suggested sub-headings for quantitative studies, qualitative studies, case-reports, and reviews, all of which follow to some degree or other the basic format used here. Furthermore, if readers care to examine current practice [or Table 4 in 14], they will find that even though the sub-headings used in this article are typical, they are not always rigidly adhered to. Editors normally allow authors leeway in the sub-headings that they use.

Finally we might note that the research on structured abstracts is limited in two other ways. Firstly, it is not made clear in many studies whether or not the titles of the journal articles have been presented along with the abstracts, and clearly a title might help the reader to recall and possibly interpret the text under study [15]. Secondly, in addition, no one to my knowledge has studied complete articles with either traditional or structured abstracts. Here, where the abstract plays a small, but nonetheless important role, it would be of interest to see if the format of the abstract affects the readers' judgements of the quality of an article (or even of a journal) as a whole.

Despite these limitations, and the ones discussed earlier, the conclusions of this review are that the data discussed above do support the claim that structured abstracts are an improvement over traditional ones. Not only is more information presented - which is helpful for the reader - but also the format requires that the authors organise and present their findings in a systematic way. Furthermore, this consistency is made more obvious by the typographical layout. Advances in 'text mining', 'research profiling', and computer-based document retrieval systems, can only profit from the use of these more informative abstracts.

ACKNOWLEDGEMENTS

I am grateful to the editor and three independent referees for helpful comments on an earlier version of this article which was originally published in the Journal of the Medical Library Association, 2004, 92, 3, 368-371.

Author's address for correspondence: James Hartley, B.A., Ph.D., Department of Psychology, Keele University, Staffordshire, ST5 5BG, United Kingdom; email, j.hartley@psy.keele.ac.uk

REFERENCES

• 1. Ad Hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical articles. Ann Intern Med 1987 April 106: 598-604.
• 2. Harbourt, AM, Knecht LS, Humphries BL. Structured abstracts in MEDLINE 1989-91. Bull Med Libr Assoc 1995 83(2):190-195.
• 3. Hartley J. Is it appropriate to use structured abstracts in social science journals? Learned Publishing 1997 10(4):313-317.
• 4. Hartley J. is it appropriate to use structured abstracts in non-medical science journals? J Info Sc 1998 24(5):359-364.
• 5. Hartley J. Applying ergonomics to Applied Ergonomics. Appl Ergonom 1999 30(6):535-541.
• 6. Hartley J. Clarifying the abstracts of systematic reviews Bull Med Libr Assoc 2000 88(4):332-337.
• 7. Hartley J. Improving the clarity of journal abstracts in psychology. Sc Comm 2003 24(3):366-379.
• 8. Kostoff N, Hartley J. Open letter to technical journal editors. J Info Sc 2002 28(3):257-261.
• 9. Liu XL, Qiano HC, Bo-Rong P, Jun D, Su-Juan W. Structured abstracts in Chinese biomedical journals: the current situation and perspective. Euro Sc Editing 1993 No 48 (January):p20.
• 10. British Psychological Society. Centenary Annual Conference: Final Programme and Abstracts. Leicester: British Psychological Society, 2001.
• 11. Hartley J. Recalling structured text. Does what goes in determine what comes out? Brit J Ed Technol 1993 24(2):84-91.
• 12. Hartley J. Do structured abstracts take up more space? And does it matter? J Info Sc 2002 28(5):417-422.
• 13. Bayley L, Wallace A, Brice A. Evidence based librarianship implementation committee research results dissemination task force recommendations. Hypothesis (Newsletter of the research section of MLA) 2002 16(1, Spring): 6-8.
• 14. Hartley, Clarifying the abstracts of systematic reviews, op. cit
• 15. Anon. For the record - titles and introductions. Teachers College Record, 2000 102(3):447-479. http://www.tcrecord.org ID number: 10710, date accessed: 6/18/2003.