McClure, C.R., Lankes, R.D., Gross, M. and Choltco-Devlin, B. Statistics, measures and quality standards for assessing digital reference library services: guidelines and procedures. Syracuse, NY: Syracuse University, Information Institute of Syracuse, [n.d.] v, 104 pp. [No ISBN] $25.00

Saxton, M.L. and Richardson, J.V. Understanding reference transactions: transforming an art into a science. Amsterdam, London, etc.: Academic Press, 2002. xviii, 208, [2] pp. ISBN 0-12-587780-3 £56.95

It's a little ironic that a book devoted to quality standards in digital reference libraries should pay so little attention to the standards of publishing. True the publication by McClure and colleagues is the report of a project, rather than a monograph, but the failure to give any date of publication or ISBN, or event statement of copyright seems perverse. The work has been advertised for sale and booksellers do like an ISBN to identify the work in their order processing systems.

However, to the key matters regarding these two publications. They are on related subjects, although one is set in the digital world, the other in the day-to-day world of reference work. Nevertheless, one might expect that they would have something to say to each other and, collectively, to the librarian who is trying to determine how effective his or her reference services, digital or human, might be.

Quite why one would want to 'transform' an art into a science, I am unsure, and whether one benefits by so doing is probably an open question. Long, long ago S.R. Ranganathan laid down a 'scientific' approach to the reference interview, using the facets of his classification scheme (personality, energy, space and time) to structure how the enquiry should be managed, and since then there have been all kinds of efforts devoted to studying the reference interview and reference work generally. I recall, for example, one of my first Ph.D. students, Marian Barnes, using non-participant observation of reference enquiries in public libraries (Barnes, 1981), and causing consternation in some quarters when the results showed how little time was devoted to the enquirer - one reference librarian wanted to sue!. A little earlier, another Master's student did soem similar work in Sheffield City Reference Libraries (Turner, 1975) and caused disbelief in the external examiner for the same reason (so much so, in fact, that the examiner wanted to reduce his grade because of the data he had discovered!)

One is also reminded of the numerous studies, using sureptitious questioning, with standard lists of questions, to explore how accurate were the responses received from different libraries. Apparently, we can never have enough evaluation.

Saxton and Richardson refer to some of this latter kind of study in their book but, sadly and as usual with texts from the USA, rarely cite non-US sources. The book, therefore, has a rather parochial air about it. The book is largely based on Saxton's doctoral thesis and it certainly has the feel of a very 'academic' work: how many practitioners will be moved to plod their way through the dense thickets of technical prose and the even denser statistical analyses is open to question.

The basic premise of the book is that it will be possible, through multivariate analysis of a number of variables to determine which are the 'best predictors of reference performance' (p. 12). The dependant variables, signifying 'performance' are utility (with two measures, completeness and usefulness), satisfaction with the outcome, and accuracy. The independent variables (sixteen of them in all) relate to the nature of the question and the experience of the user, the librarian, the collection, and service policy. The sample consisted of transactions recorded in thirteen public libraries in Southern California over a three-week period in 1998 - one wonders how the rise of digital reference work might affect things? Users (1,704 of them) were invited to complete a survey form and 1,148 of them did so. It is not entirely clear whether all users who visited the library in person were asked to participate, or whether some sampling process was adopted.

Four hierarchical linear models were produced from the data, each model being related to one or other of the four dependent variables. The authors report that the two 'utility' outcome measures (completeness and usefulness) have 'a greater ability to discriminate between levels of good and poor reference service that the satisfaction or accuracy measures...' (p. 95)

An interesting point is that, 'The variables describing experience and education of the librarian failed to predict any of the outcome variables. This finding contradicts conventional theory' (p. 98). This closely corresponds to Turner's 1975 discovery of virtually the same phenomenon - a fact that suggests that it is important to get student research into print! The key factor so far as the librarians were concerned appears to be whether or not they used the RUSA (Reference and User Services Association) guidelines on reference practice.

The final chapter of the book sets out a set of data flow diagrams showing how the practising reference librarian can benefit from the research and there are extensive appendices covering the research data.

This is a dense text, but one worth persisting with, if those 'conventional theories' are to be replaced by understanding of the reality. Whether this is the transformation of an art into a science, I am not sure - perhaps it has more the character of standardising a craft.

McClure and his colleagues define the purpose of their manual as:

...a first effort to begin to identify, describe, and develop procedures for assessing various aspects of digital reference service.

and they attempt to meet this purpose by identifying a set of measures, statistics and standards that libraries can use to assess the effectiveness of their digital reference services.

This is a valuable exercise at this early stage in the development of such services, but it is a little ironic, given Saxton and Richardson's conclusion, that the authors should choose user satisfaction as their main evaluation measure. Of course, it may be easier to collect the data on user satisfaction, rather than to conduct the occasional survey using the other measures proposed by Saxton and Richardson.

Naturally, the majority of measures proposed can be collected automatically, and they include, for example, log analysis and measures such as the number of questions received, and completion time. An Appendix provides sample forms, report formats, logs, worksheets and survey instruments that could be adopted virtually unchanged by any library operating digital reference services.

The manual is a first attempt to delineate the characteristics of digital reference work and I would anticipate that it will not be long before further research, in the mode of Saxton and Richardson, is able to refine the process of evaluation so that libraries need collect only a very small set of data to determine the success of their efforts.

To sum up: both of these books, in their different ways, throw light upon the reference process; both should appear on reading lists for students.

References

Barnes, Marian (1981) Relationships between public library staff and users: implications for service effectiveness, in service training and public relations. 2 vols. Ph.D. dissertation, University of Sheffield.

Turner, R.J. (1975) Enquiry times in a public library: an explanatory analysis. MA in Librarianship Dissertation. University of Sheffield

Professor T.D. Wilson
Editor-in-Chief

How to cite this review

Wilson, T.D. (2003) Review of McClure, C.R., Lankes, R.D., Gross, M. and Choltco-Devlin, B. Statistics, measures and quality standards for assessing digital reference library services: guidelines and procedures. Syracuse, NY: Syracuse University, Information Institute of Syracuse, [n.d.] and of Saxton, M.L. and Richardson, J.V. Understanding reference transactions: transforming an art into a science. Amsterdam, London, etc.: Academic Press, 2002.    Information Research, 8(3), review no. R098    [Available at: http://informationr.net/ir/reviews/revs098.html]