Vol. 12 No. 2, January 2007
Vol. 12 No. 2, January 2007 | ||||
Electronic distribution of journals became practical with the development of the World Wide Web (Web). In 1994 there were just a handful of electronic scholarly journals. Today, the majority of scholarly journals are available through the Internet and electronic dissemination has quickly become the dominant means by which these journals are distributed (Van Orsdel & Born 2002).
While electronic publishing has streamlined the whole publication process, the most salient difference is in the cost of distribution. Given the incremental cost of distributing each copy of a paper journal, the only practicable means of funding these journals has been through subscription fees. With electronic publication, funding a journal by other means and disseminating the content of these journals at no charge has become feasible. Almost as soon as journals began appearing in digital form there were calls for their content to be freely accessible. (Harnad 1990). By 2002 the movement organized itself into what is commonly called the open access initiative (Budapest Open Access Initiative 2002).
The efficiencies in the publishing process brought on by electronic publication have made it possible for small groups of colleagues or even a single individual to develop and maintain high quality, peer-reviewed, scholarly journals. Over the last decade, thousands of electronic journals have been created. While many have languished or disappeared all together, others have prospered and are slowly becoming an established and accepted means of scholarly communication.
This paper discusses the development of Medical Education Online (MEO), a peer-reviewed, open access, electronic journal in health professional education, which I started in April 1996.
The impetus for the journal came in part from my frustration with the existing journals in medical education. I felt a more author-friendly option for publishing was needed. I was also intrigued with the idea of electronic journals and the concept of making scholarship freely available. In 1995 starting a new online journal just seemed like an interesting experiment.
I had first encountered electronic journals in the mid-1980s operating as listservs on Bitnet. Although an fascinating concept, given the technology at the time, they were clearly not a viable alternative to traditional paper journals. A decade later with access to the Internet growing quickly and the introduction of HTML and the HTTP protocol, it seemed that digital technology had reached the point where electronic journals made sense and could provide a viable alternative to traditional paper journals (Solomon 1996).
During the summer of 1995, I ran across an electronic journal in educational policy, Educational Policy Analysis Archives (EPAA), started by Gene Glass a well known educational researcher and statistician. This rekindled my interest in electronic journals. I had mastered the basics of HTML coding and the process of creating and operating an electronic journal seemed feasible. I proposed the idea to my colleagues in the Office of Educational Development (OED) at the University of Texas Medical Branch and several volunteered to help. We presented the idea of starting an electronic journal in medical education at the annual meeting of a small professional conference, the Generalists in Medical Education in November 1995. A number of people at the conference were also interested in participating. Over the next several months, we kept the conversation on creating a journal going by e-mail. With advice and encouragement from several of these people and help from my colleagues in OED I started planning for the journal. We had additional help from two of the University's librarians and an attorney in the Office of the General Counsel who helped develop the copyright, complaint policy and author agreement for the journal.1 The director of OED, Gwendie Camp, PhD was extremely supportive and gave me some protected time to work on the journal. Don Powell, MD, the chairman of the Internal Medicine Department in which I had a joint appointment allowed me to use their newly purchased Web server to host the journal during its initial phase.
I asked several colleagues around the country to write articles for an initial issue to get the journal off the ground. While about eight agreed, only three actually completed their articles. Along with an editorial of my own, we launched the journal in April 1996 with these three articles.
Based on the discussions we had over the preceding months, MEO was conceptualized as a forum for disseminating information on educating physicians and other health professionals. We envisaged that the site would include a variety of resources, with its centerpiece being a peer-reviewed electronic journal. The site also included a repository for educational resources, a posting section for notices of interest to educators in the health professions and a bulletin board for interactive discussions.
The submissions to the posting section were largely hyperlinks to Web sites of potential interest for health professions educators. Although such lists are useful, many existed and some were far more comprehensive than the MEO posting section. The original intent of the section was to post informational messages such as calls for conference papers, job postings and requests for grant proposals. While several of these were submitted each year, this function is largely duplicated by an excellent listserv for medical educators, DR-ED. In retrospect, I decided a listserv is a better vehicle for this type of communication and dropped the section a few years after MEO was launched. The bulletin board never caught on and failed to generate extended discussions. Again, DR-ED seemed to be more effective in fulfilling this role and the bulletin board was dropped.
The resource section remains part of the journal though it has received relatively few submissions over the years. The need for a repository such as MEO's resource section has lessened as self archiving has become more feasible. There are also at least two open access repositories for medical education resources, MedEd Portal and HEAL. Both are peer-reviewed, indexed and well respected. Although there are better options for disseminating resources, I have left the resource section in place because it is still accessed frequently and to honour what I feel is a commitment to the authors who contributed the material for the section. Beyond that, we still receive occasional submissions.
A book review section was added in 1998 initially based on a request from an author asking to have a textbook he wrote reviewed. Over the years since that first review, several other authors and publishers have asked to have books and, more recently, instructional software reviewed. Initially, the requests came from authors and small publishers. More recently, the journal started receiving requests from larger, mainstream publishing houses. While we have been able to find qualified people to review books that are sent to the journal, we have found it much more difficult to get them to complete the reviews. I find this embarrassing and for the last few years, I have been reluctant to accept requests to review books.
We added a letter to the editor section in 2003. Three medical students wrote a very thoughtful paper that did not stand on its own as an article but that I felt was worth publishing. We agreed to publish it as the initial paper in a letter to the editor section. Since then, we have encouraged and received papers submitted as letters. In addition, when we receive papers that have some merit but that we feel are not publishable as peer-reviewed articles, we have offered the authors the option of publishing them as letter and in most cases, they have accepted. The section is not peer-reviewed but is screened by one of our editors for merit before being accepted for publication.
Establishing an electronic journal is a difficult task and I suspect this is why so many of these journals languish and never really get off the ground. First, few authors want to risk publishing articles in a new electronic journal that does not have a record of accomplishment. Secondly, many people in academia still do not take electronic journals seriously and do not give publishing in these journals the same 'academic credit' as publishing in a traditional paper-based journal. Although the situation has improved over time, in 1996 many people in academia did not even understand the concept of an electronic journal. After starting MEO, I tried where possible to publicize the journal through poster sessions and displays at professional meetings. People often told me that, while the concept was interesting, they did not like reading manuscripts off their computer screen. I would point out that electronic referred to how the journal was distributed, not necessarily read and they probably had a printer connected to their computer. It was amusing to watch the light bulb go on.
There is no easy solution to getting a new journal established. From my experience, it takes patience, perseverance, and constant effort. Announcements on relevant list servers, publicizing the journal at professional conferences and submitting the site to as many search engines as possible all help. Most electronic journals, including MEO, start with a series of invited articles. The more articles you can get and the better known the authors, the better off you will be.
In some respects, starting MEO in 1996 might have been easier than today. Many of the early manuscripts came from people who saw the communication potential of the Internet and were willing to risk publishing their work in the journal because, like me, they were intrigued with the idea of an electronic publication. While the novelty has faded, electronic journals are also slowly becoming an accepted means of publication within academia.
Over the last decade, the peer-reviewed journal component of MEO has become well established and widely accessed. Submissions started slowly and we published between five and ten articles a year until about 2000 when access and interest as well as submissions began to increase. Figure 1 shows the number of manuscripts published between 1996 and 2005.
While I had quite a bit of help getting the journal off the ground, once established I ran MEO myself until 2005. During the late 1990s when the journal was receiving fifteen or twenty submissions and publishing five to ten articles a year, it was not much of a burden and I enjoyed both the editorial tasks as well as mechanics of publishing the journal. In 1999 I transferred to Michigan State University where I had been previously employed and took MEO with me. As submissions grew steadily after 2000 the workload of maintaining the journal began to become a real burden. In 2004 the journal received approximately eighty submissions and published twenty-four articles. Unique IP addresses accessing the Journal had grown to over 12,000 per month coming from over 100 countries and I was getting overwhelmed by the tasks of maintaining the journal by myself. I also decided it was time for someone else to take over the journal. I put out a call on the DR-ED listserv for groups, who might be interested in taking over the journal.
A number of groups responded including several offices of medical education and one scientific society. Two of my former colleagues at the University of Texas Medical Branch, Steve Lieberman, MD and Ann Frye, PhD offered to take over the journal and I decided to turn the journal over to them, given all the support I had received there for starting the journal and the fact I knew they would do a good job. Over the spring of 2005 we worked together to transfer the journal back to Texas and by early summer Drs. Lieberman and Frye were running the journal. Later in the summer, a number of events coalesced to disrupt their ability to maintain the journal. Hurricane Katrina shut down the two New Orleans based medical schools and the University of Texas Medical Branch, as one of the closest medical schools temporarily enrolled several dozen of the students from these schools. Dr. Lieberman as the associate dean for education directed this effort leaving little time for the journal. Approximately a month later, Galveston Island was evacuated by the approach of hurricane Rita and the University of Texas Medical Branch was shut down for a week. I took over maintaining MEO during this period but the disruption of both storms helped to create a large backlog of submissions. Later that fall both Drs. Lieberman and Frye were promoted to more demanding positions and Dr. Lieberman could no longer maintain his role as co-editor of the journal.
I took over from Dr. Lieberman co-editing the journal with Dr. Frye in the fall of 1995 and we began working our way through several months of backlogged submissions. It was clear we needed additional help as the number of submissions continued to increase. In early 2006 we reorganized the structure of the journal to include an expanded editorial board that would act as a panel of review editors each taking on the responsibility of managing the review and revision of a subset of the manuscript submissions. Dr. Frye and I continued as managing co-editors. One of us does the initial review for each submission and handles the feedback to authors when a submission is not within the scope of the journal or clearly not publishable. Other submissions are assigned to one of the review editors who manage the peer-review and revision process for the manuscripts they are assigned.
This new structure for managing the workload of editing the journal is working exceedingly well. We currently have six review editors.2 Dr. Frye and I act as managing editors and a librarian, Julie Trumble, MLS helps with copyediting, reference checking and journal style issues. The workload of managing the processing and peer-review of submissions is now quite manageably distributed among six review and two managing editors. If it becomes necessary in the future, we can add additional review editors. We are still struggling with how best to handle the workload for copy editing, indexing and formatting manuscripts for publication and are experimenting with ways of of streamlining the copy editing and formatting process.
There was considerable debate among the people who helped me establish MEO whether or not it should be peer-reviewed. I finally decided to make the journal section peer-reviewed, mainly for the benefit of authors who could then could count publications in MEO as peer-reviewed. I believed it was also necessary to maintain the credibility of the journal, particularly at a time when these journals had so little credibility to begin with. Over time I have come convinced that it was a very wise decision for the original reasons but also because the constructive feedback from reviewers significantly improves the quality of the articles and is generally appreciated by the authors.
MEO solicits reviewers on its Web site. Individuals who are interested in reviewing simply fill in a Web-based form with their interests and expertise. Currently MEO has around 250 active reviewers from all over the world and with very diverse backgrounds. I have never worried about a reviewer's credentials. If they express and interest in reviewing they are given a chance as long as the information they provide concerning their interest and expertise seems credible. We generally ask six to eight reviewers to review each manuscript and usually four to six agree. We always try to include a few experienced reviewers who have previously done a good job. This helps ensure good quality feedback on each manuscript. More often than not we are pleasantly surprised with the quality of the reviews from new reviewers.
The reviewers are given a month to complete the review and at the end of the review period, the publication decision and feedback to the author(s) is based on the reviews that have been received. We generally do not postpone sending out the feedback more than a few days to accommodate late reviews. By asking six to eight people to review manuscripts I have found you generally end up with an adequate number of reviews within a four or five week period we give reviewers to complete their reviews. We make a real effort to ensure authors receive feedback and a publication decision within six to eight weeks of submitting a manuscript.
I am continually surprised and pleased with the amount of thought and effort that most reviewers put into the review process. Originally we took the ratings and comments from each reviewer and append them together into a single feedback document for the author. I would write a cover e-mail with the publication decision and summarizing what I felt to be the most pertinent issues identified by the reviewers. This was sent to the author with the reviewer feedback document appended as an e-mail attachment. I also sent the e-mail as a blind carbon copy to each of the reviewers. The feedback we have received suggests reviewers want to know how the other reviewers evaluated the manuscript and clearly appreciate receiving this feedback. The mechanics of the review process are now handed by a comprehensive Web-based journal management system described below.
As noted above, I originally I operated the review process manually using e-mail to handle correspondence with the reviewers and authors. In 2001, I partially automated the process through a series of scripts I wrote in PHP, a server-side scripting language. The scripts allowed authors to submit their manuscripts through a Web-based form that stored the author contact and other information in a MySQL relational database and uploaded the manuscript to our server notifying me by e-mail when a new submission arrived. Before this, reviewers volunteered and supplied information about their expertise by e-mail. I added a script that allowed people interested in reviewing to enter their contact and expertise information into a MySQL database table through a Web-based form. I also wrote a script that allowed me to scroll through the reviewer database viewing the reviewers' qualifications and review history selecting potential reviewers for a manuscript. The script then sent out an e-mail request to review to each selected reviewer and recorded the request to review in each selected reviewer's database record. Reviewers still responded their willingness to review by e-mail and I provided them with a copy of the manuscript and review form and conducted the rest of the review process by e-mail.
Even this partially automated system saved considerable time and effort over a completely manual system. I considered developing a more automated system but resisted the urge to fully automate the process because of the time involved and because I enjoyed corresponding with the reviewers, some of whom I got to know quite well through our e-mailing back and forth.
After Dr. Frye and I decided to reorganize the editorial process to include multiple review editors, it became clear we needed a much more comprehensive and automated review management software system to handle the workload and manage the additional complexity of having multiple review editors. Although I was aware of Open Journal System (OJS) which is an excellent open source journal management system, I decided to revise and expand our existing journal management software. I chose to develop my own software because I was not sure it would be possible to implement our new hierarchal review system with OJS. Also, moving to a new journal management system would likely create significant legacy issues for the existing content of the journal. In late 2005, and over the subsequent New Year holidays I revised the existing software to support the new review structure, significantly automate the mechanics of the review process and archive all review correspondence in a server-side database. We began piloting the new review system in January 2006 with just Dr. Frye and myself acting as both managing and review editors. After some modifications and bug fixes we found the system worked extremely well.
The new software automates much of the mechanics of operating the peer-review system while tracking the progress of each manuscript under review and maintaining all the correspondence concerning each manuscript in an easily accessible database. Some screen shots of the software can be viewed at the journal site.
When a new submitted manuscript is uploaded, the system notifies Dr. Frye and me by e-mail. One of us downloads the manuscript by 'clicking' on a link in the e-mail and does a preliminary review. If the manuscript is not within the scope of the journal or is clearly not publishable, we notify the author and give them feedback. The feedback is entered through a Web-based form in the review system which sends it to the author by e-mail and archives our feedback and changes the status of the manuscript to rejected. If we feel the manuscript should be sent out for external review, we indicate which review editor is assigned and the system sends them an e-mail requesting they manage the review with a link to download the manuscript and identifies them as the review editor for the manuscript in the database.
The review editors prepare the manuscript for review by removing any information identifying the authors and adding in headers on each page indicating it is copyrighted material that is not for distribution. The review version is uploaded to the server via the review management software. The review editor then selects reviewers via a form in the review system. Each selected reviewer is sent an e-mail request to review. The e-mail includes a link they can use to download the review copy of the manuscript and a link to a Web-based review form that can be used to enter their ratings, comments, and their suggested disposition for the manuscript (accept, accept with specified revisions, or reject). They indicate their willingness to review the manuscript by downloading it via the link in the e-mail. The review editor can check the status of the reviews via a review status form that indicates whether each selected reviewer has agreed to do the review by downloading the manuscript and completed the review by filling out the Web-based feedback/rating form. The review editor can view the individual ratings and comments from each reviewer by 'clicking' on a link. They can also view a summary form with the aggregated ratings and comments from all the reviewers.
When all the reviews have been received, the review editor completes a feedback letter to the author(s) (for an example, see Figure XX) through a form in the review system and assigns their final disposition of the manuscript (rejected, accepted with revisions or accepted as is) The system sends the feedback as an e-mail to the lead author with a link that displays the feedback summary of the ratings and comments from the reviewers with the exception of the comment section for the editor only. The feedback summary e-mail is also sent to each of the reviewers who completed their review. For manuscripts that are accepted with revisions (the most common disposition) the review editor works with the author to complete the revisions. This correspondence is done by e-mail outside of the review system. Each manuscript's status is updated in the database as it moves through the review process. As noted, all correspondence generated through the review process is archived and available for viewing at a later date if necessary.
When MEO was launched, I became concerned about the legal issues surrounding ownership and access of the material. I contacted Georgia Harper, J.D. who was the lead intellectual property attorney for the University of Texas. Ms. Harper was extremely helpful and developed the Journal's legal documents and policies including the publication agreement for authors. We decided to use a limited license to distribute as the legal framework for disseminating material rather than obtaining copyright from the authors. This approach gives the journal sufficient rights to accomplish its purpose while leaving authors free to do what ever else they choose with their own manuscripts.
Ms. Harper suggested and helped me develop a complaint policy. In the ten-year history of MEO, there have only been a few complaints and disputes. One of these was serious involving a case of plagiarism. In that case, which turned out to be relatively easily to resolve, having a clearly defined complaint policy was a real asset and something I would strongly suggest for any journal.
In the last year we have added policies covering the use of human subjects in research and the disclosure of conflicts of interest. The decision to formally state these policies and require authors to declare their adherence to these policies in the submission process reflects a growing world-wide concern with these issues at least in the field of medicine.
The specific policies of the journal can be accessed at the Website and anyone who wishes to use them for their journal in part or whole is welcome to do so. No specific permission or acknowledgement is necessary.
From its inception until 2005, articles in MEO have been published in both PDF and HTML format. In 1996, PDF was a relatively new format but offered the ability to provide higher quality documents for printing. For the first several years, the vast majority of users accessed the HTML version and the feedback we received suggested many of our readers had difficulty downloading and installing Acrobat Reader necessary to access the PDF version. Over time our server logs have indicated access has shifted more towards the PDF format to the point that by 2004 the vast majority of readers were downloading the PDF version of at least our newer articles. In 2005 we began publishing only the PDF versions of our manuscripts given the significant extra effort it took to create two versions and the fact virtually everyone was downloading the PDF version. Interestingly, the HTML versions of our older manuscripts are still heavily accessed. I suspect this is due to existing external links to the manuscripts that exist all over the Web. The HTML version of one of the first set of three manuscripts MEO published in April 1996 still is being accessed around 500 times per month.
Originally I handled copy editing and formatting the manuscripts for publication along with the other tasks of publishing the journal. In late 2005, Julie Trumble, MLS took over the copy editing task and has done a far better job than I could manage. She however has become overloaded with the workload of this very time consuming and tedious task.
We are currently exploring other options and probably will go back to only offering limited copy editing for manuscripts leaving most of the onus on authors to ensure their manuscripts are well written and largely free of typographical, spelling and grammatical errors. Our current plans are to have the review editors assess this issue and if necessary, require the authors to pay for professional copyediting. There are quite a few freelance copy-editing services available through the Web. In those cases where authors are from developing countries or for some other valid reason cannot afford to pay for copy editing, we will find ways of assisting them.
I continue to format accepted articles for publication. Originally, I did this directly in Microsoft Word, the format we receive virtually all our submissions and then convert the finished document to PDF format. More recently, I have been using Indesign®, Adobe®'s desktop publishing software. This has greatly streamlined the process of typesetting as well as substantially improved the finished product. Although somewhat time consuming, the process is not too onerous and I generally do it in the evening when I do not feel like working on something that takes a great deal of concentration. We are currently exploring providing much more prescriptive instructions on the format of manuscripts and a template to reduce the effort necessary to format manuscripts.
Over the years access to the journal has increased steadily and at this point continues to rise. During 2006 we received 100 submissions. Approximately 50% of the submissions are rejected before being sent out for review. The majority of articles that are sent out for review are accepted with revisions. The journal published 34 articles.
We are receiving between 800 - 900 visits a day where a visit is defined as a unique IP address accessing the journal being counted only once over a two-hour period. During non-holiday periods, we are having 18,000 unique IP addresses accessing the journal each month from well over 100 countries. We have many articles that have been accessed more than 10,000 times and one that has been accessed more than 30,000 times. These access statistics continue to grow steadily.
Over the last 10 years, Medical Education Online has become an established journal and a well accepted venue for disseminating research and innovations in the field of health professional education. In my experience, forming an electronic journal is not beyond the reach of a small group of individuals such as a special interest group of a professional organization, a small university department or even a single faculty member though I would discourage anyone from attempting to maintain a journal by themselves.
Developing the site has been extremely rewarding giving me the opportunity to communicate with hundreds of people from all over the world. I regularly receive heartfelt thanks for maintaining MEO from authors and readers particularly from developing countries where accessing the scholarly literature is especially difficult. MEO along with hundreds of other successful open access electronic journals have demonstrated this is a viable means of archiving and disseminating scholarly information and one that is likely to play a significant role in the future communication of research and scholarship.
| Find other papers on this subject | ||
© the author, 2007. Last updated: 17 December, 2006 |
|