Difficulties in Evaluating Scholarly Communication
by Cynthia Rieck
I give permission for my final project to be made available through the
LIS Learning Showcase web server.
Introduction
There are many things which make the evaluation of scholarly communication
difficult. "Scholarly communication is the complex matrix of intellectual
property, publishing practices and economics, legislative action, technological
developments, and academic expectations for tenure, promotion, research, and
publication." (Ogburn, 2002)
In the realm of science, ideas are constantly being built upon. Each new
idea is studied, reviewed, and modified. Change and growth are the hallmarks of
modern science. To understand this phenomenon, scholar's look to study the
trends and trails of citation. Also called the "scholars spoor", these trails
of citation show the importance of the documents cited. Eugene Garfield, the
father of the scientific citation index, saw the index as a method for searching
scientific publications.(Cronin, 2000)
Two of the processes that evolved from the citation index are scientometrics
and bibliometrics. Scientometrics concerns how often and in what situations a
scientist writes a paper citing a scientist in another field. This in turn, has
led to a study of the development and politics of science and the scientific
community. Bibliometrics focuses on the publications themselves. It concerns
the mathematical and statistical analysis of patterns that arise in the
publication and use of scholarly materials.
Scholars, using the idea of citation indexing, can now study the ideological
linkages of a cited work. The evaluation of citations can give scholars
information regarding development, collaboration, politics, and interdiciplinarity.
Some of these trends make the evaluation of the products and processes of scholarly
communication difficult.
Collaboration
Collaboration is a growing trend among scientists. Collaboration occurs
on many different levels, from working together on the same project to the
expression of an opinion to sharing resources but working on different projects.
There are several types of collaboration. One of these is international
collaboration among scientists in the same field. Another is the professor who
allows the addition of his name to a student's project. Collaboration also
occurs between individuals in different fields. (Bordons, 2000) One of the
major influences prompting this trend originates with the scientists themselves.
They are so specialized as to need collaboration in research and for publishing
papers.
Collaboration is also used by scholars to increase the number of papers
they have published. By publishing more papers they boost their position with
regard to tenure track positions at Universities, and other such perks as sitting
in as a member of a review board. Many of the collaborations for these purposes
are not true collaborations but only a noted scholar lending his or her name to
a paper to provide legitimacy to the paper or other equally political reasons.
The universities that the scholars are associated with also influence the politics
of collaboration. By getting a well known scholar from a leading university to
lend his name to a paper produced by scholars from a "lesser" ranked school the
paper attains a legitimization at it would not other wise receive. This also
influences which journal will publish the work.
Collaboration occurs for several reasons. One is the need to share the
cost of the equipment necessary for complex scientific research. The larger
and more expensive the frame work needed, the more people will be involved in
the collaboration. Another reason for the growth of collaboration is the
increased speed of travel for both information and the scientists themselves.
The internet and faster air travel are keys to the birth of the scientific
community as a global community. Thus, collaboration is a trend that will
only expand as the global community itself continues to expand. Access to vast
stores of knowledge and instantaneous communication with colleagues fosters
extreme specialization. Extreme specialization is another reason scientists
collaborate. Significant advances in science arise from the collaborations of
scientists with different specialties.
Interdiciplinarity
"Interdisciplinarity can be defined as the integration of concepts and
epistemologies from different disciplines".(Coleman, 2002) Interdiciplinarity
also concerns the collaboration of scholars from differing fields of study. As
mentioned above scientists are becoming more and more specialized. This
increased specialization is prompting the need for collaboration across
disciplines. An example of this as pointed out by Anita Coleman is the Digital
Library.
The Internet Public Library (http://www.ipl.org/) hosted by the School of
Information at the University of Michigan offers a look that one form of digital
library. This is based on the public library with reading rooms and a children's
area. It also has a special collections which includes literary criticism,
Native American authors and exhibits. Every link at the IPL is a web site
including the reading room which contains sites with full-text publications.
The librarians who run the site must use traditional library skills
and have a working knowledge of internet technology. Each year, students
in the University of Michigan, most from the School of Information, seek
out ways to improve the IPL, whether by maintaining and expanding its subject
collections, enhancing usability, proposing new functionality, or building
special collections. Not all of the work is concerned with improving the
collection, some is also technical, such as the real-time chat reference
service piloted by two students. Another student updated the HTML code in
the special collection area to XHTML standards.
Technology
Today's scholarly communication involves many diverse technologies and
their repercussions. Some of these technologies evolved from the internet.
These include chat, email, instant messaging, teleconferencing, e-publishing
and virtual labs.
Informal methods of communication in the form of instant messaging, chat,
and email are increasing the speed with which scholars communicate. While
these methods increase speed and break down barriers between long distance
scholars they do pose problems. The informality of these methods can lead to
casual and sloppy communication (not everyone checks email for spelling and
grammar errors).
Formal communication exists with teleconferencing, electronic publishing or
e-publishing, and virtual labs. One of the major advantages of e-publishing is
searchable text. Another is availability, anytime day or night. E-publishing
is also more cost effective than print publishing. This is due to having no
need for extensive warehouses and expensive presses. Mailing e-publications
is also faster and cost efficient due to not having to use the postal service.
As soon as a journal is declared ready to print it can be sent electronically,
using a vast database of subscribers, instantaneously.
Not all technology is perfect. As mentioned above, ease of communication
can lead to a degradation in quality. Another technological con is that it
is not uniform world wide. For example, different countries may have different
computer languages which are not compatible. Another problem exists in
developing countries which do not always have access to electricity. A third
problem arises when some scholars have access to emerging 3-D capabilities and
others do not.
Disintermediation
Another of the pitfalls of technology is the ability for individuals to bypass
the "middleman" and go straight to the publisher for books and periodicals. The
publishers seek to protect their hold on the information they publish by offering
bundled packages of electronic journal access. They also form databases which
may be accessed for a fee. In the business world eliminating the cost of the
"middleman" benefits the consumer. In the realm of scholarly communication this
is a great handicap as the "middleman" is often the library.
The solution to this problem is by offering "value-added" services. In
other words, the library must be added back into the equation in such a way as
to provide a better service. One way libraries are attempting this is by offering
ask-a-librarian services. Technology plays a major role in making this possible.
Publishers and not-for-profit organizations can use e-publishing to offer
subscribers full-text searching. E-publishing can also be used for simultaneous
archiving of published materials in a format which allows instant world wide access.
Internet searching is another area that impacts scholarly communication. One
way to provide "value added" service is to improve searching capabilities. IBM has
recently developed a new type of internet searching that is revolutionary. Not
only does the software continuously crawl the Web updating its database, it
also has the ability to answer "why" questions. This software, called WebFountain,
"is the ultimate way to figure out what everybody is saying about something"
according to Dean Takahashi of Mercury News. He goes on to state: "WebFountain
sifts through the online data to discover patterns that even the most dedicated
librarian can't find."(Takahshi, 2004) IBM is not trying to replace the librarian
but to build a tool for the business market.
The scholarly community could use this software to track the trends in
publishing and many other areas of communication. One of these other areas
would be to track the change over time of what is being discussed online about
a new medical procedure or drug. It could even track citation networks and impact
factors. The biggest problem is cost, "about $200,000 a year" from Factiva who has
licensed it. (Takahshi, 2004) At this price it is only economical for corporate use.
Economics
The god of publishing in the new millennium is the almighty dollar. Money
drives what gets published, what gets archived, and who has access. Universities
are also feeling the powerful sovereignty of the dollar. "Scholarly communication
has been transformed from a means of communicating research results to a multi-billion
dollar business." (Create Change, 2000)
Since the late nineteen nineties many universities have studied the statistics
regarding the cost of purchasing journal subscriptions by libraries. The results of
these studies show a large portion of the libraries' budgets going to a few large
publishers for access to a small number of titles. "In 1997, an informal survey of
ARL members revealed that, on average, the typical North American research library
purchased 378 titles from Elsevier Science (a major company owned by Reed Elsevier).
Libraries reported that that represented about 3.5% of the total journal titles
purchased. At an average cost of $1661, these journals cost a library $628,000, or
almost 21% of the total journals expenditures -- over one-fifth of the journals budget.
During the same year, the average unit cost of all journal subscriptions was $235. At
this unit rate, $628,000 would buy 2,672 titles." (Create Change, 2000)
The profits gained by commercial publishers entering the field of academic
publishing makes it very lucrative. Studies looked at two different comparisons.
The first being the ratio between the cost of the journal and its impact factor
while the second was the cost per 1000 characters. In both cases the commercial
journals came out more expensive than the not-for-profit and association publications.
One of the most expensive journals held by the University of Arizona is the
International Journal of Remote Sensing published by the Taylor and Francis Group
(a commercial publisher). The 2004 subscription cost for this journal is $4700US.
The current impact factor is 1.154. (Taylor and Francis Group, 2004)
The University of Arizona currently subscribes to thousands of journals. The
University of Arizona Library must reduce costs in the face of a reduced budget. For
the following fiscal year, librarians are asking the faculty to review the journal
titles held in their fields with an eye toward cancellations. The table that follows
shows where available the number of titles under consideration for cancellation by
general field.
| Field of Study | # of Journals to Cancel | # of Journals Retained | Total Journals |
| Science and Engineering | 1286 | 1088 | 2374 |
| Fine Arts and Humanities | na | na | 2544 |
| Health Sciences | 70 | na | na |
| Social Science/Business/Education | na | na | ~2530 |
| Newspapers/Indexes/General Periodicals | na | na | 292 |
| Totals | 1356 | 1088 | ~7740 |
Note: Table compiled from information contained in the
"Library Resources Spending Reductions 2004-2005" web page.
na = information not available
While canceling journal subscriptions is not the answer to balancing the
budget it can be a short term fix. Many scholarly institutions are resorting
to forming consortiums and bargaining with publishers to lower the cost of
subscriptions. Another method for libraries to fight high costs is to become
part of the "Open Access Movement." (University of New Mexico, 2004) There are
two parts to this, open access journals and open access archives. The journals are
free of charge and contain peer-reviewed research articles. The archives, in contrast,
are depositories contributed to by the scholars themselves providing a database of
perpetually free materials. (University of New Mexico, 2004)
Conclusion
As stated above, the evaluation of scholarly communication is hindered by
many different factors. The growing trend toward collaboration can make
determining an individual scholar's contribution difficult. Extreme specialization
increases the need for collaboration across disciplines. Increased technology
makes collaboration and interdiciplinarity easier, both of which make evaluation
more difficult. Evaluation of scholarly communication has also changed with
the increase of technology as scholarly communication branches into electronic
formats. Disintermediation is a problem as publishers form databases of their
journals and charge high fees for use. This leads to the problem of the cost
both for journal subscriptions and to access publisher's databases.
To combat these problems, scholars are utilizing some of the same technologies
that make evaluation difficult. Some scholars are calling for a re-evaluation of
the tenure process to eliminate the perceived need to publish just to be published.
This would decrease the total number of published articles but the hope is that they
would be better articles. By forming free access databases and e-journals, scholars
are combating the high costs of access to commercial publishers' products. These same
e-journals and databases can be full-text searched, which makes them easier to use
when researching a specific topic. The WebFountain will be a valuable tool when it
becomes available to the non-corporate world. Technology is both a bain and a
blessing for scholarly communication. All of the changes that have happened in
the last 40 years will only continue and scholars can only hope to try and keep
the doors of communication open by providing access to publications and fighting
the high costs of the commercial publishers.
Cited Works
Bordons, M. & Gomez, I. (2000). "Collaboration Networks in Science." The Web of
Knowledge: A Festschrift in Honor of Eugene Garfield. Medford:
Information Today.
Coleman, A. (July/August 2002) Interdisciplinarity: The Road Ahead for
Education in Digital Libraries. D-Lib Magazine, 8. Retrieved May 2, 2004
from the World Wide Web: http://www.dlib.org/dlib/july02/coleman/07coleman.html
Create Change. (2000). Scholars Have Lost Control: The Transformation of Scholarly
Communication Into Big Business. Retrieved May 4, 2004 from the World Wide Web:
http://www.createchange.org/faculty/issues/scholars.html
Cronin, B., Atkins, H. B. (2000) The Scholar's Spoor. The Web of Knowledge: A
Festschrift in Honor of Eugene Garfield. Medford: Information Today.
Internet Public Library. (2004) About the IPL. Retrieved May 6, 2004 from the World
Wide Web: http://www.ipl.org/
Ogburn, J. "Scholarly Communication, 2002." University of Washington Library
Newsletter. Retrieved May 4, 2004 from the World Wide Web:
http://multilibraries.missouri.edu/engr/schcomm.htm
Takahshi, D. (2004) Monster Librarian at Work. Retrieved May 7, 2004 from
the World Wide Web: http://www.siliconvalley.com/mld/siliconvalley/7870991.htm
Taylor and Francis Group. (2004). International Journal of Remote Sensing. Retrieved
May 6, 2004 from the World Wide Web: http://www.tandf.co.uk/journals/titles/01431161.asp
University of Arizona Library. (2004). Library Resources Spending Reductions 2004-2005.
Retrieved May 6,2004 from the World Wide Web:
http://www.library.arizona.edu/library/teams/sst/ua_libraryexpenditures.htm
University of New Mexico Library . (2004). The Crisis in Scholarly Communication.
Retrieved May 4 2004 from the World Wide Web: http://hsc.unm.edu/library/sc/