Information and communication technologies are re-defining scholarly communication; there is a trend towards interdisciplinarity, collaboration, and disintermediation and these trends make evaluation of scholarship difficult.
"Undoubtedly, the greatest challenge -- not only for Scientometrics, but for the whole scientific publication system is the advent of electronic publication and communication...." (Schubert, 2002). Continued advances in information science and technology provide state-of-the-art techniques, equipment, and processes. The technological infrastructure allows the growth of communication networks. As network and storage resources become increasingly available and decline in cost, electronic distribution of information and communication, including communicating scholarly results, becomes the more effective and efficient methods. For example, technological advances enable us to have digital libraries which encompass a large scope of scholarly communication.
Electronic publishing and the digitization of information redefines scholarly communication. A globally accessible telecommunication network brings a multi-faceted array of opportunities for communication among scholars, from casual messaging to complex software systems. Scholars can exchange ideas and review each others’ work without leaving their own desks. Sustainable and expanding networks impact the information distribution process by providing the speed, functionality, and the cost efficiency of electronic publishing.
A survey of the scientific literature assists in putting scholarly communication into the context of technological change.
Communication is the "[e]xchange of information between individuals by means of a common signal system" (von Ungern-Sternberg, 2000). Scholars use and disseminate information through formal and informal channels (Coleman, 2003; Cole, 2000). Computer networks facilitate electronic collaboration between individuals or teams, allowing segments of research projects, for example, to take place thousands of miles apart. On a grand scale, networks facilitate the "registration, evaluation, dissemination and archiving of human knowledge" (von Ungern-Sternberg, 2000).
Interdisciplinarity is best understood by first discussing disciplines. A discipline is a body of specialized scientific research of a homogenous subject matter (Coleman, 2003). The subject matter is the sector of a material field identified and focused on by the discipline. This establishes a point of view from which a discipline operates.
Technology effects different disciplines differently. Disciplines differ in their levels of theoretical integration, which is effected in part by its state of maturity. They differ in whether knowledge is constructed empirically or theoretically. Methods differ, including those used to gather data and interpret and transform it into information. Analytical tools differ, as do the applications in practice.
Looking at the criteria for a discipline also reveals an immense overlap in material (i.e. psychology and neuroscience). This overlap brings about issues of interdisciplinarity, the integration of concepts and epistemologies from different disciplines to work toward an agreed upon goal. The occurrence of an interdisciplinary endeavor serves to change existing disciplines or create new ones. "The ‘inter-discipline’ of today is the ‘discipline’ of tomorrow" (Coleman, 2002).
Interdisciplinary study is modeled by Eugene Garfield’s life and by The Scientist, of which he was publisher and editor while he was at ISI. Scholarly articles published in that journal were from the varied disciplines of history, sociology, information science, and others.
Collaboration occurs when "two or more scientists work" together on a joint research project, sharing intellectual, economic and physical resources" (Bordons, 2000). Collaboration among individuals and/or institutions for the performance of scientific research is on the increase and is easier than ever, given technological advances. It is fostered by policymakers, funding agencies, and the growth of research in "big" science that requires vast resources (i.e., laboratories, instruments, personnel) (Koehler, 2001).
Collaboration is a complex social process but only its formal activities can be measured. (Bordons, 2000) The study of collaboration using bibliometrics is limited to databases that include names of all authors and addresses of all institutions. Science Citation Index (SCI) is a multidisciplinary and internationally broad-based database and is suited to the study of collaboration, because it covers international and mainstream journals (Bordons, 2000). Eugene Garfield called the citation indexes an "association-of-ideas index" (Garfield, 1956).
Disintermediation occurs when the user is given direct access to information that would otherwise require a mediator (i.e., librarian, lawyer, doctor). It is a result of technological advances and would probably not occur without the existance of computers and networks. Disintermediation occurs in scholarly communication, for example, with electronic database aggregation. (von Ungern-Sternberg, 2003).
Typically scholarly publishing has been done on paper in a linear, essay-type format. Technology allows electronic publishing, which enable non-sequential presentation and browsing of material. Electronic media replaces paper, and citations can be replaced with hyperlinks that join works together. Visuals, such as sound, film, and animation, can be incorporated, and interactive presentations can add to the content of the work.
Bibliometrics is the family of techniques used to identify relationships in published literature. This gives a simplified picture of communication and is the basis for studying scientific communication. It is a quantitative method that came into being when content-based and technological tools were developed (Colman, 2003). Bibliometics is also involved with collection development and has two distinct research traditions: bibliometrics and citation analysis (Narin, 2000).
Bibliometrics relies on the fact that scientists who have something important to say, publish their research. The exchange of research results, either formally or informally, is crucial. This allows the work of individuals, institutions, countries, and disciplines to be evaluated, including collaborations between institutional sectors (i.e., industry, university, government). Bibliometric methods provides indicators that are useful tools for studying collaboration in science, as well as to evaluate scientific communication and activity, including research or scholarly productivity and quality. It enables cross-disciplinary collaboration links (Bordons, 2000).
Citation analysis is the measure of impact and quality of scientific work and intellectual influence of scholars and scientists. It is also a measure of collaboration and movement of scholarly ideas from one field to another. It is the measure of intellectual influence and the impact of an individual’s work on peers. Citation analysis is used to rank universities and colleges and evaluate the suitability of its members for tenure. (Cole, J.R., 2003).
SCI’s main significance is bibliometrics, scientometrics, and research performance assessment. It is a "beautiful example of the application of an old concept (citation indexing) in a new context (science) with new technology (computers)" (Wouters, 2000). SCI is built on citations, not references. Citations are the metrics by which the importance of research is measured and are used as a measure of scientific quality.
The communication and evaluation of scholarly research regulates and channels the growth of knowledge, whether it be core, "textbook" knowledge or frontier knowledge which is newly published in journals. Core knowledge is "theories, analytic techniques, and facts that represent the given." This knowledge differs by discipline, as does using it to establish consensus. (Cole, 2000)
Scholarly publishing of research and knowledge connects authors and readers. The review process filters quality, however, it is biased toward authors and segments of research. It signifies consensus, but delays in the publication of material are the price. These barriers function to help in the process of self-selection.
"Cross-fertilization of subject fields is one of our most important problems in scientific literature" (Lederberg, 2000). Advanced networks and electronic communication should make it easier to collaborate and produce research results, but the concept of interdisciplinarity can be misleading in that it does not imply cooperation. Instead interdisciplinarity increases the opportunities for misunderstanding and conflict. Disciplines tend to divide knowledge into components and professionals into autonomous fiefs.
In theory, scientific productivity is determined by the amount of publications and citations. A scientist’s career depends on the evaluations of his/her work by other scientists (i.e., publication of research, grants, promotions, etc.). The functions of publishing scientific work is the researcher’s contribution to the growth of common knowledge. The importance of scholarly communication is to establish ownership of an idea, to gain societal recognition, to claim priority, and to establish an accredited community of authors and readers (Fjalbrant, 1997). Electronic publishing, however, presents barriers.
The academic reward system (i.e., promotion and tenure), does not value collaboration and interdisciplinarity the same way it rewards publication in top journals Scholarly publication must be measurable if it is to be recognized. (Coleman, 2002). It is not if publication takes place outside the traditional publication system (i.e., in a journal).
With electronic communication and publishing, it is easier for the communication and evaluation of scholarly research to take place outside the traditional "system". While this "non-system", socially-oriented communication traditionally was verbal, it is now mostly electronic (Fjalbrant, 1997). The community of scholars may be visible (i.e., professional societies, conferences, workshops, listservs, etc.) but is just as likely to exist in invisible colleges, networks of scholars who cite each other and maintain informal communication (Bordons, 2000). But invisible networks make the evaluation of scholarship difficult.
The system of pre-print servers enables and promotes "non-system" communication of scholarly work. Pre-print repositories contain scientific results deposited prior to publication elsewhere, or perhaps not published at all. Scientists can download and provide feedback, and the research process continues without delays from the traditional publication system. This non-traditional system results in a decrease in the market for journals and results in publishers getting very nervous.
Disintermediation challenges the relationship between users and service providers (von Ungern-Sternberg, 2003). This includes publishers and library and information science professionals. Electronic publishing effects the existing distribution culture, which is comfortable, both economically and socially, with things the way they are. Publishers want to maintain control of the dissemination of their property. Electronic publishing , however, offers opportunities for direct marketing, which does away with the "middle-man".
Bibliometics studies indicators of scholarly communication -- not the social process. Pre-print repositories, and other non-system communications, are not indexed so there is no way to track scholars work until it is published (Coleman, 2002).
Citation analysis measures the value of the cited document. In theory, more citations mean more value. But citations can also be criticisms or corrections. While the subject content of the citing document is related to the document cited, the role of cited documents range from being central and important to the citer’s research to trivial attention paid for social or political reasons. Also, the purpose of citation analysis is to keep track of a scholar’s citation trail, both the trail they follow and the one they leave behind. But it is difficult, if not impossible (depending on the venue), to follow the trail of electronically published work.
Disciplines within the scientific community have established learned societies, publishers, and libraries which shape the scientific environment and structure. Scientific communication has degrees of social consequences, depending on the discipline. Common standards and specific rules and ethics vary. Scientists establish their political and social positions through recognition of their scientific contributions. The works of scientists are designed to persuade readers in the scientific community of the acceptability of the claims presented. So authors cite other works to support claims (von Ungern-Sternberg, 2000).
The results of a collaborative project are communicated through publication and co-authorship. Co-authorship can be considered a proxy for studying collaboration, but collaboration doesn’t always produce co-authored papers. Interdisciplinary research may warrant multiple publications in multiple versions and venues in order to reach all its audience members. A problem of co-authorship is the assumption that all authors named participated in the research and that their contributions are merged and inseparable. But this is not always the case. It is often unapparent how credit for scientific work should be distributed (i.e. given to the first author or to all authors in equal fractions) (Bordons, 2000).
Collaboration and interdisciplinary research are enhanced by technological advances and can result in an increase in productivity of participating institutions and countries (Bordons & Gómez, 2000). Also expensive resources can be optimized. But, from the point of view of an individual research, it can also be argued that there is a decrease in productivity due to the large amounts of non-research time spent in negotiation, project coordination, logistics, and learning. (Colman, 2003). No amount of effort can make up for this time, not even technology.
Advances in information and communication technologies have resulting in new definitions of scholarly communication. The practices that technologies enable, such as interdiciplinarity, collaboration, and disintermediation make evaluation of a scholar’s work difficult. One of the key methods of evaluating scholarship, citation analysis, has become difficult (or impossible) within electronic publishing.
Bordons, M. & Gómez, I. (2000). "Collaboration Networks in Science" The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ. p. 197-213.
Cole, J.R. (2000) "A short history of the use of citations as a measure of the impact of scientific and scholarly work" The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ. p. 281-300.
Cole, S. (2000) "The role of journals in the growth of scientific knowledge" The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ. p. 109-141.
Coleman, A. (2002) "Interdisciplinarity: The road ahead for education in digital libraries" D-Lib Magazine 8(7/8) Online at http://www.dlib.org/dlib/july02/coleman/07coleman.html. Last visited 4/16/03.
Colman, A. (2003). Website for IRLS 589, Scholarly Communication - Spring 2003. Last visited 5/9/03.
Fjällbrant, N. (1997) http://educate.lib.chalmers.se/IATUL/proceedcontents/fullpaper/nfpaper.html. Last visited 2/5/03.
Garfield, E. (1955). "Citation indexes for science". Science (122) in The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ., p. 108-111.
ISI Web of Knowledge (2002). Company brochure. http://www.isinet.com Last visited
Koehler, W. (2001). "Information science as 'Little Science': The implications of a bibliometric analysis of the Journal of the American Society for Information Science". 51(1), p. 117-132.
Lederberg, J. (2000) "How the Science Citation Index got started" The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ. p. 25-64.
Schubert, A. (2000) "The web of Scientometrics: A statistical overview of the first 50 volumes of the journal." Scientometrics, 53(1), p. 3-20.
Thackray, A. & Brock, D.C. (2000) "Eugene Garfield: History, scientific information, and chemical endeavor" The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ. p. 11-23.
von Ungern-Sternberg, S. (2003) "Scientific communication and bibliometrics". http://www.abo.fi/%7Esungern/comm00.htm
Wouters, P. (2000) "Garfield as alchemist" The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Information Today:Medford, NJ. p. 65-71.