Catching the Information Bug

Evaluating Scholarship in a Time of Change

by Gail M. Hicks

Please be advised: I give permission for my final project to be made available through the LIS Learning Showcase web server.

1. INTRODUCTION

Sarah von Unger-Sternberg used the analogy of an intellectual epidemic, first introduced by Goffman in 1964 (Von Unger-Sternberg, 2000), to compare the science of citation indexing to the spreading of a virus. Unlike the average, garden-variety virus, this one is desirable to catch. More than that, mankind as a whole would benefit from its spread. The goal therefore is to put as many susceptible people in contact with the virus as possible. The virus is scholarly communication or, more specifically, scientific research in a given field. Those most susceptible to catching, carrying, and spreading the virus are scientists in that same field. Recent advances in technology have made the virus easier to catch. In fact, the information is now easier to spread and easier for scientists in other fields to contract. What has this done to the virus? Has it mutated? Do scientists in fields other than that field in which the virus was generated pass it on in its intended form? Do the symptoms remain the same? Information and communication technologies are re-defining scholarly communication; there is a trend towards interdisciplinarity, collaboration, and disintermediation and these trends make evaluation of scholarship difficult.

2. BACKGROUND

Scientific researchers have long documented their findings in scholarly articles usually published in scientific or scholarly journals. Other researchers were able to use the reviewed findings published in those articles to further their own research projects. They built on prior findings and published their findings crediting, of course, the prior researcher.

In the mid part of the twentieth century, Eugene Garfield developed a system of collecting and sorting these citations so that citations would be more accessible and more useful. The number of times an article was cited in the works of other scientists came to be a mark of the quality of the research and the successfulness of a research scientist.

A panel of other scientists in the researcher’s particular field first reviewed all of the articles published in these journals. The information, therefore, was and is believed to be accurate and reliable. This peer review process ensured the integrity of the information. Unfortunately, this process tended to be slow, and research conducted last year may not actually be published until next year.

Additionally, this process tended to be somewhat discriminatory. Unpaid authors must purchase reprints of their published findings in order to disseminate them to reprint-requesters (Harnad, 1996). Small institutions, especially those of developing countries and independent research scientists not only may not be able to afford the costly scholarly journals, but also may not be able to afford to distribute their works in the same fashion as an institution or individual with more expendable cash. For smaller, poorer scientists or their supporting institutions, expensive subscription-based journals may be the only way they have of distributing their works. Those researchers that can afford to give their research away obviously can better broadcast their findings. (Muguet, 2003) If you cannot get your work out there, other scientists cannot cite your work.

Distribution is very limited. Journals are increasingly costly, and not all collections can afford to house all the relevant journals in a particular discipline. Again, this limits accessibility and exposure. Also, databases pooling citation indexes heavily favor specific publishers (the Mathew effect). The cause of this phenomenon is not entirely clear (Braun, 2000).

Perhaps necessity is, indeed, the mother of invention. Scientists need to get their findings to other scientists. They need to spread the virus. In the social sense, researchers are collaborative by nature. Citation indexing has brought to light the very intricate and interesting social structure of researchers (White, 2001). Researches are, more and more, archiving their manuscripts before the peer review process in on-line repositories that are accessible for free (Olbert, 2003). Many researchers believe that authors and publishers should focus on making scholarly communication more accessible through collaboration, interdisciplinarity and disintermediation in order to maximize impact, minimize redundancy, and speed scientific progress (Lawrence 2003).

This line of reasoning has raised concerns about the quality of preprint articles and in fact all scientific research available on-line and for free. Is the information reliable before other knowledgeable authors review it? Which authors should be reviewing which articles if scientific disciplines are, in fact, overlapping? Who is interpreting this information and are those persons knowledgeable enough to properly use new scientific findings? Through collaboration, interdisciplinarity and disintermediation, research scientists have found ways to broadcast their viruses in ways no pure strain of illness ever have been. Like all epidemics, this intellectual one has conjured up some legitimate fears concerning the legitimacy of this fast-moving information.

3. LITERATURE REVIEW

I surveyed the scholarship available in this area to determine if any of the following trends in scholarly communication created any difficulties in the evaluation of scholarship:

· Collaboration

· Interdisciplinarity

· Disintermediation

The literature available on this topic is largely available on the internet. This makes sense as the new technology that fosters and facilitates collaboration, interdisciplinarity and disintermediation appears to center around the computer and, more specifically, around the internet and its exploding popularity and availability. Most of the materials are in the form of on-line articles. There are also many journal articles and discussion postings. There are fewer books addressing the issues, and most of those appear to be collections of essays or articles, such as The Web of Knowledge (Cronin and Atkins, 2000).

Within this small body of literature, little attention was paid to the difficulties arising from the latest trends in scholarly communication. More of the available information dealt with the positive aspects of new information and communication technologies and trends. However, several articles presented a few difficulties.

3.1 Collaboration

Howard D. White, in his 2001 article, “Authors as Citers Over Time,” and his 2000 article “Author-centered bibliometrics through CAMEOs: Characterizations automatically made and edited online,” explained the use of author profiles derived from studying and collecting data on an author’s habit of citing other authors. White raises several issues that might affect the quality of research such as collaboration, self-citing, and varying interests.

A study by Wallace Koehler entitled “Information science as “Little Science”: The implication of a bibliometric analysis of the Journal of the American Society for Information Science,” explores the possibility that more practitioners are contributing to the body of published research due to recent trends in scholarly communication.

3.2 Interdisciplinarity

Anita Coleman in her article “Interdisciplinarity” explores this new trend in the university setting. She raises several questions concerning this principle when applied to scholarly communication.

3.3 Disintermediation

Dr. Ina Fourie discusses disintermediation in the context of scholarly communication in her article entitled, “Should we take disintermediation seriously?” Dr. Fourie believes that the answer to that question lies in the intermediary’s vision of his or her role in the dissemination of information. Dr. Fourie is cautiously optimistic about this concepts effect on the evaluation of scholarship.

David Pollard is not optimistic at all. His article, “Why Complex Intranets Don’t Work and ‘Librarians’ do: Re-Intermediation,” gives stark examples of the negative effects of eliminating the intermediary in the field of knowledge dissemination.

4. ARGUMENT

4.1 Collaboration

Scientists, like most humans, are social creatures. However, the production and dissemination of knowledge is a particularly social activity. Publishing scientists and writers in general can be identified by their social networks (White, 2001). When scientists or any publishing writers are influenced by other authors, these authors form a social network. These networks can be mapped through the use of author-centered bibliometrics. Social profiles can be useful in creating “long-term search strategies for the retrieval of documents, maps of the author’s subject matter in various vocabularies, maps of the author’s relationship with other authors who are socially and/or intellectually connected and studies of individual publishing careers over time” (White, 2001).

It may be useful for other researches to know the particular work of a particular author and to be able to follow his progress over time. Additionally, by knowing which other authors a particular author most frequently cites, and therefore associates with, a researcher can be connected with other sources of knowledge within the same field or discipline. This type of author profiling is helpful to that end.

When two or more authors collaborate on a particular project, this process of mapping social networks becomes more convoluted. Just because two authors collaborate from time to time, does not mean that their profiles will be similar. Additionally, this information can be misleading if the two collaborating authors also pursue areas of study other than the one collaborated on (White, 2000). Also, the authors’ citation styles may vary causing inconsistency in their profiles (Ibid.) This might make the profiles and the study of an author’s oeuvre a bit less reliable than if a profile were based only on a single author’s solitary works.

Additionally, the fact that most collaborators are in fact practitioners and not strictly research scientists adds to the numbers of collaborators and further confuses the field (Koehler, 2001). Practitioners are likely to published less and have smaller, more convoluted profiles. To liken this situation to the spread of our desirable virus, if we cannot tell from whom the virus came, we cannot be certain that the virus is the one we intended to spread or to catch.

4.2 Interdisciplinarity

Different disciplines within the sciences use different forms of discourse—different rhetoric—in order to communicate within the discipline (Coleman, 2002). Epistemology within a discipline is often unique as well. This does not mean, however that disciplines do not cross each other. Separate scientific disciplines can, and often do use the tools, methods, language, and, in fact, the research of other disciplines to define, characterize, and further their own findings.

Often, in order to solve a particularly complex problem, several disciplines must and do come together to share information and research (Coleman, 2003). Many funding organizations appear to be increasingly interested in this sort of approach preferring to solve a perceived problem using whatever means or disciplines are necessary in order to complete the task, to simply conducting research to further the knowledge structure within a particular discipline (Coleman, 2002).

The combining of disciplines to further research and the dissemination of knowledge has come to be known as interdisicplinarity. Interdisciplinarity has caused some problems and concerns in the area of scientific research. If researchers are trained in a particular discipline, it may not be reasonable to assume that they will be able to accurately interpret the findings of scientists in other disciplines.

With the development of information and communication technologies that allow freer access to scholarly communications across many different disciplines, many more scientists are using a wider variety of research to expand their own knowledge bases. Universities are being encouraged to develop interdisciplinary curriculums and to establish “learning communities” wherein students can obtain a deeper understanding of the information they are presented with and can integrate materials both in classes and in the professional arena after graduation (Coleman 2002).

This all sounds well and good. However, if not all disciplines ascribe to the same epistemology—if they don’t all use the same language to disseminate knowledge—then students may not be getting any clear information. They may be encouraged to misinterpret facts and findings within one discipline by using the language of another discipline to discern those facts. New interdisciplinary research may, then, be less than accurate.

If this questionable new research is reviewed before publication in scholarly journals, the problems of misinformation will likely be uncovered and the world saved from false conclusions. However, with the current trend toward archiving pre-reviewed findings, there is an increased threat of the dissemination of false information. Then again, who should the peers be that review such collaborative and interdisciplinary studies? If the panel for a particular journal is not trained or experienced in recognizing the language of another discipline, their ability to evaluate the article and the information contained therein would appear to be greatly reduced. In terms of our intellectual epidemic, similar symptoms caused by similar viruses may be mistaken for the pure strain we were attempting to broadcast.

4.3 Disintermediation

At first glance, disintermediation would appear to be the best possible way to spread our virus to absolutely everyone. No longer would our epidemic be spreading, but would, in fact, be spread, if everyone that wanted the virus could simply visit a repository and freely contract it at will. However, if the end-user of our hypothetical virus does not truly know what s/he is after, or does not recognize the virus for what it is, s/he may misuse it, pass it over, or pass it on to those who are better left uninfected. We would do better to leave the spread of the virus to those who specialize in spreading it. The dissemination of information is no less technical. Intermediaries are just as necessary to the spread of accurate scholarly communication.

Information on the internet is still confusing and largely disorganized. On-line searching requires specialized skills in spite of the existence of databases that claim to be user friendly (Fourie, 1997). Without some guidance, inexperienced users may end up with an incomplete view of the desired search area. If end-users use incomplete or outdated information to generate further research projects, those projects could end up as inaccurate articles. If those articles are not reviewed before publication to the general public, there may be an epidemic of misinformation.

In a more commercial sense, if a commercial research organization becomes disintermediated one of two things may happen. First, if research is left to the end-user, in this situation, the research scientist, more time is spent gathering resources than in conducting new research. Second, if the research is left to administrative staff, then the information gathered is likely to be incomplete because staff members are not likely to have the knowledge base necessary to conduct complete and accurate searches (Pollard, 2003). Again, the quality of the end product of the research scientist is potentially compromised and one’s ability to differentiate between quality scholarship and that which is incomplete and or inaccurate is diminished.

5. CONCLUSION

While the trends toward interdisciplinarity, collaboration, and disintermediation are part of the overall movement toward the broader dissemination of knowledge and freer access of information, they might also be seen as weakening the quality of much needed information. In the apparent furor to make all the information and research available to everyone, some serious and impartial research is needed in the area of information quality control. Much focus has been put on extolling the virtues of cooperation between the sciences, ready, easy, and free access to new research, and the goal of scientists to inform the public at large. However, little energy has gone into exploring those same virtues. Little time or paper space has gone into discovering how new information and communication technologies may have made the evaluation of scholarship more difficult.

6. REFERENCES

Braun, Tibor, Wolfgang Glanzel and Andras Shubbert, “How Balanced is the Science Citation Index’s Journal Coverage? A Preliminary Overview of Macrolevel Statistical Data.” The Web of Knowledge, A Flestschrift in Honor of Eugene Garfield, 2000, 251-277.

Coleman, Anita, “Interdisciplinarity, The Road Ahead for Education in Digital Libraries.” D-Lib Magazine July/August. 2002. <http://www.dlib.org/dlib/july02/coleman/07coleman.html>

Coleman, Anita, “Module 2: The Scientific Literature” IRLS 589 – Scholarly Communication - Spring 2003. <http://moltar.ccit.arizona.edu:8500/SCRIPT/10680WEBCTPID531RID2SEMSpring2003/scripts/student/serve_bulletin>

Fourie, Ina, “Should We Take Disintermediation Seriously?” SAOUG Archive, Fourth SA Online, 1997. <http://www.saoug.org.za/archive/1997/9706.htm>, 2003

Frazier, Kenneth, “The Librarian’s Dilemma.” D-Lib Magazine, March 2001. <http://www.dlib.org/dlib/march01/frazier/o3frazier.html>

Harnad, Stevan, “Harnad Comments on APA Interim Internet Publishing Policy.” <http://www.trauma-pages.com/harnad96.htm> December 1996

Koehler, Wallace, “Information Science as ‘Little Science’: The Implications of a Bibliometric Analysis of the Journal of the American Society of Information Science.” Scientometrics, Vol. 51, No.1, 2001.

Lawrence, Steve, “Free Online Availability Substantially Increases a Paper’s Impact.” <http://www.nature.com/nature/debates/e-access/Articles/lawrence.html>, 2003

Muguet, Francis, “Open Society and Open Access to Intellectual & Artistic Works, Getting a More Global Picture.” <http://www.ensta.fr/~muguet/openaccess/opensociety.html>, 2003

Olbert, Doug, “Dlis/Library Literature Comparison.” <http://moltar.ccit.arizona.edu:8500/SCRIPT/10680WEBCTPID531RID2SEMSpring2003/scripts/student/serve_bulletin?VIEWMAILATTACH+BULLETIN+265> 2003.

Pollard, David, “Why Complex Intranets Don’t Work and ‘Librarians’ Do: Re-Intermediation.” <http://blogs.salon.com/0002007/2003/04/02.html>

Von Ungern-Sternberg, “Scientific Communication and Bibliometrics.” <http//www.abo.fi/~sungern/comm00.htm> 2002

White, Howard, “Author-Centered Bibliometrics Through CAMEOs: Characterizations Automatically Made and Edited Online.” Scientometrics, Vol. 51 No. 3, 2001.

White, Howard, “Authors as Citers Over Time.” Journal of the American Society for Information Science and Technology, Vo. 52 No. 2, 2001.