I give permission for my final project to be made
available through the LIS Learning Showcase web server.
Melinda M. Dodd
IRLS 589: Scholarly Communication
Dr. Anita Coleman
9 May 2003
Final Paper, Option 1
The Future of Scholarship:
Evaluating the New Landscape
Introduction:
The development and rapid movement in the field of technology has changed the face of modern society from what it was twenty years ago. Information and communication technologies have changed dramatically even in the last ten years. Electronic mail, listservs, and the Internet, to name a few, are all parts of the new technology that is re-defining scholarly communication. In her article entitled “Scholarly Communication” Christine Borgman states that “[r]esearch was clustered around three variables: producers of the communication . . ., artifacts of communication . . . and communication concepts.” (146) The impact of the new communication technologies to scholarly communication cannot be understated. While the benefits are extensive, it also raises a number of questions. Kahin notes that “the digitization of information and the growth of research networking raise a distinct set of issues: How should the flow of research information be managed in the interests of scholarship and technological progress?” (1) In part because of this new technology as well as the ease and speed with which communication can take place and information can be disseminated, there is a trend within the scholarly community towards interdisciplinarity, collaboration, and disintermediation. In many ways, these three concepts overlap with one another creating an entirely new landscape for scholarly communication, research and education. When discussing interdisciplinarity it is first necessary to come to an understanding of what constitutes a discipline, its purpose and its function. In “Interdisciplinarity: The Road Ahead for Education in Digital Libraries”, Anita Coleman states that “[a]cademic disciplines have long been used to organize knowledge for teaching purposes. They are also the basis of organizing higher education and the professions engaged in teaching and research. The division of knowledge into disciplines, however, doesn’t impose a pre-ordained order and doesn’t easily transfer to the modern workplace. Many reports have stressed the importance of incorporating interdisciplinarity into the undergraduate experience at the research university.” (13) Elsewhere she states that “[d]isciplinarity is specialized scientific exploration of a particular homogenous subject matter, producing new knowledge and making obsolete old knowledge. Therefore, disciplinary activity results in continuous formulations and reformulations of the present body of knowledge about the subject matter.” (Coleman “The New Scholarship” 1) With this in mind, interdisciplinarity occurs when “curricula and/or research and innovation are organized around a problem that is too broad to be studied/solved using just the methods and knowledge in one discipline (Coleman “Interdisciplinarity” 9) When such a problem is addressed within the scholarly community, interdisciplinarity leads directly into collaboration. Maria Bordons and Isabel Gomez define scientific collaboration as “two or more scientists working together on a joint research project, sharing intellectual, economic and/or physical resources.” (198) This definition however, can easily be expanded to all areas of the scholarly community.
As more and more interdisciplinary and collaborative teaching and research occurs, there is more direct contact between scholars, studies, and institutions, aided once more by the new technology. This then, brings us to the third trend in scholarly communication: disintermediation. Disintermediation has been described as “removing the middleman . . . Many people believe that the Internet will revolutionize the way products are bought and sold, and disintermediation is the driving force behind this revolution” (webopedia). Although this definition refers more to the business world and society at large, it sill has application within the scholarly community. “Removing the middleman” can mean a variety of things depending on the placement and function of the institution within the big picture. Regardless, the role of all people and institutions is changing with the elimination and restructuring of traditional roles within the scholarly community. This means that it will also be necessary for the community as a whole to determine if the traditional modes of evaluation are still relevant, or even possible, and how they must be changed to accommodate the new structures within the community.
There is no doubt that there are both pros and cons to all of these issues: collaboration, disintermediation, and interdisciplinarity, but regardless of the varying opinions on them, the fact remains that they are now a part of the scholarly community and they must be addressed in terms of their evaluation and how the scholarly community as a whole is going to address these issues. Indeed, the difficulty of evaluation is a point of frustration for many of those involved in research. These issues, now that they are here, will only continue to spread. Although collaboration is primarily only currently ‘popular’ in the scientific community (arts and humanities utilize collaboration to a much smaller degree) its presence indicates a trend in the scholarly community that will not abate.
Literature Review:
The literature regarding the evaluation of scholarly communication covers a broad range of issues. Of particular interest here however, are those relating to interdisciplinarity, collaboration, and disintermediation. In his article “The Role of Journals in The Growth of Scientific Knowledge” Stephen Cole states that “[f]rom a sociological point of view, journals should . . . be considered . . .one mechanism in the evaluation system.” (112). He goes on to say that “[t]he fact that virtually all articles can be published presents a significant problem for the journal system in serving its evaluation function . . . the journal system can aid in the evaluation of new knowledge by concentrating most of the high-quality new contributions in a very small number of journals.” (116) While there are certainly many other aspects of the overall evaluation of both scholars and scholarship, much contemporary thought focuses on the scholarly journal and its place in the new scholarly community. Three of the main areas that are being addressed in regard to this are: 1 – the rights of authorship and scholarly work; 2 - bibilometric databases and citation indexes; 3 – publication and new technological formats.
In their article “Collaboration Networks in Science” Maria Bordons and Isabel Gomez discuss collaborative work and the difficulties regarding authorship evaluation. They state that “the rate of collaboration in scientific activity has grown and, today, it is a key element in the development of research in certain areas. A shift away from the traditional individualism of early scientific activity toward a more collective process has occurred.” (197) With this movement toward collaboration come the complexities involving multiple authorship in publication and who receives credit for the research and subsequent publication. Bordons and Gomez note in their discussion that options for this differ among publishers, databases, and institutions, with credit being assigned in a number of ways, among them “total credit to the first author (first author counting), assigning full credit to every author (total counting), or giving an equal fraction of the credit to each of the authors (fractional counting).” Additionally, there is often a problem with the lack of normalization of author, center, and even country names. (201)
Brian Kahin expands these ideas and applies them to the new technology by pointing out that “[j]oint authorship means joint ownership, and a joint owner can legally license the work for publication – or simply distribute it over the network without permission of the other joint owners. The joint owner who distributes the work is liable to the others only for their share of any money received.” An added complexity is that aspects of one project may be owned by different people or even institutions that are not affiliated with one another except through a single project. (4) This becomes even more problematic when one author modifies jointly owned work, creating in effect a new piece of scholarship that she has exclusive rights to (5). Although Kahin does allow that difficulties in the claim to authorship are not new with the increase in collaborative projects, he argues that prior to networked collaboration research projects were supervised by a “senior figure” responsible for the “review of all material intended for publication. Remote collaborations are likely to involve two or more individuals of relatively equal stature, each of whom is accustomed to acting independently, thereby increasing opportunities for misunderstanding.” (3)
Bibliometric Databases and
Citation Indexes
The issues involving authorship have a direct effect on the evaluative process of bibliometric databases and citation indexes. Bordons and Gomez point out that “bibliometric indicators have emerged as useful tools for the study of collaboration in science. Collaboration between several authors in the development of any research work is usually reflected in the authorship of the resulting scientific documents.” (197). However, they go on to say that “very few databases are comprehensive in relation to the names of all authors and their institutional addresses.” (200) Indeed, many databases policy states that they only register partial information regarding authors and institutions (198). Although they appear to be quite supportive of bibliometric indicators as being scientific and reliable in term of methods and results, they admit that it does have its limitations. Among these are the reliability of authorship (“honorary co-authors” have been listed for “purely social reasons”), collaboration within the research does not necessarily mean that the final published material will be co-authored, all bibliometric databases do not include the names of all authors or all of their addresses, and so on. (Bordons 199-200) However, other scholars also support the value of bibliometric databases and citation indexes, stating that “[w]hatever the limitations of citation indexing and analysis, it would be difficult to conclude that the aggregate, collective voice of the global scientific community should be ignored.” (Davenport 522)
Kahin examines at great length the role of publication, and ultimately publishers in the new technology. He states that “[a]uthors, research programs, and scholarly organizations may find themselves negotiating with complex multi-publisher systems (the next generation of online vendors) to design dissemination programs that fit the institutional landscape, culture, and other information resources of their specialty. How much flexibility should authors (and research projects) have to recast their work in the light of new knowledge or for the benefit of new audiences?” (10-11). He also offers alternatives to the electronic journal, pointing out that “the textbook which is reissued in new editions every year or two may be a better model for some forms of network publication.” (Kahin 11) Cole also looks at alternatives, basing his ideas on an “ideal communication system . . . in which each individual scientist could easily and rapidly learn about all other work that was relevant in more or less direct ways to her own.” (110). He points out that researchers are dependent upon their community to identify relevant scholarship. He states that “journals only provide a place for new work to be published”. Much more important is the communication and evaluation system within the community. (Cole 111) Kahin also appears to share this view and proposes an “intelligent networked environment [in which] the article is emancipated from its archival home in journal time and space. It is instead found and used in functional space, linked to related articles, reviews, commentary, abstracts, and references, and perhaps available in multiple forms and versions.” (12) He goes on to say that “networked digital information is sometimes envisioned as a leveling, democratizing force, capable of transcending any distance and rendering all information equally available to all points on the network.” (13)
Arguments
As discussed in the literature, the problems involving evaluation with the increase of interdisciplinarity, collaboration, and disintermediation are myriad. Indeed, as much as the prospects for the future of scholarly communication are exciting, they also raise a number of questions regarding research, authorship, and education. Cole states that one “component of knowledge, the research frontier, consists of all the work currently being produced by all active researchers in a given discipline. . . [it] is linked to the core through the evaluation process . . .Papers that are ignored are functionally equivalent to those which were never written; they have no impact on the growth of communal knowledge . . . only a tiny handful pass through the evaluation system from the frontier to the core.” (110) If the research frontier is where all new knowledge within a discipline is created, then when research is interdisciplinary, to whom does the new knowledge belong? Obviously it has relevance in a number of areas, thus the interdisciplinarity of the research in the first place, but given the difficulties of regarding authorship, ownership, and the ultimate dissemination of information discussed previously, how can we ensure accurate credit is given? Additionally, if the evaluation system linking research to the core theory of a discipline is more or less eliminated through disintermediation, how then can we determine the relevance and importance of the research? How then can we continue to create new knowledge and increase understanding within a discipline if we have no set manner in which to link it to the core theories of the discipline? Similar questions can be applied to the issue of interdisciplinarity: if the results of the research are such that it relates directly to the core theory of one of the disciplines involved but not to the other, does that invalidate or lessen the contribution of the other contributor/discipline?
Part of the problem with the evaluation of collaborative research is that, as Bordons and Gomez point out, there are various classification of collaboration: theoretical, technical, student-teacher, assistant-director, researcher-technician, peer-to-peer, etcetera. Additionally, “[s]ome contributions can be very valuable, while others are almost negligible. The nature and magnitude of the contribution of each collaborator can even change during the course of a research project. Finally, an additional problem in attempting to measure collaboration is that it covers not only formal activities, as represented by funded projects, but also informal cooperation not embodied in any kind of contract.” (198) There have always been problems with consistency of names, spelling, institutions, address, and so on; problems that are compounded with international collaboration, bringing forward language and cultural barriers in addition to possible inequities in funding, equipment, and other factors. These affect the evaluative process between and within databases and/or disciplines, not to mention the increasingly global scholarly community. This bring us, once again, to the importance of journals and citations. Davenport offers the following example:
“If B has published his
results in a prestigious journal, known for its high standards and rigorous
peer review, then A’s acceptance of B’s findings is in part a function of the
perceived trustworthiness of the journal and its editorial policies. The more trustworthy the journal, the
greater the presumed credibility of the reported results. A’s trust in B arises from A’s belief in the
trustworthiness of the institutional arrangements . . . rather than from a
direct assessment of B’s professional or personal integrity.” (522)
So if the value of the scholar and of the work is based to a large extent on where it is published, it is even more important that the information regarding authorship and contribution is accurate and consistent. Even with cross-referencing in multiple multi-disciplinary databases, we have to ask: what is the cross over? What about standards for inclusion? How much work was done by each author? And the ultimate question, should collaborative work ‘weigh’ the same as single author work? Additionally, it leads to speculation regarding the future place of interdisciplinary studies within the scholarly community. So the question then becomes, does the movement toward interdisciplinarity constitute the rise of another ‘discipline’ or simply a further negation of traditional boundaries?
Another issue in collaborative evaluation is, as Bordons and Gomez point out, that there is “a wide range of situations” that can be considered collaboration, including, but not limited to, “expressing an opinion, exchanging ideas and data; working together during the course of a project; working separately on different parts of a project with the aim of the final integration of results, and so forth.” (198) Often progressive research will invoke opinions given informally (i.e. not published) but still may have an affect on the research, possibly contributing to new direction or thought. How do we evaluate this contribution to the scholarly community? Kahin states that “[w]hile very short comments are not considered protectable by copyright, at some ill-defined point contributions become protectable expression. Such contributions remain the property of their authors, unless it is clearly understood that they are dedicated to the public domain, either by the statement of the authors or by and explicitly stated policy on contributions to the list.” (Kahin 7) Once again, the question becomes one of ownership and contribution. We may be able to evaluate the research in terms of its ongoing contribution, but how then do we evaluate the contribution of the individual researchers? If a project is extremely long term and the research team changes over time, how much credit to we as a scholarly community allot to the participant? What of those who helped ‘refocus’ a research project but do not participate in the writing of the analysis? Because of the value placed on individual contributions for scholars (grant funding, tenure, reputation, raises, lecture requests, institutional ‘perks’), it is important that the system in place for ‘objective’ evaluation be effective from the outset.
With disintermediation (i.e. removing the middleman), research studies, publication and even work/research in progress may become directly available to other scholars and students in the field. The question then becomes, how do we insure that the information regarding the work is given to the appropriate bibliometric database? As previously discussed, although the bibliometric database is not a complete evaluative tool in and of itself, its importance, as discussed by Bordons and Gomez as well as other, cannot be ignored. If the role of the publisher is removed, who will insure correct author names, which is already a problem, or correct order, for those that base value on first author only? For that matter, if we begin removing the publishers and disseminators of scholarly research, allowing say, the author to ‘publish’ the findings, what then constitutes true publication that is concrete enough to be evaluated for grants, tenure, raises etc. For that matter, if the institutions becomes responsible for disseminating or ‘owning’ the results of the research as some have suggested, then when dealing with collaborative work, which institution ‘owns’ the rights and for what duration of time? Although we certainly shouldn’t devalue the importance of bibliometric indicators, as discussed earlier by Bordons and Gomez, we must keep in mind that they are not all inclusive and there is much that they cannot show because of their dependence on the formal document itself. In many ways, they are indicators much as statistics are: they are true, as far as they go. And while they are a good baseline for evaluation of the importance and value of a document or author(s), because of their limitations they are not all inclusive. Without additional information, they are only part of the picture.
It should be noted that it is not being suggested that any one part of the scholarly community, including publishing companies, be eliminated, but rather that a change in their function and operating procedures must occur and be properly evaluated in terms of the greatest benefit to the broadest base. Indeed, the restructuring has already begun in earnest, albeit in a direction that many find disconcerting. In “The Librarian’s Dilemma” Frazier argues vehemently for librarians and scholars to refuse the blandishments of the publishing industry to accept their “one-size fits all package” of scholarly journals. His concern is that “[l]ibrarians will lose the opportunity to shape the content or quality of journal literature through the selection process.” (2) He discusses the alternatives for librarians and scholars to maintain some voice within the evaluative process and goes on later to say that “[i]n investing in these new forms of scholarly communication, we are steadily building the publishing infrastructure so that future scholars may never have to publish in an expensive commercial journal in order to be academically successful.” (7)
Conclusion:
The opinions and views regarding the influence of new technologies on the scholarly community are varied and encompassing. Regardless, the fact remains that the technology will only increase and the roles of scholars, researchers, institutions, and educators will all be changed dramatically. Interdisciplinarity, collaboration, disintermediation are a few of the ways in which it has already begun to change. The true test of the scholarly community will be its ability to adjust and expand in a manner that is beneficial in all disciplines. As Bordons and Gomez point out, “some of the most significant scientific advances come about when integrating methods or concepts from different disciplines, and this can only be achieved by means of collaboration between experts in each of the disciplines.” (199) This same idea can be applied to the changing roles and structures that are inevitable within the community as a whole. The possibilities are endless if we can look forward to the ideal while maintaining a strong foundation of knowledge to build upon.
Works Cited:
Bordons, Maria and Isabel Gomez. “Collaboration Networks in Science.” Cronin and Atkins 197-214.
Borgman, Christine L. “Scholarly Communication.” Cronin and Atkins 143-162.
Cole, Stephen. “The Role of Journals in The Growth of Scientific Knowledge.” Cronin and Atkins 109-142.
Coleman, Anita. “Interdisciplinarity: The Road Ahead for Education in Digital Libraries.” D-Lib Magazine. 8.7/8 (2002). 5 May 2003 <http://www.dlib.org/dlib/july02/coleman/07coleman.html>.
Coleman, Anita. “The New Scholarship.” Scholarly Communication Class Notes. University of Arizona. 5 May 2003.
Cronin, Blaise and Helen Barsky Atkins, eds. The Web of Knowledge: A Festschrift in Honor of Eugene Garfield. Medford, New Jersey: Information Today, Inc., 2000.
Davenport, Elizabeth and Blaise Cronin. “The Citation Network as a Prototype for Representing Trust in Virtual Environments.” Cronin and Atkins 517-534.
Frazier, Kenneth. “The Librarian’s Dilemma: Contemplating the Costs of the “Big Deal”.” D-Lib Magazine. 7.3 (2001). 5 May 2003 <http://www.dlib.org/dlib/march01/frazier/03frazier.html>.
Kahin, Brian. “Scholarly Communication in the Network Environment Issues of Principle, Policy, and Practice.” 18 Feb. 1992. 5 May 2003 <http://www.eff.org/IP/kahin_scholarly_communication>.
Webopedia. Jupitermedia Corporation. 5 May 2003. <http://www.webopedia.com/TERM/D/disintermediation.html>.