I give permission for this
project to be made available through the LIS Learning Showcase.
Bibliometric Study of the International
Journal of Geographical Information Science
and
One of Its Top Cited Authors, Fulong Wu
Gary F. Daught
School of Information Resources and
Library Science, University of Arizona, Tucson, AZ
IRLS 589 Scholarly Communication
Professor Anita Coleman
May 7, 2004
Introduction
Bibliometrics, is a
word derived from a compound of two Greek terms, biblios "book" and metrikos "a measurement, or a
measuring." But the "measurement" that is sought from the "book" is not
its physical dimensions. Rather, bibliometrics is interested in
measuring the life and impact of a discrete (traditionally written)
form of communication within an intellectual community or the broader
society. Borgman (2000) alludes to two classic definitions of
bibliometrics: 1) the "counting and analyzing various facets of written
communication" to shed light on the development of a discipline
(Pritchard, 1968); and 2) the process of "assembling and interpreting
statistics relating to books and periodicals...to demonstrate
historical movements (Raisig, 1962)" (p. 144). As applied to scholarly
communication, bibliometrics help researchers discern (through
quantified data) the development of a discipline, and the patterns of
communication among scholars and researchers.
The bibliometrician's primary data are citations. Cronin and Atkins
(2000) quote Eugene Garfield (1979, p. 1) who writes: "Almost all the
papers, notes, reviews, corrections and correspondence published in
scientific journals contain citations. These cite--generally by title,
author, and where and when published--documents that support, provide
precedent for, illustrate, or elaborate on what the author has to say. Citations are the formal, explicit
linkages between papers that have particular points in common"
(p. 2, italics added). This
idea is captured nicely by Joshua Lederberg, who termed reference
citations as marking the "parent-offspring relationships of
publications" (Cronin and Atkins, 2000, p. 3).
This paper will present a short bibliometric study of the International
Journal of Geographical Information Science (IJGIS), one of the first scholarly
journals dedicated exclusively to the interdisciplinary field of
Geographical Information Systems (GIS), and one of IJGIS's top cited authors, Dr.
Fulong Wu. Following, the paper will evaluate the usefulness of
bibliometric methods in gaining an understanding of patterns of
scholarly communication in a chosen discipline, and speculate about the
potential of 'webmetrics' for analyzing Web-based scholarly
communication.
International
Journal of Geographical Information Science (IJGIS) [ISSN:
1365-8816]
Publisher: Taylor & Francis,
London
Journal
web page <http://www.tandf.co.uk/journals/titles/13658816.asp>
Current
annual subscription rates (combined
print and online):
Institutional US$987; Individual US$463
Editor:
Professor Peter F. Fisher, Department of Geography, University
of Leicester
North American Editor: Professor Harvey Miller
Western Pacific Editor: Professor Brian Lees
Book Review Editor: Dr. Nicholas J. Tate
35 Member Editorial Board with broad international representation
IJGIS has been published continuously since 1987. The founding editors were
J. Terry Coppock (Department of Geography, University
of Edinburgh) and K. Eric Anderson (Eastern Mapping Center, U.S.
Geological Survey). It was one of the first
journals
dedicated to exclusive publication of research articles in GIS
(Geographical Information Systems) as it
was emerging as a distinct field of study. Originally named International
Journal of Geographical Information Systems [ISSN 0269-3798],
the journal included research articles, a News Section (to post
information on meetings and developments in GIS), Reviews, and
frequently an Editorial addressing some current aspect of GIS.
The News Section was eventually dropped and the number of editorials
were reduced. However, publication frequency steadily
increased in the first ten years, allowing for a greater
number of
research articles to be printed: Quarterly
(4 issues/year) 1987-1991; Bimonthly (6 issues/year) 1992-1995; and
then 8
issues/year
1996-present. These trends indicated a settling-in of the journal's
purposes, a formalization of the journal's
readership network, and a maturing of the GIS field itself. While more recent
Editorials are intended to "check-in" with readership about trends and
developments or to introduce special issues (often with guest editors),
earlier ones sought to "educate" the readership as to what to expect
from the journal. Dropping the News Section probably reflected the
impracticability of trying to keep-up with happenings in such a diverse
and expanding field (other communication channels being more
efficient). Meanwhile, increasing the number of research articles would
be a clear indication of the maturing of the technical and scientific
basis of the field.
In the very first issue of IJGIS (January 1987), the editors offered an
extensive Editorial review (pp. 3-11), outlining historical
developments in GIS,
and proposing the objectives, intended subject areas, format and
editorial policy of the journal. Their comments included these words:
Geographical information systems (GIS)
represent a rapidly developing field lying at the intersection of many
disciplines--among them cartography, computing, geography,
photogrammetry, remote sensing, statistics, surveying and other
disciplines concerned with handling and analysing spatially-referenced
data. They are also of interest to a wide and increasing range of
users, such as land and resource managers, market researchers,
planners, property assessors and those responsible for utilities, to
say nothing of administrators and policy makers in departments of
government at all levels.…It is this very diversity and communality
that has indicated the need for a journal that will be a source of
information on advances and experiences and a meeting place for
researchers, developers and operators of systems, and users of such
systems, actual and potential.…We intend to take a broad view of the
term GIS and provide a forum for the exchange of ideas, techniques,
approaches and experiences, with a strong emphasis on applications. (p.
1)
[I]t seemed to many of those working in the field that a journal
devoted to GIS was needed. Relevant developments were either not being
reported publicly or were appearing in a wide range of disciplinary
journals not normally seen by many of those interested in GIS… (p. 5)
IJGIS began with a clear
sense of the interdisciplinary
and international nature of
this emerging field of study. A particularly strong emphasis is placed
on the GIS practitioner
and the user. In
1997 the name of the journal was changed to International
Journal of Geographical Information Science. The rationale for the
name change was explained in an Editorial
by Peter Fisher in the January 1997 issue (Volume 11, Issue 1):
Over the 10 years of the publication of
IJGIS the field to which it contributes has evolved considerably, and
particularly the meaning of the 'S'.…Indeed, it is apparent that, in
the 10 volumes of IJGIS to date, articles have contributed not to the
understanding or development of systems as such, but to the science
which underpins and exploits the systems.…Arguably, a natural system is
there to be understood and researched, but an artificial one is built
through the construction of the inter-related bits and is not a matter
for research. Understandings of the issues which underlie the systems,
on the other hand, comes about through research. The consequences of
the whole and the best way to build them are therefore the subject of
research, and more correctly described as science.…Therefore from
Volume 11 the International
Journal for Geographical Information Systems is to be known as the International
Journal for Geographical Information Science.…The intention in renaming
the journal is primarily to reflect better the nature of the material
carried in it, as so to inform better potential readers and reviewers.
(pp. 1-3)
The
name change
reflected a confidence (by the Editor and Editorial Board) that
geographical information systems
had evolved a strong enough
theoretical base to be appreciated as a science unto itself, with a greater
emphasis on research. A happy coincidence
allowed the journal to keep the abbreviation IJGIS unchanged! But who decides when/if a
field of
study has matured sufficiently to claim the coveted mantle of science? On the one hand, it is
observed
that this assessment came lately (after only 10 years of publication)
and by a proponent within the field. Might the interdisciplinary nature
of GIS be creating a perception that
the field of study still lacks a sufficient core, notwithstanding its
own claim to the contrary?1 On the other hand, the field has
clearly matured, and research activity has expanded since IJGIS began
publication in 1987. This is reflected within the journal itself by its
steady increase in the number of issues published yearly (from 4 to 8),
and outside by a proliferation of other GIS journals.2 Too,
appreciation
for greater collaboration across disciplines seems to be increasing
generally in the sciences. Whatever the answer in
absolute terms, the determination will undoubtedly become clearer as
journals like IJGIS continue to publish the results of GIS researchers.3 Indeed, some of the
trends observed above--particularly the increased frequency of
publication, and the accompanying increase in theoretical research--do
suggest what Koehler (2001) (following
the rubric of Derek J. de Solla Price) described for Information
Science, as a
transition in GIS from "small science" to "big science."4
Productivity
Analysis of International
Journal of Geographical Information Science Covering
Years
1998-2002
The following tables summarize findings of a
productivity analysis of IJGIS for the years 1998-2002 utilizing the ISI Web of Science databases.
Table 1:
Document Types and Frequency: Article,
Book Review, Correction, and
Editorial Materials
Year
|
Article
|
Book Review
|
Correction
|
Editorial Materials
|
Total Documents/Year
|
2002
|
40
|
19
|
0
|
1
|
60
|
2001
|
41
|
4
|
0
|
3
|
48
|
2000
|
43
|
14
|
0
|
2
|
59
|
1999
|
36
|
10
|
1
|
3
|
50
|
1998
|
44
|
27
|
0
|
3
|
74
|
5 Year Totals
|
204
|
74
|
1
|
12
|
291
|
Mean (rounded)
|
41
|
15
|
< 1
|
2
|
58
|
IJGIS increased publication to 8 issues/volume/year in 1996. Findings
here reveal a stable editorial format focusing on research articles
(average of 41/year), book reviews (averaging 15/year4), and the occasional
Editorial
(average 2/year).
Table 2: Five
Top Cited Authors/Articles in the years 1998-2002
Top Author(s)
|
Times
Cited
|
Stockwell D, Peters D
The GARP modelling system: problems and solutions to automated spatial
prediction
INT J GEOGR INF SCI 13 (2): 143-158 MAR 1999 |
41
|
Clarke
KC, Gaydos LJ
Loose-coupling a cellular automaton model and GIS: long-term urban
growth prediction for San Francisco and Washington/Baltimore
INT J GEOGR INF SCI 12 (7): 699-714 OCT-NOV 1998 |
30
|
Devogele T, Parent C, Spaccapietra S
On spatial database integration
INT J GEOGR INF SCI 12 (4): 335-352 JUN 1998 |
19
|
Wu FL
SimLand: a prototype to simulate land conversion through the integrated
GIS and CA with AHP-derived transition rules
INT J GEOGR INF SCI 12 (1): 63-82 JAN-FEB 1998 |
19
|
Bishr Y
Overcoming the semantic and other barriers to GIS interoperability
INT J GEOGR INF SCI 12 (4): 299-314 JUN 1998 |
17
|
Table 3: Number of Documents (all
types), and Number of Research
Articles published in IJGIS (1998-2002) by authors named in the
Five Top Cited Authors/Articles above.
Author
|
Documents
|
Articles
|
| Bishr Y |
1
|
1
|
| Clarke KC |
2
|
1
|
| Devogele T |
1
|
1
|
| Gaydos LJ |
1
|
1
|
| Parent C |
1
|
1
|
| Peters D |
1
|
1
|
| Spaccapietra S |
1
|
1
|
| Stockwell D |
1
|
1
|
| Wu FL |
3
|
3
|
Table 4: Citivity and Subjects
(Keywords)
in a sample issue of IJGIS (Volume 16, Issue 8 [December 2002])
#
|
Author(s)
Title
(Page Numbers)
|
Document
Type
|
References
Cited
|
Subjects (Keywords)*
|
1.
|
van Oosterom, P; Maessen,
B; Quak, W
Generic query tool for spatio-temporal data
(pp. 713-748)
|
Article
|
56
|
GIS: error,
accuracy, quality, legislation (data processing)
|
2.
|
Stefanidis, A; Agouris, P;
Georgiadis, C; et al.
Scale- and orientation-invariant scene similarity metrics for image
queries
(pp. 749-772)
|
Article
|
20
|
DATABASES; GIS: methodology (image
analysis)
|
3.
|
van der Poorten, PM;
Jones, CB
Characterisation and generalisation of cartographic lines using
Delaunay triangulation
(pp. 773-794) |
Article
|
18
|
Cartography
|
4.
|
Wu, FL
Calibration of stochastic cellular automata: the application to
rural-urban land conversions
(pp. 795-818) |
Article
|
29
|
SAN-FRANCISCO, MODEL, GIS, SIMULATION,
DYNAMICS, INTEGRATION, EVOLUTION, GROWTH, FORM; Cartography (GIS)
|
5.
|
Zhou, QM; Liu, XJ
Error assessment of grid-based flow routing algorithms used in
hydrological models
(pp. 819-842) |
Article
|
43
|
DIGITAL ELEVATION
MODELS, TOPOGRAPHIC INDEX DISTRIBUTION, DRAINAGE NETWORKS, DIRECTION
ALGORITHMS, REPRESENTATION, EXTRACTION, ACCURACY, TOPMODEL, SLOPE,
AREAS; Meteorology and Climatology (error analysis)
|
6.
|
Mather, PM
Manual of geospatial science and technology
(pp. 843-844) |
Book Review
|
2
|
No keywords given
|
7.
|
Chandler, J
Digital photogrammetry
(pp.
844-846) |
Book Review
|
1
|
No keywords given
|
Research
Articles = 5; Total References Cited = 166; Mean = 27.67
References/Issue;
Median = 29
Book Reviews = 2; Total References Cited = 3; Mean = 1.5; Median = n/a
*Sources
ISI KeyWords Plus, GeoBase descriptors
|
Selection
of a single recent issue (Volume 16(8), December 2002)
confirms the format emphasis (Table 1)
on research articles (5) and book reviews
(2) in virtually the same proportions as the yearly average for the
entire period (41 articles/15 book reviews; 8 issues/year). The average
(mean)
for references cited per article was 27.67 and per book review was 1.5.
Subjects and keywords underscore the interdisciplinary nature of the
GIS field.
Table 5: IJGIS Impact Factor and Immediacy Index
derived from ISI
Journal Citation Reports for IJGIS
(2002)
Impact
Factor 0.821
"The journal impact factor is a measure of the frequency with which the
'average article' in a journal has been cited in a particular year. The
impact factor will help you evaluate a journal's relative importance,
especially when you compare it to others in the same field." |
Immediacy
Index 0.024
"The Immediacy Index is a measure of how quickly the 'average article'
in a journal is cited. The Immediacy Index will tell you how often
articles published in a journal are cited within the same year." |
Given the Impact Factor for the journal as
a whole, approaching 1 cite
per "average article" published in 2000-2001 (0.821), cites from the
five top author(s)/articles in the period 1998-2002 (Table 2) are most
remarkable. For example, the top authors, Stockwell D, Peters D (1999), had 41
cites! These are not "average" articles, marking them as especially
strong with a continuing influence on other research. Indeed, a survey
of the citing titles range
across many
fields of study, and over time from August 1999-February 2004. In
general the top authors (Table 3)
during
this period contributed only one
research article to IJGIS each (with the exception of Wu, FL who contributed 3
articles,
and Clarke, KC who also
contributed a book review).
Table 6: IJGIS Impact Factor and Immediacy
Index Rankings compared to other journals in broad subject categories.
As absolute measures, it strikes me that although the Impact Factor for
IJGIS is pretty good (approaching 1 cite per "average article" in
2000-2001), the Immediacy Index seems quite low (only 1 cite out of
41 published articles in 2002). To get some appreciation for what the
numbers might mean in a comparative context, I conducted a journal
search by Subject Category in
both the JCR Sciences Edition
(2002) and JCR
Social Sciences Edition (2002).
|
Total Journals
|
IJGIS Impact Rank
|
IJGIS Immediacy Rank |
JCR Science Edition (2002)
|
|
Computer Science
(Information Systems)
|
77
|
28th
|
54th
|
|
Geography
(Physical)
|
31
|
21st
|
28th
|
JCR Social Sciences Edition (2002)
|
|
Geography (Human)
|
35
|
21st
|
25th
|
|
Information
Science & Library Science
|
55
|
17th
|
35th
|
These
subject categories are quite broad. Still, as to Impact, IJGIS holds up
pretty good especially in the Information Science & Library Science
(17th of 55) and Computer Science (Information Systems) (28th of 77) subject
categories. Although IJGIS's Immediacy ranking is low (consistent with
the absolute measure), the only really dismal rank appears in the
Physical Geography category (28th of 31). A more accurate picture of IJGIS's
performance would arise by comparing it to other GIS journals
specifically. For example, the results for IJGIS could be compared to
the results gleaned from GIS journals by other students in the course.
The picture is further obscured by the
fact that ISI Web of Science
doesn't index all of the GIS journals, so such information should be
used with caution.
Online access and
services.
The IJGIS
web
page enables non subscribers to browse contents (title,
author with position title, address and email
address, and page numbers) for Volume 10, Issue 1 (February
1996) through Volume 18, Issue 2 (2004), and read article abstracts
(without
references) for Volume 11, Issue 1 (January 1997) through Volume 18,
Issue 2
(March 2004). The web site also includes a list of the
services in which the journal is indexed and abstracted. Persons can subscribe to a free
emailed Table of Contents alerting service (SARA: Scholarly Articles Research Alerting),
and following
a simple
registration procedure, non subscribers can view a sample full text
issue
online.
Online
full text
for Volume 11, Issue 1 (January 1997) through Volume 18, Issue 2
(March 2004)
are available to subscribers directly from Taylor & Francis' web
site. EBSCOhost Academic Search Premier Publications, a third party ejournal
subscription service to which the University of Arizona subscribes, has
full text from Volume 12, Issue 5 (July 1998) through Volume 17 Issue 2
(March
2003). (Due to publisher restrictions there is a 12 month
delay for full text, although article and review abstracts
are available through current issue.)
Keyword Analysis
The use of keywords
can assist in analyzing the foci of a given discipline. I looked at
only a very narrow slice of this through a subject/keyword survey of a
single issue of IJGIS (Table 4).
Do the subjects/keywords appear to fit a discipline, Geographical
Information Science, or the use and application of a tool, Geographical
Information Systems to some problem or area of study? The most
frequently cited subjects are GIS methodology and processing, and
improvement of accuracy and quality. Notwithstanding Peter Fisher's contention
(above) that "articles [in IJGIS] have contributed not
to the
understanding or development of systems as such, but to the science
which underpins and exploits the systems" (italics added), the goal of the research
still appears to be oriented toward the refinement and utilization of
tools for analyzing spatially-referenced
data, which are then applied to some problem or area of study. The
strong emphasis, as at the start, is still on application and user.
Personal Bibliometric Profile of
Fulong Wu
Among the top cited
authors in IJGIS for the five year period I
analyzed (1998-2002) was Fulong
Wu.6 Dr. Wu is currently Senior
Lecturer in Geography at the University of Southampton School of
Geography, Southampton, England. He received his doctorate from
University of Hong Kong in
1995. Previous to his current position at the University of
Southampton, Dr. Wu held positions at The Centre of Urban Planning and
Environmental Management, University
of Hong Kong (1995), and the Department of City and Regional Planning,
University
of Cardiff (Wales) (1996-1998). The focus of much of his research
writing is the process of rapid urbanization in China in the context of
its transition to a globalized and market driven economy.
Two fascinating studies conducted by White, H. D. (2001a, 2001b)
propose that a nuanced bibliometric analysis can give us much more
information about a particular author than simply his/her number of
cites, or how many times he/she is cited by others. The studies were
based on the observed tendency of authors over the course of their
publication career (in their body of work, or oeuvre) to repeatedly cite certain
"orienting figures [as] recurrent points of reference" (2001b, p. 87).
These names represent the core of the author's thought, with other
names scattered with decreasing frequency (many that are never cited
more than once). While most citation analysis is done anonymously (being interested
mainly in the counts), White proposes that linking citers and citees by name gives us, using
quantitative means, insight into an author's intellectual history,
research style, motivations and influences. This set of cited names
ranked in descending order is known as the author's citation identity. Although the
author may not always be conscious of the process, "by the time they
have created an oeuvre, it is
highly likely that their citation identities will be as distinctive as
fingerprints" (2001b. p. 88). Of course, in order to interpret the
significance of this identity, a person needs a knowledge of the
discipline and beyond--researched topics, persons doing the research,
what contribution they have made/are making.
Citation identity is
distinguished from the author's citation
set (what White (2001a) calls the citation image-makers), which is a
list of authors who cite the author, and the citation image, which is a list of
authors who are co-cited with the author. Whereas the author creates
his/her own identity, others create his/her image. As to the social dimension
of intellectual relationships implicit in an author's identity or
image, White (2001a) guesses that "citation is affected by a sense of
intellectual seniority. Stated baldly, there is an apparent tendency to
cite up [junior to senior] or
across [perceived intellectual
peers] the seniority chain much more than down [senior to junior]" (p 625).
For example, the student would be more apt to cite his/her teacher than
the other way around.
These
sets are constructed from data supplied on ISI's Web of Science databases. White
alerts that account must be taken for the inherent limitations of these
databases. For example, there is no coverage prior to 1974, not all the
journals in which an author may have published are indexed by ISI, and
there are "unintentional inconsistencies in citing and data-entry
practices" (e.g., homonyms--same
spelling for different entities, and allonyms--different
spellings for the same entity). White removed co-authored pieces from
his counts, though he admits this may be "over fastidious." I did not
remove co-authored pieces. Additionally, I used "Wu FL" as the search
form for Fulong Wu. I didn't attempt to sort through the more general
form "Wu F" (which returned 765 documents!) though I knew that some
citations for my author were entered using this form (as an allonym).
As it was, I had to exclude homonyms for "Wu FL"--a chemist, a
physicist, and a human physiologist.
White (2001b) adds a further point
about self-citation--where an
author cites him/herself in subsequent writing. Some bibliometricians
would remove self-citations as an indication of an egotistical
tendency.
He is more judicious, suggesting that rather than removing self-cites
from a proportional count beyond the author's control, any charge of
egotism "should perhaps be reserved for failings more clearly personal,
such as citing one's own work when it is irrelevant" (p. 89). I have
chosen to retain self-cites as part of the historical record.
I compiled Fulong Wu's Citation
Identity (authors cited) from ISI
Web of Science using "General
Search." The result was 32
documents (22 sole-authored, 10 co-authored; 26 Articles, 4 Reviews and
2 Editorials), covering the years 1995 to 2004. I removed all
institutional citations (e.g., census data sources).7 I
compiled Dr.
Wu's Citation Set (authors
citing) using "Cited Reference Search." The result was 21 documents (13
sole-authored, 8 co-authored; 20 Articles and 1 Editorial), covering
the years 1995 to 2003. I attempted to compile a Citation Image using a
Dialog Search. However, as my knowledge of Dialog commands was limited,
my search was not successful, and I was unable to generate Dr. Wu's
image. The results follow in Table 7.
Table 7: Fulong Wu's Citation Identity
and Citation Set
|
Citation Identity |
Citation Set
|
Rank
|
Number
|
Name
|
Number
|
Name
|
1
|
21
|
Wu, FL
|
25
|
Wu, FL
|
2
|
18
|
Yeh, AGO
|
9
|
Yeh, AGO
|
3
|
15
|
Harvey, D
|
8
|
Li, SM
|
4
|
12
|
Logan, JR
|
8
|
Lin, GCS
|
5
|
11
|
Bian, YJ
|
5
|
Huang, YQ
|
6
|
11
|
Dowall, DE
|
4
|
Ma, LJC
|
7
|
11
|
Knox, P
|
4
|
Ng, MK
|
8
|
10
|
Wang, YP
|
4
|
Wang, YP
|
9
|
9
|
French, RA
|
4
|
Wu, WP
|
10
|
8
|
Andrusz, GM
|
3
|
Wei, YHD
|
11
|
8
|
Ma, LJC
|
2
|
21 Others |
12
|
8
|
Marcuse, P
|
1
|
119 Others |
13
|
8
|
Nee, V
|
|
|
14
|
8
|
Walder, AG
|
|
|
15
|
7
|
Lin, GCS
|
|
|
16
|
7
|
Lo, CP
|
|
|
17
|
7
|
Short, JR
|
|
|
18
|
7
|
Szelenyi, I
|
|
|
19
|
7
|
Zhou, M
|
|
|
20
|
7
|
Zhu, J
|
|
|
21
|
6
|
8 Others
|
|
|
|
5
|
12 Others
|
|
|
|
4
|
22 Others
|
|
|
|
3
|
42 Others
|
|
|
|
2
|
81 Others
|
|
|
|
1
|
275 Others
|
|
|
Since I am unfamiliar with GIS as a discipline, I am not in a position
to offer any insightful interpretation of the results. I can see,
however, that Dr. Wu is just now completing his first 10 years of
research scholarship. It is reasonable to conclude that he has many
years ahead. Coupled with the findings in Table 2 above, he appears to be
making a mark in the field, especially as his research relates to the
social and economic scene in urbanized China. Frequent self-citation,
rather than hinting at egotism
may reflect subsequent research built upon earlier. Dr. Wu also
co-authored a number of articles with Anthony G. O. Yeh. They
frequently cite one another, though Dr. Wu cites Dr. Yeh twice as
frequently than Dr. Yeh cites Dr. Wu. Dr. Yeh teaches at the University
of Hong
Kong (where Fulong Wu got his doctorate), and based on publication
dates, he appears to be an older contemporary of
Dr. Wu--perhaps his major professor? If so, this would follow White's
guess about the "seniority chain" above.
The Usefulness of Bibliometric Methods
Applied to Scholarly Communication,
and
the Future Potential of 'Webmetrics'
Communication is
implicit in scholarly pursuit because knowledge can only accumulate as
a human resource if it is shared. In scholarly communication, the goal
is not only to add to human knowledge generally, but also to engage in
collaborative relationships to advance knowledge within fields of
study. Scholarly
influence--given and received--is a form of communication. A student
whose thought and career direction has been impacted and shaped by a
respected teacher; friends or colleagues who share a common subject
interest;
an author long deceased whose written words continue to speak in a
powerful way; or even a scholar whose research is vigorously disputed8
can contribute influence. Such communication is not always carried on
in public, but the results of it are frequently observable to all in
the products of a person's scholarship. "Influence…is recorded openly
and unambiguously in the list of bibliographic references affixed to
monographs and journal articles" (Cronin and Atkins, 2000, p. 4). Each
reference (citation) is a sign--a form of shorthand or surrogate--pointing to and making a
connection with a larger context of another's thoughts and ideas. These
relationships can be traced at the personal level, revealing the
origins and development of that particular scholar's thought. But the
lines drawn between individual scholars quickly accumulate as each have
and are impacted by their own spheres of influence. Tracing lines
of reference back and forth in time, or across and between disciplines
can produce a remarkably complex--a seemingly chaotic--picture. But as
the lines are analyzed patterns begin to emerge. Shared
influences, geographical or language clusters, knowledge cores and
boundaries of given subject and discipline, etc. All this, the
intellectual relationships of an individual scholar and the complex
picture of interrelationships created by many scholars, is the domain
of bibliometric analysis. And it all starts with a citation.
Scholarly communication is a social process. Bibliometrics is able to
analyze this social process by quantitative means because citations are
left as artifacts of that process. Artifacts, being real "things,"
can be counted. Still, to appreciate the methods and goals of
bibliometrics, some appreciation must be gained for the social process
of scholarly communication. J. C. Cole (2000) introduces his teacher,
Robert Merton, as "the putative father of the sociology of science,"
who suggested that
sociologists could treat the organization
of
science--its social structure, its reward systems, its norms and
values--as an object of study that would yield results of both
theoretical and practical value--and that science was just as
legitimate an object of sociological study as was the family, the
economy, deviant behavior, religion, and other major social
institutions. (pp. 283-284)
Sara von Ungern-Sternberg (2000) provides
an excellent summary of the relevant theories of Merton, but also
of Goffman's (1964) intriguing "theory of intellectual epidemics,"
which
compares the diffusion of an idea into a scientific population to an
epidemic caused by the spreading of an influenza virus in a population
of people, and "the principle of cumulative advantage" formulated by
Derek de Sallo Price (1976). Price states:
Success seems to breed success. A paper
which has been cited many times is more likely to be cited again than
one which has been cited little. An author of many papers is more
likely to publish again than one who has been less prolific. A journal
which has been frequently consulted for some purpose is more likely to
be turned to again than one of previously infrequent use. (Quoted in
Intellectual epidemics section, ¶ 3)
Of course, the sociology of
scholarly
communication begins with the psychology
of the individual scientist or scholar. Von Ungern-Sternberg, deriving her
thoughts from Garvey (1979) writes:
The psychology of scientific communication
emphasises [sic] the
interaction between the scientist and his environment. Each scientist
brings to each situation a particular cluster of psychological
attributes: personality, skills, style, experience, habits etc., which
in combination with specific circumstances in the research process
gives the individual scientist a predisposition to perceive and detect,
assimilate and associate what is happening with his research at any
given moment. As a consequence, given similar circumstances two
scientists may perceive them differently, assimilate different
information or apply the same information to fulfil individual needs to
advance his particular research. (Quoted in introductory section,
¶ 7)
While I
feel like I am out of my element when it comes to the bibliometric laws
of Lotka, Bradford and Zipf, based on frequency distribution, empirical
bibliometrics, especially those applied to the social and psychological
aspects of scholarly communication are very intriguing to me. In this
regard, I found the recitation
analysis studies of H. D. White (2001a,
2001b) most interesting and insightful. The broad outlines of White's
approach were applied above in the personal bibliometric profile of
Fulong Wu. His key point bears repeating:
Many citation studies feature only cited
authors or works and leave citers anonymous.
Recitation analysis links citers and citees by name, implicating
characteristics of both. [This approach] reveals orienting figures in
an author's thought--other authors who are recurrent points of
reference, taking that term to cover anyone from guru to sworn foe. One
needs domain knowledge to interpret the list of names. If one has that,
recitation analysis can be quite engaging--a source of intelligence
that, unlike much in information science, is not depersonalized. (2001b, p.
87, italics added)
Indeed, although I appreciated that the personal bibliometric profile
conducted above was for demonstration purposes, the most frustrating
part was not being able to interpret the significance of my findings,
because I didn't have domain knowledge.
The other aspect of the bibliometric study conducted above involved
looking at the productivity of the GIS journal, International
Journal of Geographical Information Science. If time would have
allowed, the study could have been expanded to cover the entire
publication life of the journal as suggested in the studies conducted
by Koehler (2001) or Schubert (2002). Koehler was looking for evidence
from a bibliometric analysis of the Journal
of the American Society for
Information Science that the field of information science was
becoming "big science." This was the rubric of Derek de Sallo Price
describing the evolution of science as once inhabited by serious
amateurs (what he called "little science") to a formalized discipline
inhabited by professional "scientists." Koehler nuances the application
of terms, replacing amateur with "practitioner," and then he writes:
Little science in the Twentieth Century is
the arena of both the professional scientist and of the practitioner.
There is not much space for the practitioner in big science. One way to
mark the transition from little science to big science for any given
discipline is to document the change from practitioner to scientist
among its participants. (p. 119)
Koehler
proposed that by looking at the journals of a discipline it is possible
to point to the evolution of the discipline. He selected six variables
to document this evolution--the basis for his bibliometric analysis: 1)
outside government, foundation, or other sponsored research funding, 2)
increases in the number of authors per article, 3) increased content
per issue and volume, 4) changes in author affiliation from
"practitioners" to "scientists," 5) changes in co-authored patterns,
and 6) greater globalization of authorship (p. 121).
I found Koehler's proposal particularly intriguing viewed against the
editorial claim made by Peter Fisher on the occasion of the name change
of the International
Journal of Geographical Information Systems to International
Journal of Geographical Information Science (see above). It
seems that applying Koehler's bibliometric method to the IJGIS would be
one way to test the claim of the editor. If the fact was already
establishable that GIS was (at least) "little science" when Fisher made
his remarks, Koehler's approach could offer evidence that GIS was (at
least) in a transition toward "big science" in that same context.
From bibliometrics
to 'webmetrics.' The growth of electronic publishing and
increasing online dissemination of scholarly research over the Internet
and the World Wide Web suggest a technological shift that may replace
the printed journal as the preeminent conduit for scholarly
communication. The benefits of speed, focus (custom
disaggregation), multimedia, and global access are revolutionary for
scientific and scholarly research. However, some very important
functions that have been performed by the traditional print journal
need to be addressed before such a replacement can be successfully
effected. Von
Ungern-Sternberg (2000) cites Kircz (1998), who described four
functions of scholarly communication that will need to be established
as "measurable units"--the basis for bibliometric analysis--in the
electronic environment:
- The certification function, which
concerns the validation of research quality and has to do with
scientific standards within a research programme.
- The registration function, which
relates particular research to an individual scientist, who then claims
priority for the research. This function is closely connected to
ownership protection, and the reward system, and to a large extent
influences the social dynamics within the system.
- A third
function is the awareness function,
which leads to information needs.
- The last
function is the archival function,
and this function relates to storage and accessibility of information. (Quoted in introductory
section,
¶ 11)
As has been
discussed, citations establish the links of intellectual relationship
in scholarly communication. In traditional print journals, citations
are durable artifacts, and as such, they can serve as a key tool in the
bibliometric analysis of scholarly communication. In adapting
traditional scholarly communication and its analysis to the
Web-environment ('webmetrics') a way to secure the "durability" of
citations is absolutely necessary. The 5th edition of the Publication Manual of the American
Psychological Association requires that Web-based citations
include a retrieval date along with the URL. Behind this practical
requirement is a tacit admission that durability cannot as yet be
guaranteed. The next time the URL is visited, it may be broken, and the
document effectively vanishes. If the link is broken so is the
relationship. Current
webmetrics that analyze data on mouse clicks and visit rates to digital
libraries and document repositories, while useful, do not explicitly
establish the intellectual relationships that are more clearly
established through citation analysis. If this hurdle can be overcome,
and a project like NEC's CiteSeer <http://citeseer.ist.psu.edu/cis>
which provides "autonomous citation indexing" suggests that a solution
may be coming soon.9 The future potential seems great.
Notes
1
The suggestion that GIS may still lack a
sufficient core of theoretical research surfaced for me in a brief
conversation I had with Professor Anita Coleman on March 8, 2004.
2
The
growth within fields and subfields
like GIS contributing to the overall increase in research article
publication follows the observation made by Meadows, J (2000).
3 I am thinking here, too, of the distinction made by
Stephen Cole (2000) between "core" and "frontier" knowledge. Cole writes: "The core
consists of a small set of theories, analytical techniques, and facts
that represent the given at any particular point in time [based on a]
substantial consensus....The other component of knowledge, the research
frontier, consists of all the work currently being produced by all
active researchers in a given discipline. The research frontier is
where all new knowledge is created....The research frontier is linked
to the core through the evaluation process [i.e., peer reviewed
journals like IJGIS]....[O]nly a tiny handful [of research articles]
pass through the evaluation system from frontier to the core" (pp.
109-110).
4
The methodology described by Koehler (2001) could be applied to IJGIS
to quantitatively test this suggestion.
5
Excepting
1998 with a high of 27 book
reviews, and 2001 with a low of 4. Was the exceptionally high year due
to a burst
in GIS monograph publishing? Was the exceptionally low year due to limitations of
journal space,
or a dearth of monograph publishing?
6
Although
Dr. Wu was not the top-cited
author he was the most prolific--contributing 3 articles to IJGIS in
the period 1998-2002 (Table 3).
7 When counting for the citation identity, it is important
to count each unique author only once per citing article. When counting
for the citation set (and citation image), it is important to count
each unique citing article only once.
8 Cole, J. R. (2000, p. 293) refers to the work by economic
historians
Robert Fogel and Stanley Engerman, "Time of the cross" (1974), which
received large numbers of "negative" (critical) citations but which
(ironically) became the catalyst for many major works on slavery in the
antebellum South.
9 It is worth noting, however, that when James Pringle, vice
president for development at Thomson ISI (who is collaborating with NEC
on CiteSeer), was asked whether Thomson ISI planned to archive any of
the "often-evanescent research on the Web," he indicated that "the
company doesn't see archiving as its role. 'Our role is navigation,
building access tools'" (Quint, 2004).
Reference List
Borgman, C. L.
(2000). Scholarly communication and bibliometrics revisited. In B.
Cronin & H. B. Atkins (Eds.) The
web of knowledge: A
festschrift in honor of Eugene Garfield [ASIS Monograph Series]
(pp. 143-162).
Medford, NJ: Information Today.
Cole,
J. R. (2000). A short history of the use of citations as
a measure of the impact of scientific and scholarly work. In B.
Cronin & H. B. Atkins (Eds.) The
web of knowledge: A
festschrift in honor of Eugene Garfield [ASIS Monograph Series]
(pp. 281-300).
Medford, NJ: Information Today.
Cole, S. (2000). The role of journals in
the growth of scientific knowledge. In B.
Cronin & H. B. Atkins (Eds.) The
web of knowledge: A
festschrift in honor of Eugene Garfield [ASIS Monograph Series]
(pp. 109-142).
Medford, NJ: Information Today.
Coppock, T. and Anderson E. (1987). Editorial review. International
Journal of Geographical Information Systems, 1(1), pp. 3-11.
Cronin, B. and Atkins, H. B. (2000). Introduction: The scholar's spoor.
In B.
Cronin & H. B. Atkins (Eds.) The
web of knowledge: A
festschrift in honor of Eugene Garfield [ASIS Monograph Series]
(pp. 1-7).
Medford, NJ: Information Today.
Fisher, P. (1997). Editorial. International
Journal of Geographical Information Science, 11(1), pp. 1-3.
Koehler, W. (2001). Information science as "Little Science": The
implications of a bibliometric analysis of the Journal of the American Society for
Information Science. Scientometrics,
51(1), pp. 117-132.
Meadows,
J. (2000). The
growth of journal literature: A historical perspective. In B.
Cronin & H. B. Atkins (Eds.) The
web of knowledge: A
festschrift in honor of Eugene Garfield [ASIS Monograph Series]
(pp. 87-107).
Medford, NJ: Information Today.
Quint, B. (2004, March 1). Thomson ISI to Track Web-Based Scholarship
with NEC’s CiteSeer. Information
Today, Inc. Retrieved May 7, 2004, from http://www.infotoday.com/newsbreaks/nb040301-1.shtml.
Schubert, A. (2002). The web of scientometrics: A statistical overview
of the first 50 volumes of the journal. Scientometrics, 53(1), pp. 3-20.
von
Ungern-Sternberg, S. (2000). Scientific
communication and bibliometrics. Retrieved May 6, 2004, from http://www.abo.fi/~sungern/comm00.htm.
White, H. D. (2001a). Author-centered bibliometrics through CAMEOs:
Characterizations automatically made and edited online. Scientometrics, 51(3), pp. 607-637.
White, H. D. (2001b). Authos as citers over time. Journal of the American Society for
Information Science and Technology, 52(2), pp. 87-108.