The Impact of Doctoral Research in Information Science and Librarianship

Outstanding Impact of Citations: Two Case Studies

As mentioned earlier, two of the fifteen award-holders who form the sample population of this project stand out as having most impressive records of citations. Although the publications of both have been cited extensively, the citations differ in interesting ways as to subject matter, range and pattern of the citing, and audiences upon whom their research has impacted.

The first case is that of David Ellis, currently Senior Lecturer in the Department of Information Studies, Sheffield University. Dr. Ellis was appointed as Lecturer in the Sheffield department in 1984 and was already publishing in his research area by the time his Ph.D. was completed there in 1987. As noted, he has indicated that twenty of his forty-seven publications to date are directly related to his doctoral research. Accordingly, these twenty publications (nineteen of which he is the sole author) form the basis of the citation search conducted for this project.

The first citation of one of D. Ellis' publications was by an author in Germany in 1985 (two years before Ph.D. completion). Two Canadians cited him in 1986 and in 1988. The numbers have steadily increased each year, to a high of twenty-seven citations in 1996 by authors in Denmark, the United States (Indiana, North Texas and UCLA), Germany, Italy and the UK.

The twenty publications Dr. Ellis has indicated as being directly relevant to his doctoral research (all but one of which he is the sole author) have been cited a total of 151 times. These include 112 citations by other authors and thirty-nine self-citations. Seven of the citations are in fact reviews of his book, New Horizons in Information Retrieval (London: Library Association, 1990). One publication, "A Behavioural Approach to Information Retrieval System Design, " (Journal of Documentation, v. 45, 1989, p. 171-212), has been cited thirty-nine times between 1991 and 1996. An overall citation count by year is given in Chart 1.

The majority who cite Ellis' publications are scholars in LIS departments, many in the United States. All but a few of the citations appear in LIS journals. An analysis of his citations by geographical distribution is given in Chart 2.

On the chart, the 33% within the US include citations by colleagues at Berkeley, Drexel (with collaboration from the department of Psychology at Princeton), Indiana, Illinois, North Texas, Tennessee, Rutgers, Syracuse, UCLA, Washington and Wisconsin. Canadian LIS departments include McGill, Toronto and Western Ontario. Within the UK he has been cited by authors at Queen's University in Belfast, the City University, De Montfort, the University of London, Huddersfield, Sheffield, and Wales. Citiations from the EU countries other than the UK include scholars in Denmark (two who cite him extensively), Germany, Italy, the Netherlands and Spain. The "other" category on the chart includes the countries of Australia, Japan, New Zealand, Nigeria and South Africa.

A yearly analysis of the Ellis citations by journal is as follows:

It should be noted that Dr. Ellis was cited so often in JASIS in 1996 that the page numbers of the citing articles run almost continuously -- from 23-36 to 37-49; then 287-300, 311-325, and 333-335; then 493-503, 504-518, and 603-609. Nearly the same number of citing articles appear in the Journal of Documentation in 1992.

All of the citations to Dr. Ellis' publications have been found in Social Scisearch; some have appeared additionally in the databases of Scisearch and Arts & Humanities Search.

This analysis of David Ellis' publications and their citations shows him to be a productive scholar of international stature. The impact of his doctoral research can be measured quite clearly and directly by the very consistent publication of his work in the major refereed journals of librarianship and information studies, together with the steadily growing number of citations to these works appearing from around the world. In fact it could be said without exaggeration that, in the world today, there is no LIS scholar active in the area of information retrieval research who does not know of Dr. Ellis' work.

Equally impressive in its world-wide impact, though in quite different ways, is the doctoral research of another Sheffield graduate, Helen Grindley, who completed her Ph.D. in the Department of Information Studies five years later in 1992. The differences between the two award-holders already appear quite clearly at the point of their career paths: Dr. Grindley currently holds the post of Academic Support Officer in the Sheffield department - a position which she notes has "little or no" relevance to her doctoral research.

The second difference between the two lies in their subject areas. H. Grindley's doctoral thesis, entitled "Automated Searching of the Protein Data Bank for 3-D Structural Motifs Using Techniques from Graph Theory," places her research within the scientific area of structural molecular biology as well as the more technical areas of information science and computer science.

From this basic difference in subject area spring a whole host of other differences, including the way her publications are authored, the journals in which these publications appear, the types of journals in which the publications are cited, where the citations appear in the ISI databases, the citation pattern, etc.

As noted earlier, Dr. Grindley lists thirteen publications relevant to her doctoral research. Of these she appears as sole author of none and first author of six. However, the subject area in which she has written is one which has been built up over the years between two departments at Sheffield (Molecular Biology and Biotechnology, and Information Studies), attracting a diverse number of research students and staff who most often produce the results of their research as joint publications. Within this rather large and variable group the practice when authoring is to list the names in alphabetical order, regardless of the size of contribution. As it happens, one who has written often with the group has a surname beginning with the letter "A," the result of which is that seven of H. Grindley's thirteen jointly-authored publications list this author first. In nearly every case the number of authors is large - there are eleven authors for one publication!

Given the size of this scientific group and their mode of authoring, the concept of "self citation" for Dr. Grindley becomes very difficult to define. Part of the difficulty lies in the fact that the joint authors of her papers are not all colleagues in the Department of Information Studies. On the contrary, the group often includes members of the Department of Molecular Biology and Biotechnology and the Krebs Institute of Biomolecular Research at Sheffield, and researchers at other UK universities as well as the Free University of Amsterdam. If some members of the group cite others, is this "self citation"? What happens when H. Grindley cites publications by the group where her name does not appear, or when other members of the group cite H. Grindley?

In the end it was determined for this project that self-citations for H. Grindley occur when she, writing as first author, cites another publication of which she is first author. Not surprisingly, this rather narrow definition results in a small number of self citations. Authors within such a large and productive group, however, do tend to cite other publications of the group's work quite often. Therefore, although the number of actual self-citations is limited in this case, the group's citations of their own work swell the numbers of citing authors within the UK (even though at times the group includes authors from other countries, e.g., the Netherlands, citing articles with a majority of UK authors have been counted as UK in origin).

Searches in the ISI databases have yielded 103 Grindley citations in the period between 1992 and 1996. The citations appear exclusively in Scisearch; limiting the search to Social Scisearch would result in their being missed altogether. Of the 103 citations, four are self citations according to the criterion adopted above and fifty-seven are by UK authors. The geographical distribution of Dr. Grindley's citations is shown in Chart 3.

The geographical distribution is especially impressive given the fact that several of the citing publications are themselves multi-authored. Besides Grindley and her various groups who cite each other, citations have been made by other researchers in a wide variety of institutions in the UK, the rest of Europe (Belgium, France, Germany, Italy, the Netherlands and Spain), the US, Asia, and other parts of the world including the Czech Republic (written in Czech), Israel, Lithuania and Russia. Some of the citations have collaborative authors from different countries, e.g., France (University of Paris) and the US (Johns Hopkins University), France and the US (the National Institute of Health), the National Cancer Institute in the US and the University of Tel Aviv (Faculty of Medicine and School of Mathematical Science) in Israel, Israel and the UK (Cambridge University). Citations are also by collaborative authors within countries, e.g. Germany (University of Heidelberg and Humboldt University, Berlin; universities of Bonn and Dusseldorf) and the UK (Oxford University Molecular Biophysics Laboratory and the University of East Anglia School of Biological Science; National Institute of Medical Research and the University of London Departments of Molecular Biology and of Biochemistry and Molecular Biology).

Those in academia who cite Dr. Grindley in her various groupings are, in contrast to those who have cited Dr. Ellis, almost entirely in departments other than LIS. A partial list showing the range of departments included are Biochemistry, Biochemistry & Molecular Biology, Biological Science, Chemistry & Biochemistry (Harvard University, the University of Delaware and the University of Kyoto, Japan), Crystallography, Information & Computer Science (University of California at Irvine) Mathematical Science (University of Tel Aviv) and Physics (University of Oregon). The group has also been cited by scientists at the Academy of Science of the Czech Republic and the Russian Academy of Science.

Citers of Grindley et al. include authors from a diverse range of research laboratories and institutes including the National Cancer Institute and National Institute of Health (Bethesda, MD), the Imperial Cancer Research Fund (UK), the Strangeways Research Laboratory, the Molecular Biology Laboratory and the Cambridge Crystallography Data Centre (all three at Cambridge University, UK), the European Molecular Biology Laboratory (Heidelberg, Germany) and the Vilnius Biotechnol Fermentas Institute (Lithuania). One citing group represents three research institutions: the Max Plank Institute of Biochemistry in Germany and, in the US, the Institute of Molecular Biology in Oregon and the Research Institute of Molecular and Cellular Biology in Palo Alto, CA.

Other authors who cite Grindley et al. represent the impact of her work on various pharmaceutical companies, etc. and their laboratories, e.g. Abbott Laboratories (UK), Pfizer Ltd. (UK), Proteus Molecular Design Ltd. (UK), Merck Sharp & Dohme Research Laboratories (US) and Warner Lambert Parke Davis (US). The varied list of corporate citations includes Barnard Chemical Information Ltd. of Sheffield.

A yearly analysis of the Grindley citations by journal is as follows:

Certainly it could be said that the impact of the research of this productive group of largely Sheffield-based scientists, revealed by the impressive and far-ranging citations of their published work, goes beyond the impact of Grindley's doctoral research alone. It should be noted, however, that of all the publications of this group which include Grindley as author, one does not follow the usual practice of listing the names in alphabetical order. In this case Grindley appears clearly as principal author. This article (Grindley, H.M., Artymiuk, P.J., Rice, D.W., Willett, P., "Identification of secondary structure resemblance in proteins using a maximal common subgraph isomorphism algorithm," Journal of Molecular Biology, v. 229, 1993, p. 707-721), has in fact been cited far more often than any of the others to which she contributed - forty-three times in the four years from 1993. The geographical range of the forty-three citations includes the UK (twenty-three citations including the universities of Bath, Cambridge, Durham, London and Sheffield, the Imperial Cancer Research Fund, Pfizer Ltd. and Proteus Molecular Design Ltd.), the US (six citations including the University of Oregon, Johns Hopkins University, the National Cancer Institute, the National Library of Medicine, and Warner Lambert Parke Davis), Germany (three citations including the universities of Bonn and Dusseldorf and the European Molecular Biology Laboratory), Israel (three citations from the University of Tel Aviv), Japan (three citations from the universities of Kobe and Kyoto), Belgium, Italy, the Netherlands, and Spain (two citations from the University of Girona).

Note: Includes an Ellis review article with eleven self citations.