This paper reports the results of a survey of the students from the Department of Information Studies, University of Sheffield who successfully completed research degrees (MA, MSc, MPhil or PhD) in librarianship or information science in the period 1968-96. An open-ended questionnaire was distributed to the 123 former students for whom a contact address was available, and usable responses were received from 66 students, giving a response rate of 54%. The questionnaire sought to identify publications derived directly and indirectly from the research programme, and the students' views of the benefits and drawbacks that accrue from having carried out their research programme. A total of 393 publications were identified as arising directly from the research degrees, with these attracting at least 1593 citations, thus demonstrating the impact of research-degree work on the profession. Students predominantly take up employment in the academic sector, and find their research training invaluable in their subsequent career. The great majority (71%) of the respondents would recommend such training to others; the main problem identified is the lack of a long-term career structure for those wishing to enter the UK higher education system.
For many years, research played only a minor rôle in the activities of vocationally oriented academic departments, such as those of librarianship and information science (hereafter LIS). However, the introduction of performance indicators such as the Research Assessment Exercises (RAEs) in the United Kingdom (Elkin and Law, 1994; Oppenheim, 1995) have meant that the quality of a department's research plays an increasingly important function in its ability to attract funding and, in consequence, able students and staff. Much research, in LIS as elsewhere, is carried out by postgraduate students working for research degrees under the supervision of a full-time member of the teaching faculty. There has been a steady flow of publications reviewing aspects of dissertations (Repp and Galviano, 1987; Rutledge, 1994), some, indeed, based on dissertations themselves (Boyer, 1973), but there appears to have been little effort to quantify the effectiveness of the research training that is provided and of the theses or dissertations that provide the primary, and in many cases only, output of research degrees.
The Department of Information Studies at the University of Sheffield (hereafter the Department) has attached particular importance to research ever since its foundation (Saunders, 1964) and in this paper we report the results of a survey of the 137 students who have successfully completed a research degree between 1968, when the Department awarded its first such degree, and mid-1996, when the survey was carried out. In all, there have been 89 PhD theses, 39 MA or MSc by research theses, and 9 MPhil theses, as detailed in Table 1. The figures in this table demonstrate the increasing importance of research over the period under review, with 30 theses being submitted during 1968-77, 38 during 1978-87, and 69 from 1988 to mid-1996.
| Year | PhD | MA/MSc | Year | PhD | MA/MSc | Year | PhD | MA/MSc | MPhil |
|---|---|---|---|---|---|---|---|---|---|
| 1968 | - | 1 | 1978 | 1 | 1 | 1988 | 7 | - | 1 |
| 1969 | - | - | 1979 | 1 | 1 | 1989 | 6 | - | - |
| 1970 | - | - | 1980 | 1 | 5 | 1990 | 5 | - | 1 |
| 1971 | 1 | - | 1981 | 2 | 1 | 1991 | 6 | 1 | 1 |
| 1972 | - | 4 | 1982 | 4 | 2 | 1992 | 10 | - | 2 |
| 1973 | - | 7 | 1983 | 3 | 2 | 1993 | 9 | - | 2 |
| 1974 | - | 4 | 1984 | 1 | 1 | 1994 | 5 | - | 1 |
| 1975 | 1 | 5 | 1985 | - | - | 1995 | 9 | 9 | 9 |
| 1976 | 1 | 1 | 1986 | 5 | 1 | 1996 | 2 | - | 1 |
| 1977 | 3 | 2 | 1987 | 6 | - |
This paper reports the results of a survey of these 137 former MA, MSc, MPhil and PhD students (hereafter referred to simply as students). The study had two objectives: firstly, to identify publications derived from the research programmes, thus providing information about the value of the research to the LIS community as a whole; secondly, to probe the students' views of the benefits and drawbacks that accrue from having carried out their research programme, thus providing information about the value of the research to the individual student. Full details of the project are presented by Santos (1996).
Contact information was available for many of the students from Departmental records or from members of staff who had remained in touch with their former students, with additional information being obtained from reference sources such as the IFLA Directory, the ALA Directory, the World of Learning, and Web homepages. In all, contact information was obtained for 123 students and a questionnaire was sent to them either by e-mail (43 of the students) or by post (80 students), with follow-up letters and e-mails being sent where necessary to maximise the response. The questionnaire requested details of all publications and other research outputs resulting directly or indirectly from their degrees, together with a series of open-ended questions that enabled respondents to express freely their views and experiences of doing research in the Department.
Completed questionnaires were returned by 66 of the 123 former students for whom a contact address was available, giving a response rate of 54% of the sampling frame (and of 48% of the total student body). The distribution of students, as between full-time, part-time or a combination of both, and as between UK and overseas, is shown in Table 2. The 23 overseas students came from 13 different countries: Argentina (1); Australia (3); Botswana (1); Canada (4); China (1); Finland (2); India (3); Mexico (1); Netherlands (1); Portugal (1); Singapore (1); USA (2); and Zambia (2).
| UK | Overseas | Total | |
|---|---|---|---|
| Full-time | 23 | 18 | 41 |
| Part-time | 14 | 1 | 15 |
| Combination | 6 | 4 | 10 |
| Total | 43 | 23 | 66 |
As a result of the responses to the questionnaires and of discussions with current members of the Department's teaching staff who previously either had been research students themselves and/or had supervised research degrees, it was possible to produce a list of publications derived directly from the research degrees. Here, the term 'publication' was defined as including books, book chapters, reports, journal articles and published conference proceedings, with other, minor research outputs, such as book reviews or unpublished conference contributions, being excluded from further analysis.
In all, 393 publications (in addition to the original research theses) were identified as resulting from 120 research projects. Note that the latter figure is much greater than the number of responses to the questionnaire because the supervisors were able to provide publication data for many students from whom a response had not been obtained. Of these 393 publications, 88 (22%) were books, book chapters or reports, 259 (66%) were journal articles, and 46 (12%) were contained in conference proceedings. Within this total, just 4 projects had not produced any publications derived from their thesis work, 47 projects had produced 1 publication, 23 projects had produced 2 publications, 9 projects had produced 3 publications, 8 projects had produced 4 publications, and no less than 29 projects had produced 5 or more publications. On average, each thesis produced a mean of 3.28 publications.
Each of the books, book chapters, reports and journal articles was searched online in the DIALOG implementations of the Science Citation Index, the Social Science Citation Index, and the Arts and Humanities Citation Index. By far the largest number of citations was found in the Social Science Citation Index, in agreement with the findings of Cronin and Overfelt (1994) in a citation-based evaluation of academic programmes and individual faculty members in the School of Library and Information Studies at the University of Indiana in the USA.
The citation searches were initially effected using the "expand cited reference" command, e.g., one of the journal article searches was of the form ECR=CHOWDHURY GG, 1994, V13, P13?. However, the large number of searches that needed to be carried out meant that this had to be replaced by the "select cited reference" command, e.g., SCR=CHOWDHURY GG, 1994, V13, P13?
The difference between the two search strategies is that use of the ECR command ensures the retrieval of all citations to each publication by covering possible spelling errors-spellings, name changes, and variations in the first author, whereas the SCR command does not. Accordingly, the searches may well have missed some spelling errors, etc., with the result that the citation counts obtained should be viewed as lower bounds on the actual number of citations accruing to each of the publications that were searched. Citations to books were also searched using the SCR command with the author name and some title words, e.g., SCR=DYKSTRA M, 1985? AND PRECIS/TI.
The sheer number of searches necessary meant that it was not possible to include the conference proceeding publications in the citation analysis. Even without these, the searches identified a total of 1593 citations, including self-citations.
Analysis of the citation search outputs revealed 51 students who obtained no citations; 26 students who obtained between 1 and 10 citations; 15 students received between 11 and 20 citations; 10 students between 21 and 30; and 18 students who received more than 31 citations. In all, then, 120 research students who have produced 393 publications have attracted a total of 1593 citations, representing mean values of 13.28 citations per student and 4.05 citations per publication. For comparison, an earlier study that focused upon citations to publications by Departmental faculty members (Bradley et al., 1992) found that 641 publications attracted a total of 2065 citations, an average of 3.22 citations per publication. It should be noted that there is some degree of overlap between the two sets of figures, in that the Departmental supervisor (a member of the faculty) was included as an author in many of the degree-based publications, and three of the students are currently faculty members. Even so, it seems not unreasonable to conclude that degree-based publications have an impact (as measured by citation counts) that is at least comparable with that of faculty-based publications.
The citation searches demonstrate clearly that publications resulting from the Department's research degrees have had a considerable impact on the literature as a whole, supporting the findings of the earlier study by Bradley et al. (1992). The number of citations is, however, strongly dependent upon the subject area in which the research was carried out. There are, currently, three main research groups within the Department. The Computational Information Systems Research Group (hereafter CISRG) focuses on the development of novel techniques for the representation and searching of databases of biological, chemical and textual information, and also has interests in expert systems, multimedia applications and neural networks. The Information Management Research Group (hereafter IMRG) focuses on the rôle of information in business performance, the management of information needs, information seeking behaviour, health information management and Internet-based, computer-mediated communication in education, training and organisational development. Finally, the Library Management and Public Policy Research Group (hereafter LMPPRG) focuses on issues concerned with public library management, the role of libraries in the community, and library management in general. The students, and their accompanying publications, were categorised according to which of these three areas best fitted their research topic. This was simple for the more recent students, who would have been associated with one of the three research groups during their project, but more problematic for at least some of the older projects that had taken place many years prior to the establishment of these groups within the Department. The results of this categorisation are shown in Table 3.
| Research group | Number of students | Number of publications | Number of citations | Mean number of publications per student | Mean number of citations per publication |
|---|---|---|---|---|---|
| CISRG | 84 | 247 | 1352 | 2.94 | 5.47 |
| IMRG | 15 | 71 | 226 | 4.73 | 3.18 |
| LMPPRG | 21 | 77 | 15 | 3.67 | 0.19 |
The Department has a long-standing, international reputation for its research on computational methods for the storage and retrieval of chemical and textual information retrieval (see, e.g., Lynch and Willett, 1987). It is thus hardly surprising that this area of research is very well represented in terms of numbers of students, with 84 out of the 120 being allocated to the CISRG (i.e., the students were members of the CISRG or would have been members if the group had been in formal existence when their research was carried out). The other two areas of research are comparable in size with each other but have noticeably fewer students associated with each of them. than in the case of the CISRG The numbers of students naturally have an effect on the numbers of publications, albeit not a directly proportional one since there are differences in the mean number of papers per student for the three groups, with the IMRG students being noticeably more productive than the other two groups of students (as shown by the figures in the fifth column of Table 3). However, the differences between the three research areas are far greater when the citation data are considered. Here, there are very many CISRG citations, this arising both from the large number of publications and the high citation rate per publication. The IMRG publications are much less cited than the CISRG publications, but the difference here is minor when compared with the minuscule number of citations accruing to each of the LMPPRG publications; specifically, the final column of Table 3 shows the three areas have mean citation rates in the ratio 28.8:16.7:1. The high citation rates for publications in the more science-based aspects of LIS has been noticed previously (Ban Seng and Willett, 1995).
While there is considerable discussion as to the merits of citation analysis as an indicator of research performance (Baird and Oppenheim, 1994; Cronin, 1984; Liu, 1993; MacRoberts and MacRoberts, 1989), it has been extensively used in the context of the teaching faculty in LIS departments (Anderson et al., 1978; Craghill and Wilson, 1987; Cronin and Overfelt, 1994). Here, we have applied it to a departments' research students and demonstrated, for the students studied here at least, that their work can attract at least as wide a degree of acknowledgement in the open literature.
In addition to publications, 14 (9 UK and 5 overseas) of the 66 respondents (21%) were aware that their thesis work had resulted, either directly or indirectly (e.g., from development work by a subsequent researcher), in other forms of dissemination, most obviously CAL packages or computer software. Examples of these research outputs included chemical information software that was subsequently distributed by a US commercial software company, chemical information software that was used by a consortium of German pharmaceutical and agrochemical companies, library automation software used at Newcastle Polytechnic (now University of Northumbria at Newcastle), and a kit on services to people with disabilities.
The second part of the questionnaire contained a series of open-ended questions that sought to information about the careers followed by students on completion of their degrees, the extent to which these careers had been helped or hindered by their research, and their views on the overall value of an LIS research degree.
A summary of responses to the questions "What is the job title for your first job on completing your research degree?" and "What is your current job title?" resulted in the figures shown in Table 4, where 'Other' denotes an unclassified response, 'retired', or a blank response.The figures in the table encompass a very wide range of types of position: in all no less than 45 different job titles were quoted by the 66 respondents in their answers to these two questions. The most common single title was 'post-doctoral research assistant' (or some variant thereof) for 19 first positions (15 UK and 4 overseas students) and 8 current positions (all UK students), followed by 'retired' for 7 current positions (4 UK and 3 overseas). In these days of continuing high unemployment, it is gratifying to note that 'Unemployed' was mentioned by only one overseas student.
| Type of position | First position | Current position |
|---|---|---|
| Library | 9 | 5 |
| Industry | 8 | 17 |
| Academic | 47 | 33 |
| Other | 2 | 11 |
The most striking point about the table is the huge number of students who go into (71%), and remain in (50%), academic employment. The progression from a higher degree via a post-doctoral research post to a faculty position is common in academic disciplines, but one might not have expected to find such a marked tendency in an inherently vocational subject such as LIS. These figures thus provide some support for Stoker's suggestion that LIS faculty now have better academic qualifications than used to be the case (Stoker, 1996), although he was referring specifically to the situation in the UK. Many of the first positions are, inevitably, quite junior but some of the current-position titles demonstrate the ability of the students to reach positions of authority, with responses including 'co-ordinator of university library services' (one overseas student), 'university librarian' (one overseas student), 'head of computer services' (one UK student), 'managing director' (two overseas students) and 'professor' (one UK and three overseas students).
| Value of research degree... | Little | Fair | Average | Good | Excellent | Blank |
|---|---|---|---|---|---|---|
| ...in obtaining a post | 4 | 2 | 2 | 17 | 38 | 3 |
| ...in carrying out the responsibilities of that post | 1 | 5 | 5 | 21 | 31 | 3 |
When the students were asked "To what extent have the posts you have worked in been research related?", 45 responded 'highly related', 11 responded 'partly related', 6 responded 'not related' and 4 did not respond. In view of this strong focus (with 85% of the posts being highly or partly research-related), it is unsurprising to find a very positive response (as demonstrated in Table 5) to the question "To what extent do you think that your research experience was of value in obtaining a post and in carrying out the responsibilities of that post?"
The 'Good' and 'Excellent' responses to these two questions total 83% (obtaining) and 79% (carrying out) of the sample, demonstrating the substantial career benefits that result from a research degree.
The successful execution of a higher degree requires a substantial investment by the student, in terms of time, money and intellectual effort, inter alia, and it was thus of interest to ascertain the benefits that they perceived as resulting from this investment. Specifically, they were asked: "Please list any benefits (professional, social, personal and financial) that you found during and subsequent to your research project."
Of these, the professional aspects were noted by most students, with three closely linked areas being identified as of particular importance: the opportunity to gain professional contacts, improved promotion prospects, and increasing academic responsibility. The opportunities to make professional contact were noted by both the UK"The reputation of the research group meant that I came into contact with quite a few significant people in the field"
"I was able to meet many people involved in the pharmaceutical industry and other related fields. These contacts have proved very useful in my subsequent work"
and the overseas students
"To build a very wide network of contacts that are still now very valuable at professional and personal levels"
"Because of Sheffield's international student body, my experience brought me international contacts and international perspectives"
"Contacts with researchers in the UK and internationally and in professional associations were beneficial in developing subsequent contacts"
Their research project was of use to some in achieving promotion
"It provides part of the basis for promotion"
"Qualification was an advantage in obtaining promotion"
although such benefits were mentioned much less by the mature students who had already reached senior positions in their organisations. The increased academic responsibility and reputation is evidenced by statements such as:
"Improved academic credentials: of some advantage in a highly competitive academic-job market, and in the ability to attract funding, and have publications and conference papers accepted"
"People know me. It is a great 'ego trip' to be asked for advice by someone in Washington or Sydney!"
Financial benefits, in the shape of improved career prospects, were also mentioned by many of the students
"The level at which I was appointed was influenced by the very fact that I had a PhD which involved a fair amount of research"
"Salary is higher and have more job opportunities, since there are few people with PhD degrees"
"Helped me gain a position as lecturer"
as were the social benefits.
"Sheffield was a great place to spend three years and I felt I developed socially and personally"
"The biggest benefit I found was the chance to get to know well a variety of other PhD students from overseas. I have kept in contact with many and regard them amongst my closest friends"
The social benefits that they obtained were of particular importance to the overseas students:
"It enabled me to interact with academics from a range of disciplines and get to know other students outside my normal working environment"
"Good cultural and social learning experience"
"Living in England and studying at Sheffield was a most rewarding personal, social and cultural experience. I learnt to deal with lots of different people, to cope with unexpected situations and to understand a variety of people's problems"
The final area, personal benefits, covered a range of positive responses, including:
"My research experience was excellent from the point of my own personal development and has opened up a new career for me"The students were also asked to, "Please list any negative aspects that your found during and subsequent to your research project" Several of the comments here related to the general problem of isolation:
"I benefited socially and personally by having more freedom than I imagine I would have in a company environment"
"Personal satisfaction from completing a major challenge in life"
"Wider perspective, better knowledge, more exposure"
"Turned me on to a whole new way of thinking and observing"
"It gave personal confidence in one's abilities"
"It gave me an excellent training to express my ideas clearly and consistently"
"Working in isolation without pressures of deadlines can have a negative effect"
"Felt quite isolated during research project- only one doing research in this area"
while overload was identified as a problem by several of the part-time students:
"The strain of completing the degree whilst working in my first lectureship"
"The reading and writing time required in evenings and at weekends for five years"
"Total exhaustion"
One of the mature students noted that: "Ageism was a big problem and it still is for more mature candidates to attract funding. This is an issue which should be urgently addressed" but the single most common response to the question related to the financial rewards that people may have to forego if they decide, as so many of these students have, to take up an academic career. The higher education sector in the UK has severe funding problems, in particular the decreasing number of permanent positions and the deteriorating salary levels. These problems are well exemplified by the many comments from UK students who have stayed in the university system, often in fixed-term post-doctoral research positions (which, as noted previously, formed the largest single type of employment in the sample):
"No career prospects in university-based research"
"I could probably make more money elsewhere - I make less here than I did before. All the work is on short-term contracts"
"If I had not become an academic I imagine my salary would be at least twice what it is now"
"Research tends to be in short term contracts - one to three years - and is therefore rather insecure"
Comments such as these show all too clearly the effects on morale of the changes in university funding that have taken place over the last 15 years in the UK. A related point was made by one of the overseas students, who said, "A PhD costs a lot and the salary levels of the Third World will never compensate the investment"
The final question asked, "Would you recommend the experience to someone in a comparable position to you when you commenced your research? Please give reasons for your answer."
The great majority, both UK and overseas, responded positively to this question: 47 (71%) would recommend the experience, 7 (11%) would not, 8 (12%) were unsure and the remaining 4 students (6%) did not respond. Some of the many positive comments were:
"Absolutely. A research degree gives excellent training in the process of research and in the presentation of results both orally and in writing"
"Constantly stimulating, always new challenges"
"Opportunity for valuable intellectual development"
"This gave me an opportunity to stand back from the day-to-day library problems and to reflect on the wide picture"
"There is no better way to achieve an awareness of alternative perspectives, and the power to critique one's own thoughts and actions as an information practitioner"
"As the job market gets tougher, any extra qualifications and experience really count. Research posts really train you to work independently, to develop a certain amount of lateral thinking; to keep going in the face of adversity and also to be goal-oriented in one's approach."
However, several of those who responded positively also noted the commitment required to succeed
"I would always recommend completing full-time in three years. Also, it is much better to complete a PhD before having family responsibilities. The strain of job, family and PhD is too much."or the financial problems that students might encounter:
"I would recommend the experience of a PhD with several warnings. A PhD requires a lot of money, hard work, endurance, creativity and luck. It's a difficult but possible combination"
"Yes, but only if the research is funded (probably industrially) sufficient to pay a Research Assistantship rather than just a research grant; especially when compared with former, better funded times"
"I suspect that research students are now a lot worse off than I was, since we had considerable opportunity to enhance our grant income by demonstration fees which may no longer be available"
"Yes - but only if they are convinced they want to do a PhD: the situation now is so different in financial terms that the two situations are not really comparable"
The responses from those who would not recommend the experience, or who were unsure, covered several areas:
"Research is so self-motivating that I don't think I'd recommend it to anyone. However at least here you finish on time, the work is interesting and relevant and no one seems to have any trouble getting a job"
"Depends on the personality of the person: need to be able to motivate themselves to do the work, and not get too stressed with workload"
"It is not a light undertaking and requires a lot of perseverance, self-motivation, and interest to keep at it and complete it over a period of three years. Three years is a long time to be working on one single highly specific topic!"
but the long-term financial prospects were again commented on by several of the UK students:
"There is no career structure for research scientists in an academic environment -everything is done on fixed-length contracts"
"Probably not - depends on what they want to do in the longer term - no career prospects, how old they are, how interested they are in the topic, and whether or not they will be paid, while doing it"
"As a 'practising' researcher and academic I am earning far less than I would be as a 'practising' library manager"
One of the two negative responses from overseas came from a student who stated unambiguously that: "I don't think [it] is suitable for students from the Third World countries. Different situation, especially economic and political situation, will be impractical for them"
This study has analysed the publications and citations resulting from theses produced by former research degree students in the Department of Information Studies, University of Sheffield for the period 1968-96, and the students' perceptions of the advantages and disadvantages of undertaking their research programmes.
We have shown that these degrees result in a high level of publication, with journal articles being the most common mode of dissemination, as well as additional types of research output for 21% of the sample. The 393 publications known to have resulted from Departmental research degrees have attracted at least 1593 citations; we say "at least" since, as noted previously, time constraints meant that the conference publications were not searched and that little account was given to possible variations in the spelling of author names. Publications in the more science-based aspects of LIS attract noticeably more citations than do publications in 'mainstream' information management, while publications in librarianship are very poorly cited in the literature, as represented by the coverage of the citation indices.
The responses to the various open-ended questions show that the majority (71%) of the students take up an academic first post, and that a large number of them stay there (with 50% having a current academic post). The research experience obtained in the Department has been of value to 81% of the students in obtaining a post and to 79% of them in carrying out the responsibilities of that post. Most students obtained substantial professional, financial, social and personal benefits from carrying out their higher degrees, but financial problems were noted as a potential disadvantage by many, in particular the limited career prospects available to those UK students who wished to stay in academe on completion of their degrees. Overall, however, the advantages are far greater than the disadvantages, with 71% of this body of students being willing to recommend the experience to others in a comparable position. The questionnaire had a response rate of 54%, and this must always be borne in mind when considering the generally positive findings of the study; however, we believe that it is not inappropriate to end with a comment from one of the overseas students, who stated that:
"Research is a struggle. It was then, and it continues to be for me, but it is a joyous struggle."
We thank Knight Ridder Information Services for providing access to the online versions of the citations indices, the 66 former research students who replied to our requests for information, and our Departmental colleagues for providing some of the publication data.
All correspondence regarding this paper should be addressed to Professor Peter Willett.
| Mary Dykstra Lynch and T.D. Wilson | ©British Library Board 1997 |