Uncertainty in information seeking

a research project in the Department of Information Studies

University of Sheffield
Professor Tom Wilson, Dr. David Ellis, and Nigel Ford
Research Officer, Allen Foster

Introduction: the research issues

Research into information searching in IR systems and research into information behaviour more generally have pursued separate lines over the past twenty-five years or more. While IR research has focused upon the interaction of the individual user with the system and has judged the user’s and the system’s "success" in terms of the discovery of relevant documents, the work in information behaviour has pursued qualitative approaches (particularly more recently) in an attempt to understand the diversity of information behaviour and to disentangle the multiplicity of factors that affect that behaviour.

Aim

The aim of this project (funded by the British Library Research and Innovation Centre) is to explore information search behaviour within a more general framework of information behaviour based upon a model of the problem-solving process and upon the concept of uncertainty reduction (Wilson, in preparation). The basic proposition behind this model is that problem resolution proceeds in stages by reducing uncertainty at each stage until a resolution of the problem is reached.

Objectives

Within this overall model of the problem process, this project will address research questions relating to the following:

• evaluating models of information searching in information retrieval (IR) systems;
• testing whether the proposed model of information-searching as related to problem solving (Wilson, in progress) is valid for the population in question;
• establishing whether the use of Kuhlthau’s model of information searching as a stage process fits the suggested model of multiple searches in a problem solving strategy; and
• examining whether Ellis’s behavioural model of the search process is a more appropriate model in the problem-solving context.
• exploring whether the concept of individual differences (e.g., Ford and Ford, 1993) - is valuable in explaining differences in problem solving and searching behaviour in searching; and, in particular,
• determining whether the personal characteristic of tolerance of uncertainty is identifiable and, if so, whether it affects problem-solving behaviour and search behaviour.

IR and relevance

The ambition of developing information retrieval research as a well grounded scientific discipline founded on the quantification of relevance has not been successful. From its inception information retrieval research as a distinguishable sub-discipline within the field of information studies has been plagued by the problem of measurement, and, in particular, with the difficulties encountered in employing derivatives of relevance (recall and precision) as measures of retrieval effectiveness. These difficulties are well documented both in the experimental literature, dating back to the earliest retrieval tests (Sparck Jones, 1981), and in experimental work on variations in relevance assessment (Cuadra and Katter, 1967). The general feeling is that relevance may not be a good, or even particularly reliable, basis for measurement, but, that it is the only one the sub-discipline has. In that, the difficulties in measuring relevance seem of a lower order of complexity than those for example of measuring benefit, utility, reduction of uncertainty, or any of the other candidates for quantitative evaluation

However, if the issue of measurement, or quantification of information retrieval effectiveness is placed on hold, and the question of evaluating the effectiveness of retrieval systems in the context of the tasks or problems, in relation to which they may be employed as means or tools, then the question of how to evaluate the effectiveness of retrieval systems becomes more open. In particular, the concept of relevance becomes less pre-eminent, or salient, within the family of evaluation concepts to which it naturally belongs - such as utility, novelty, importance, interest - or indeed, in relation to the system achieving goals for the user such as resolving uncertainty or raising or supporting doubts. If this different view of the role of information retrieval systems is accepted, then, the archetypal form of retrieval system evaluation based on quantification of relevance is not appropriate and alternative models of retrieval system evaluation need to be explored.

An alternative model

The recent review by Wilson (1996) of the literature on information behaviour from fields other than information science suggests that "stress/coping" theory has much to offer as a theoretical basis for the origins of information seeking. In this theory, stress (which may be of varying strength) provides the motivation for information-seeking, and we suggest that stress is the result of experiencing a problem of some kind.

Problems may range from the immediate, urgent, highly significant (such as a pressing need for advice on a major personal problem) to relatively modest problems, such as the need to identify books and journal articles that may be suitable for a student’s term essay, due in, say six weeks’ time. The amount of stress caused by these problems must vary considerably but, across the range, information-seeking activity may ensue as one strategy to cope with the problem.

The solution of the problem, the resolution of the discrepancy, the advance from uncertainty to certainty (or at least some pragmatic solution of the problem) then becomes a goal of the person and we typify the resulting behaviour as goal-seeking behaviour. We then argue that en route to the goal, the individual moves from uncertainty to increasing certainty and that there are stages in the problem-resolution process that are identifiable and recognisable to the individual. It is recognized, of course, that information may increase uncertainty - the lower feedback loops in the diagram below are intended to suggest this possibility.

A problem-resolution and solution-statement process is suggested here, consisting of the following stages:

These stages may be defined as follows:

Problem recognition: the stage at which the individual recognizes that a problem exists, i.e., recognizes that he/she has a problem and defines it as of a certain type. The question, What kind of a problem is this? may be thought of as typifying this stage. At this stage, the individual experiences a high degree of uncertainty about the problem itself and, potentially, about its boundaries and, almost certainly, about the ways of resolving the problem.

Problem definition: the person now seeks to remove some of the uncertainty surrounding the problem by defining it in terms of its characteristics. The question, What are the dimensions of this problem? may typify this stage. Some of the uncertainty surrounding the problem will be removed by understanding or information gained in this phase.

Problem resolution: this stage is likely to be the main information-seeking phase, in the course of which the person seeks to answer the question, What is the answer to this problem? or, on a more pragmatic level, How do I cope with this problem? At this stage, the person hopes that the remaining uncertainty will be removed. Clearly, however, this may not always be the case and actions other than information-seeking may be involved in a full resolution of the problem.

Solution statement: this term is used to signify any application of the solution found. It may, in fact, consist of a statement or presentation to others of the type, This is the solution to my problem. or it may involve actions to implement the solution.

The important question at this point is, "How is uncertainty resolved?" This is where we can bring in previous attempts at modelling information behaviour and, specifically, those of Kuhlthau (1991, 1993) and Ellis (1989, 1993). Our proposition is that Kuhlthau’s "stages" can be seen not as steps in a single information seeking activity, but reiterated steps that may occur in exploratory loops between each link in the problem resolution chain shown above.

Kuhlthau’s model has the stages initiation, selection, exploration, formulation, collection, and presentation. The model proposed here suggests that some of these terms can be used to identify the stages through which an individual moves to resolve uncertainty. Equally, the work may show that a stage model is inappropriate and that information seekers adopt the kind of behavioural strategies suggested by Ellis - starting, chaining, browsing, differentiating, monitoring and extracting (Ellis, 1989) - that cannot be constrained within a stage model but which are applied in different ways, depending upon the nature of the problem and the searchers’ characteristic behaviours.

The research will also explore the role of individual differences in problem solving and information seeking. There is cumulating evidence that individuals differ in the ways in which they can and do structure information in learning and problem solving contexts (Ford, 1995). The most salient difference, in terms of volume of empirical research evidence as well as applicability to the present project, is that between analytic and holistic information processing styles (Ford, 1995; Riding and Sadler-Smith, 1992; Witkin, Moore, Goodenough and Cox, 1977). This difference has been described as one of the two fundamental dimensions of cognitive style (Riding and Cheema, 1991).

Throughout the research the team will be collaborating with Dr. Amanda Spink of the University of North Texas whose research into feedback in information retrieval (Spink & Losee, 1996) and other aspects of information searching (e.g., Spink & Goodrum, 1996) is of direct relevance to the project. [Dr Spink is now in the School of Information Sciences and Technology at Pennsylvania State University]

Significance of the research

While the Project described here is seen as a contribution to basic research in the fields of information searching and information retrieval, a number of practical implications are likely to arise, both for the designers of IR systems and for information practitioners. Specifically, the research will give rise to a closer understanding of the information searching process, which should throw light on the ways in which existing IR systems should be redesigned to take account of and facilitate the multiple search process. If the problem solving model is supported by the research, there are implications for the underlying rationale of IR systems, which today support a paradigm based on searching for word matches, whereas a more appropriate paradigm may be one based on an interactive process of eliciting successive problem statements as the searcher moves through a multiple search process.

From the point of view of the information practitioner, a project that demonstrates the validity of a problem solving approach to information searching will enable the development of training strategies for information intermediaries

Preliminary results

The Project was completed at the end of September 1999 and analysis of the data is in progress on both sides of the Atlantic. Some preliminary results were reported in a seminar for the doctoral programme at the Swedish School of Librarianship and Information Science at the Högskolan i Borås.

Previous research

Professor Tom Wilson

Professor Wilson has carried out a number of investigations into information use and information-seeking behaviour, from (among others) the INISS Project, a study of communication and information behaviour in social services departments in 1975-1980, through Project LOGI, a study of information needs in local government in 1981-82, the Business Information Needs investigation of 1985-86, and the study of the Business Impact of Information Systems (1993-1994), to his most recent work reviewing information behaviour research in information science (Wilson, 1995) and information behaviour research in fields other than information science (Wilson, 1997). Professor Wilson pioneered the use of qualitative methods in information science in the INISS Project and has supervised a number of Ph.D. theses that have adopted such methods in a diverse range of settings.

Dr. David Ellis

Dr. Ellis has extensive research experience in the areas of IR evaluation and user modelling of information seeking behaviour and for IR user modelling. He has an international reputation for the application of qualitative research techniques in relation to IR applications. He has undertaken a number of related studies into the information seeking patterns of academics in the sciences, social sciences and humanities as well as with researchers in the industrial environment of an international petroleum company.

Mr. Nigel Ford

Mr. Ford has extensive research experience in the field of individual differences and information processing. With Dr. Ellis, he directed an Department of Employment research project investigating the effects of individual differences on information seeking in hypertext databases. He also co-directed a British Library project examining individual differences in searching behaviour within the context of student-centred learning. He has published extensively in the area of individual differences in the journal literature.

References

Cuadra, C. A. and Katter, R. V. "Opening the black box of relevance" (1968) Journal of Documentation 23, 291-303.

Ellis, D. (1989) "A behavioural approach to information retrieval design." Journal of Documentation, 46, 318-338

Ellis, D., Cox, D. and Hall, K. (1993) "A comparison of the information seeking patterns of researchers in the physical and social sciences." Journal of Documentation, 49, 356-369

Ford, N. (1995) "Levels and types of mediation in instructional systems: an individual differences approach." International Journal of Human-Computer Studies, 43, 1995, 241-259.

Ford, N. and Ford, R. (1993) "Towards a cognitive theory of information accessing: an empirical study." Information Processing and Management, 29 (5), 569-585.

Kuhlthau, C.C. (1991) "Inside the search process: information seeking from the user’s perspective." Journal of the American Society for Information Science, 42, 361-371.

Kuhlthau, C.C. (1994) Seeking meaning: a process approach to library and information services. Norwood, N.J.: Ablex Publishing.

Riding, R. and Cheema, I. (1991) "Cognitive styles - an overview and integration." Educational Psychology, 11 (3-4), 193-215.

Riding, R. and Sadler-Smith, E. (1992) "Type of instructional material, cognitive style and learning performance." Educational Studies, 18 (3), 323-340.

Sparck Jones, K. (1981) "Retrieval system tests 1958-1968" In Sparck Jones K. (ed.) Information Retrieval Experiment. pp. 213-255. London, Butterworth.

Spink, A. and Losee, R.M. (1996) "Feedback in information retrieval," in: Annual Review of Information Science and Technology, 31, 33-78

Spink, A. and Goodrum, A. (1996) "A study of search intermediary working notes: implications for IR system design." Information Processing and Management, 32, 681-695.

Wilson, T.D. (1997) "Information behaviour: an interdisciplinary approach." Information Processing and Management, 33, 551-572   [Original British Library RIC Report available]

Wilson, T.D. (1999) "Models in information behaviour research". Journal of Documentation, 55, 249-270.

Witkin, H., Moore, C.A., Goodenough, D.R. and Cox, P.W. (1977) "Field-dependent and field-independent cognitive styles and their educational implications." Review of Educational Research, 47, 1-64.

Web pages designed and maintained by Professor Tom Wilson. Last update 5th March 2000
WebCounter visitor number    to this page.