Schonlau, M., Fricker, R.D. and Elliott, M.N. Conducting research surveys via e-mail and the Web. Santa Monica, CA: Rand, 2002. xxiv, 118p. ISBN 0-8330-3110-4 $22.00
The use of the Web to conduct surveys is increasing in popularity as a research method: however, one has the impression, from those received, that they are being used without the research always being fully informed about either sampling, questionnaire design, or, indeed, human:computer interaction. The relatively informal mode of information access provided by the Web seems to occasion a similar informality to the use of the tool. E-mail surveys also often lack proper attention to design factors and rarely is one informed, when asked to participate in such a survey, how one has been selected.
Consequently, this little book (which is also freely available on the Web) is timely and very useful indeed. The summary lists the three main questions which the book seeks to answer:
Following a short introduction, Chapter 2 reviews the survey process: this is a useful reminder of the basic facts about the conduct of survey and will act as an aide memoire for those who may need to brush up on their Research Methods 001 course. In particular, the section on sources of error should be read and memorized!
Chapter 3 reviews the literature on Internet surveys and there is a second literature review on response rates in Appendix A. One point that emerges that those thinking of using an Internet survey is conveyed in the following:
In conclusion, when considering only postage and printing costs, e-mail and Web surveys appear to be cheaper than postal mail surveys. In actuality, it appears that Web surveys become more economical... only when the number of responses reaches a certain threshold—somewhere between a few hundred and a thousand.
One study suggested that the cut-off point is 347 responses - so, if one is expecting only one or two hundred responses, the evidence suggests that a mail survey will do better. The authors also not that even given the economics, a research who is unfamiliar with Internet surveys could find all advantages wiped out by 'unanticipated technical problems'.
Chapters 4 and 5 deal with the 'meat' of the problems of Internet surveys, covering the choice of approach and guidelines for developing Web and e-mail surveys. The decision on approach comes down to a choice of sampling methods: if one is satisfied with a convenience sample (that is, 'Any sample in which the probability of a sample member's inclusion cannot be computed...') the amount of work involved is quite limited, whereas if one needs to be able to compute accurate error rates of the kinds discussed in Chapter 2, more time-consuming ways of selecting samples have to be found, and, consequently, the survey becomes more costly to undertake. Chapter 5 offers very good guidance on the actual design of surveys, with clear, numbered points that constitute an excellent checklist.
Six surveys are presented as case studies in Chapter 6 and these offer illustrations of the points raised in the earlier chapters. Care is taken to demonstrate where the use of the Web brings savings, and where the use of postal questionnaires can bring improvements to response rates.
The final Chapter concludes that Internet surveys are here to stay, but that there are some serious problems: first, it is difficult to use such surveys for the general population because of the lack of sampling frames and the inability randomly to construct e-mail addresses (as it is possible to do with telephone numbers); secondly, if a probability sample is essential, considerable work must be put into developing the relevant sampling frame; thirdly, the conduct of the survey is more technically complex than a paper questionnaire sent by mail; and fourthly, little is known about the relevant design principles for Web-based surveys—the principles of design on paper may apply but this is not certain.
In addition to the appendix on response rates already mentioned, there is a summary table of evidence from the literature, and a study of the effectiveness of a convenience sample to supplement a probability sample.
There is an extensive bibliography but, sadly, no index. However, this book is essential reading for anyone thinking of undertaking an Internet-based survey and I strongly recommend that you buy the book, rather than using the free Web version. You'll want this as a desk tool throughout the design and preparation process.
Professor T.D. Wilson