Electronic Resources for Research Methods

Research methods

• 2H i Helenelund HB. (1999)   Personality tests on the WWW. Sollentuna, Sweden: 2H i Helenelund HB.

  Part of a general site on various kinds of tests. The tests are to be taken on-line and it is unlikely that you would be able to employ them in a questionnaire - either mail or on-line. However, they do give pointers to sources and this may be of value.

• American Association for Public Opinion Research. (2007). Question wording.
Retrieved 5th August, 2008 from http://www.aapor.org/questionwording . (Archived by WebCite^® at http://www.webcitation.org/5ZqNG0vy1)

"The general principle of question wording is that every respondent should understand the question and be able to answer it with reliability – that is, if she were asked the same question again, she would give the same answer. So question wordings are evaluated in terms of whether they can provide reliable information. A number of common problems have been identified, as well as solutions for dealing with them"

• Borgatti, Stephen P. (1998)  Principles of questionnaire construction.  Harvard, MA: Analytic Technologies.

  Bullet-point notes only - but a useful outline.

• Centre for Applied Social Surveys. (2000)  The question bank. London: Economic and Social Research Council.

  "Question Bank (Qb) is a store of complete social survey questionnaires from large-scale policy relevant UK studies."

• DSS Research. (2000)  Questionnaire design: basic considerations. Arlington, TX: DSS Research.

  "To design a good questionnaire, a number of considerations must be kept in mind: Does it provide the necessary decision-making information for management and does it consider the respondent."

• Farrall, Stephen, Bannister, Jon, Ditton, Jason & Gilchrist, Elizabeth (1997)  Open and closed question [sic] Social Research Update, Issue 17.

  "Research on the fear of crime has grown substantially in recent years. From its inception, this field has relied almost exclusively upon quantitative surveys, which have suggested that the fear of crime is a prevalent social problem. However, doubts about the nature of the instruments used to investigate this phenomenon, and in particular the use of ‘closed’ questions, have raised the possibility that the fear of crime has been significantly misrepresented. This Update suggests that our understanding of the fear of crime is a product of the way it has been researched rather than the way it is."

• Frary, Robert B. (n.d.)   A brief guide to questionnaire development. College Park, MD: Ericae.net

  "...this booklet is intended to identify some of the more prevalent problems in questionnaire development and to suggest ways of avoiding them. This paper does not cover the development of inventories designed to measure psychological constructs, which would require a deeper discussion of psychometric theory than is feasible here. Instead, the focus will be on questionnaires designed to collect factual information and opinions."

• Galloway, Alison (1997) Questionnaire design and analysis: a workbook Edinburgh: Kate Galloway.

  A useful, basic text on the subject.[Note: This site is currently having problems and gives only a 'holding' page.]

• Galloway, Alison ()  Questionnaire design and analysis activities. Edinburgh: Kate Galloway.

  Paper and pencil exercises and tests. [Note: This site is currently having problems and gives only a 'holding' page.]

• Heriot-Watt University. Learning Technology Dissemination Initiative (1999)  Questionnaires.  Edinburgh: Heriot-Watt University. Learning Technology Dissemination Initiative

  Part of the LTDI 'Evaluation Cookbook' - links to pages with brief notes and/or further links.

• Israel, Glenn D. & Moore, Jon (n.d.)  Coding techniques. Gainesville, FL: University of Florida, Cooperative Extension Office.

  Now that you have a pile of completed questionnaires or xeroxed copies of data from several books (or both) stacked on your desk, you ask yourself, "What do I do with this stuff?" The analysis of the data that you collect for planning, implementing, or evaluating programs can be facilitated by a clear structuring of the data through the use of a coding scheme. The development of a coding scheme is simply a "blueprint" for transferring data from questionnaires or other sources of data to tally sheets or computer files. The following illustrates a coding scheme that can be used for both a survey instrument and secondary data.

• O'Brien, David (n.d.)  Questionnaire design. Atlanta, GA: Georgia Institute of Technology, College of Computing.

  "Questionnaires are an inexpensive way to gather data from a potentially large number of respondents. Often they are the only feasible way to reach a number of reviewers large enough to allow statistically analysis of the results. A well-designed questionnaire that is used effectively can gather information on both the overall performance of the test system as well as information on specific components of the system. If the questionnaire includes demographic questions on the participants, they can be used to correlate performance and satisfaction with the test system among different groups of users."

• Punj, Girish (1997)  Questionnaire design. Storrs, CT: University of Connecticut, School of Business Administration.

  Presentation in .pdf format for a course: "The objective of this course is to assist you in gaining an in depth exposure to the marketing research process, as applicable to a large business organization. You will be expected to acquire competence in designing high quality marketing research projects, with particular emphasis on problem definition, questionnaire design, sampling and data analysis."

• Rosinski, Larry (1998)  Pre-testing foreign language questionnaires.  The Research Report, Vol.11, No. 3 [Produced by Maritz Marketing Research Inc.]

  "Too often... questionnaire translations are merely accepted without any modifications. At other times, the native language questionnaire is proven through “back-translation.” A better way exists. Increasingly, I am preaching the value of a survey pre-test as a tool for ensuring that we are measuring what we think we are measuring."

• Scantron Corporation. (1995)  Choosing the right scale.  Bloomington, MN: NCS Pearson, Inc. Retrieved 5th August, 2008 from http://survey.scantron.com/resources/planning/scale.htm (Archived by WebCite^® at http://www.webcitation.org/5ZqIlRFHO)

  "Choosing a scale for your survey instrument is an important decision that will shape the information you collect. In the research field, there is much discussion as to how many points and what kinds of scale labels (anchors) comprise the most effective measurement tool. Each scale has variations, some more reliable than others. This newsletter discusses pros and cons of the scales most commonly used for mail surveys, and the three scales which perform the best according to current research."

• Scantron Corporation. (1996)  Response scales: how many points and what labels? Bloomington, MN: NCS Pearson, Inc. Retrieved 5th August, 2008 from http://survey.scantron.com/resources/planning/response-scales.htm (Archived by WebCite^® at http://www.webcitation.org/5ZqJCi6pi)

  "Choosing a scale for your survey instrument is an important decision that will shape the information you collect. In the research field, there is much discussion as to how many points and what kinds of scale labels (anchors) comprise the most effective measurement tool. Each scale has variations, some more reliable than others. This newsletter discusses pros and cons of the scales most commonly used for mail surveys, and the three scales which perform the best according to current research."

• Smith, Tom W. (1993)  Little things matter: a sampler of how differences in questionnaire format can affect survey responses. Chicago, IL: General Social Survey. (GSS Methodological Report No. 78) Retrieved 5th August, 2008 from http://www.amstat.org/sections/SRMS/proceedings/papers/1995_182.pdf (Archived by WebCite^® at http://www.webcitation.org/5ZqK4uS9n)

  "It is well known that seemingly minor changes in question wording, response format, and context can appreciably alter response distributions. What is less appreciated is that non-verbal aspects of surveys such as physical layout and visual presentations can also notably influence answers."

• StatPac Inc. (2000)  Designing surveys and questionnaires. Minneapolis, MN: StatPac Inc. Retrieved 5th August, 2008 from http://www.statpac.com/surveys/ (Archived by WebCite^® at http://www.webcitation.org/5ZqKZXAHk)

  "This tutorial will teach you how to design a questionnaire and conduct a survey. You'll learn the latest survey research techniques...what works and what doesn't. You'll discover the secrets used to maximize survey response rates, and how to design a questionnaire that gets at the true opinions of your sample. "

• Summerhill, W.R. & Taylor, C.L. (1992)  Writing questions for mail questionnaires. Gainesville, FL: University of Florida, Florida Cooperative Extension Service.

  "Questions are the basic components of a questionnaire. They are designed to collect specific pieces of information related to the general research question. The quantity and quality of information collected really depends on the quality of specific questions included in the questionnaires. If the questions are not well written, you may not get what you want, or the information may be insufficient to answer your research question. The information collected will be irrelevant and your effort, money, and time will be wasted. Well written questions should be relevant and contribute to your study. The structure of the questions should be suitable for the kinds of information sought and the choice of words should be precise to maximize the validity of data collected."

• Summerhill, W.R. & Taylor, C.L. (1992)  Obtaining response to a mail questionnaire.  Gainesville, FL: University of Florida, Florida Cooperative Extension Service.

  Facilitating response is a vital step in the process of information collection. After all, the larger the response, the more information collected. According to research by Galpin..., Berdie, et al...., Blau..., and Dillman..., there are several factors that influence rate of response. Understanding the effect of these factors will not only increase response rate but will also make the program more cost effective. The following is an introduction to understanding these factors and some recommendations on increasing your response rate.

• Taylor, C.L. & Summerhill, W.R. (1992)  Formatting a mail questionnaire. Gainesville, FL: University of Florida, Florida Cooperative Extension Service.

  "The format of a questionnaire is its physical arrangement of questions on the page. It concerns its general appearance and is vital to a successful study. A well-planned and carefully constructed questionnaire will increase the response rate and will greatly facilitate the summarization and analysis of the collected data.... Some researchers believe that the questionnaire format frequently determines whether it is read or discarded. Once the respondent takes the effort to read it, he or she has some psychological commitment to completing it...."

• Taylor, C.L. & Summerhill, W.R. (1992)  Writing options for mail questionnaires. Gainesville, FL: University of Florida, Florida Cooperative Extension Service.

  "For all except open-ended questions, you need to provide options (possible answers) for respondents to choose from. Well-written options are more likely to elicit reliable and accurate responses and provide the information you want. Confusing options lead to unreliable results and, usually, low response rates. The following suggestions should help you in your practice.
  Generally speaking, depending upon what you are measuring, there are two types of options you can choose from. There are options that are points along a scale and there are options that are independent alternatives."

• U.S. Census Bureau. (2000)  Demographic surveys. Washington, DC: U.S. Census Bureau.

  Page devoted to surveys carried out by the Bureau, the links take you to specific survey sites, where you will find the questionnaires, as well as data products and methodological notes.

• U.S. Federal Committee on Statistical Methodology. Subcommittee on Questionnaire Design. (1983)  Statistical Policy Working Paper 10 - Approaches to developing questionnaires.   Washington, DC: Federal Committee on Statistical Methodology.

  "The Subcommittee on Questionnaire Design was formed by the Federal Committee on Statistical Methodology to address the general topic of questionnaire design. The Subcommittee focused on a review of methods used in developing questionnaires. The working paper discusses approaches to devising questionnaires in three broad areas: tools for developing questions, procedures for testing the questionnaire draft, and techniques for evaluating the questionnaire.
  While the report is intended primarily to be useful to Federal agencies that develop questionnaires, a broader audience may also find the report of interest."

• University of Leeds. Information Systems Services.Guide to the design of questionnaires. Leeds: University of Leeds, University Computing Services.

  "This document is based on a document written by Max Hunt in Computing Services, University of Loughborough... This document is designed for students and researchers intending to process a survey. The notes on construction of questions are written as an aid to logic and efficiency whether or not computer processing is required. The document provides a compilation of do's and don'ts of questionnaire design which if obeyed will make computational work routine and fast."

• University of North Carolina. Howard W. Odum Institute for Research in Social Science (n.d.)   Public opinion poll question database.  Chapel Hill, NC: Howard W. Odum Institute for Research in Social Science.

  "...the Odum Institute's Public Opinion Poll Question Database allows any researcher to search for specific poll questions among the more than 100,000 questions in the Institute's archive by key words, date, study number, study title, or state. The system displays the full question text, study information, and frequency distributions, so it is useful both to users interested in looking up previous question wordings to develop questions for their own studies, and to users interested in frequencies or in locating particular variables for statistical analysis."

Related software

Most of the software packages listed here are general survey management packages, but all include modules for questionnaire development. Many are intended for market research or company human relations applications but are adaptable for general research purposes.

• Absolute Survey Software.  Survey Said. De Pere, WI: Absolute Survey Software.

  "The following Survey Said 1, 2, and 3 packages contain the core programs for creating a survey, entering answers from a paper survey and analyzing the results. Optional add-on modules...[exist]. "

• American Statistical Association. Research Methods Section.  Summary of survey analysis software.  Alexandria, VA: American Statistical Association.

  "This page is a summary of available software for the analysis of surveys with complex sample designs. Specifically, it includes software that can do variance estimation with such survey data. It does not attempt coverage of the many excellent packages which do not have the particular features required to analyse such data. Information can be viewed by package or comparatively by topic."

• Apian Software Inc.  Survey Pro. Seattle, WA: Apian Software Inc.

  Includes questionnaire design as well as analysis and presentation.

• Cambridge Software Publishing  KeyPoint 5.5 Cambridge: Cambridge Software Publishing.

  "Design your own surveys by following a few easy steps – you’re guided through the process from start to finish.
  Run your surveys face-to-face, over the telephone or by post – with KeyPoint you can even use e-mail or put your survey on a website for people to fill in online.
  Analyse and cross-reference the results.
  Present your findings as charts, tables or graphs."

• Change Systems Pty. Ltd.  SurveyMaster - survey design & analysis software.  South Brisbane, Queensland: Change Systems Pty. Ltd.

  General survey research package with Web publishing feature.

• Creative Research Systems   The Survey System. Petaluma, CA: Creative Research Systems.

  "The Survey System is the most complete software package available for working with questionnaires. The Survey System handles all phases of survey projects, from creating questionnaires through data entry, interviewing or Web surveys to producing tables, graphics and text reports that make you look good. Unlike spreadsheets, databases or general purpose statistics packages, The Survey System was designed specifically for questionnaires; so The Survey System saves you time."

• Infopoll Inc.  InfoPoll Survey Software Dartmouth, NS: Infopoll Inc.

  "Infopoll is the only survey software that offers complete solutions to your survey needs from designing questionnaires to delivering complete survey analysis reports. Best of all, the Infopoll Designer is free and Infopoll Server is available for license."

• Mercator Computer Systems Ltd.  snap survey software. Bristol: Mercator Computer Systems Ltd.

  "integrated survey software with on-screen questionnaire design, data entry, analysis as tables and charts."  A variety of products available, including an Internet module for publishing questionnaires on the Web.

• Raosoft, Inc.  EZSurvey  Seattle, WA: Raosoft, Inc.

  "Raosoft® EZSurvey for the Internet is powerful software to help you gather survey responses over a computer network. You can quickly write a form and broadcast it over your company email system and post it on a personal web server. With an email survey, your colleagues can reply to your email message, making their answers directly on the email form you sent. With a web server, you can let anyone use your form on the Internet, then view their responses on your own computer. Your data is compiled into EZSurvey's database. And with the EZSurvey Professional version you can also collect your data in ODBC databases."

• Senecio Software Inc.  MaCATI Bowling Green, OH: Senecio Software Inc.

  "A sophisticated, easy-to-use set of software solutions for telephone (CATI), personal interviewing (CAPI & e-CASI), kiosk, disk-by-mail (DMS) and World Wide Web (ePoll) surveying."

• SumQuest Survey Software.  SumQuest Version 7.0. Toronto, ONT: SumQuest Survey Software.

  "SumQuest Version 7.0 is capable of conducting Internet, e-mail, mail, telephone, and personal interview surveys. It can process discrete, multiple choice, ranking, continuous and open-ended (text) questions. Users summarize responses using frequency distributions, crosstabulations, graphs, and the CellSearch report. The CellSearch feature eliminates the job of searching through thousands of crosstabulation cells by automatically reporting any significant differences in attitude across a demographic. Users can immediately determine on which issue males differ significantly from females, young from middle-aged, and so on. All SumQuest output can be either viewed on a computer screen or printed in presentation-quality format." Downloadable demo available.

• Touch Base Computing  Quik-Poll. Rome, GA: Touch Base Computing.

  "Introducing Quik-Poll for Windows for only $149.00 (US),the easy-to-use survey software system you can use
  to design and tabulate hundreds of customer surveys in-house!" Downloadable trial version.

• Training Technologies Inc.  SurveyTracker software. Mason, OH: Training Technologies Inc.

  "SurveyTracker helps you design a successful survey quickly and easily. Add an overview, questions, notes, sections, and a summary with a click of the mouse! The Edit Window makes creating and altering text simple. The Edit Window for questions provides convenient shortcuts when adding scales.
  Survey layout is automated with electronic surveys and streamlined for paper surveys! "

• U.S. Department of Health and Human Services.Centers for Disease Control and Prevention   Epi Info 2002  Washington, DC: Department of Health and Human Services.

  "With Epi Info 2002 and a personal computer, epidemiologists and other public health and medical professionals can rapidly develop a questionnaire or form, customize the data entry process, and enter and analyze data. Epidemiologic statistics, tables, graphs, and maps are produced with simple commands such as READ, FREQ, LIST, TABLES, GRAPH, and MAP. Epi Map 2002 displays geographic maps with data from Epi Info 2002." Downloadable, public domain software.

Search Google for more sites