Research Methods

COMPARISON OF ADVANTAGES AND DISADVANTAGES: Mail-Out Census versus Telephone Survey of Random Sample

Mail-Out Census: In this type of approach, a written questionnaire is sent to all the households in a community. The survey “package” may or may not include a stamped, self-addressed return envelope. The questionnaire requires respondents to follow directions and answer each question in a specific manner (circling a letter, checking a box, etc.). After a specific period of time, all residents are mailed a reminder postcard. Results are based upon the number of returned surveys at the pre-announced “cut-off” date.

Telephone Random Sample: In this type of approach, a random sample of households is selected to be interviewed by telephone. All households have an equal chance of being selected and the adult respondent in each household is also chosen randomly. Questionnaires are administered over the telephone by trained and supervised interviewers. Telephone numbers are tried at least six times during a one-week period and an additional six time a second week before another household is substituted for the original selection. Results are based upon a pre-set number of completed interviews.

Mail-Out Census Telephone Survey
Type of Sample Convenience sample, based upon who responds to the mailing Random sample, including pre-designated target households and substitutions
Representativeness Residents with more intense feelings likely to respond, potentially biasing the results Random samples can be projected statistically to the entire population within specific limits of accuracy
Probable Response Rate 10%-40% 90%+
Accuracy of Results Questionable; low response rate makes distortion by “extremists” more likely Statistically significant; e.g., 400 respondents provide results within ± 5.0% of the entire population 95% of time
Fieldwork Time 4-6 weeks 2 weeks
Cost Low-to-moderate, depending on size of questionnaire, size of the community, and number of reminders Moderate, depending on size of questionnaire and size of sample

Focus Groups: In this type of approach, potential participants are screened and invited to participate in a discussion section at a specific time and location. The facilitator engages the group in a discussion based upon a scripted series of open-ended questions. The proceedings can be videotaped, electronically recorded, or transcribed by a note-taker. If there are a series of focus group discussions taking place, the common discussion guide will facilitate later comparisons between groups. In order to ensure an acceptable participation rate among targeted individuals, financial incentives are usually offered and refreshments are served at the session.

Telephone Random Sample: In this type of approach, a random sample of individuals is selected to be interviewed by telephone. All individuals have an equal chance of being selected and interviewed. Questionnaires are administered over the telephone by trained and supervised interviewers. Telephone numbers are tried at least six times during a one-week period, before another household is substituted for the original selection. Results are based upon a pre-set number of completed interviews.

Focus Group Series of 4 Sessions with 15 People Telephone Survey Sample of 400 People
Type of Sample Convenience sample, based upon who is willing and able to participate Random sample, including pre-designated target households and substitutions
Representativeness Citizens with more intense feelings likely to respond, potentially biasing the results Random samples can be projected statistically to the entire population within specific limits of accuracy
Probable Response Rate 10%-40% 90%+
Accuracy of Results Questionable; group dynamics may invalidate results; not statistically projectable Statistically significant; e.g., 400 respondents provide results within ± 5.0% of the entire population 95% of time
Fieldwork Time 4-8 weeks 4 weeks
Cost Moderate, depending on recruitment and reminder costs, as well as incentives Moderate, depending on size of questionnaire and size of sample