Brief Report
I
Primer postcard improves postal survey response rates Abstract Objective: This study measures the effect of an intervention to improve mailed
survey response rates. Method: A randomised controlled trial of a ‘primer’postcard was performed as part of a large survey in Victoria in 1997. Prior to the survey mailout, half the sample of 800 general practitioners supplied by the Health Insurance Commission was sent, at random, a primer card to request prompt return of the survey. Results: The intervention resulted in a more rapid return of the survey and improved overall response rates from 60% to 66%. The increased cost per returned ) largely offset by fewer survey ( 4 0 ~was non-responders requiring follow-up. Conclusions:A primer postcard is a time and cost-efficient method to increase response rates in general practitioner surveys. Implications: Public health researchers should consider implementing this intervention to improve response rates to postal surveys. Reports of other response maximising strategies should report the cost per returned survey to allow better comparison. (Aust NZJPublic Health 1999;23:196-197)
Submitted: May 1998 Revision requested: August 1998 Accepted: October 1998
196
Vlarie Pirotta, Jane Gunn, S.sphen Farish and 5eorgia Karabatsos Department of General Practice and Public Health, University of Melbourne, Victoria
ollecting information from many general practitioners requires a cost and time-effective method. Selfidministered postal surveys are cheaper than elephone or face-to-face interviews‘ yet rep i r e high response rates to be achieved if ion-response bias is to be minimised.* Response rates to postal surveys are inluenccd by the number of follow-up conacts,3 saliency of the topic, administering nstitution, length of survey and target audimce; as well as time taken for the study’ ind inducements4,6*7 Osborn et aI.* recommend that before sending out a postal survey the researchers should phone the physician and inform him/ ner of the forthcoming survey. T h e i r randomised trial of telephone prompting reported accelerated and enhanced response rates. Two further local studies found that response rates were similar whether a doctor or a non-medical research assistant made the prompting telephone call^.^^'^ A more recent study, a survey regarding opportunistic approaches to cancer screening, compared medical and non-medical telephone prompting, a small gift (a pen) and an advance letter prompt. Overall, the response rates were similar in each group, however male doctors’ response rates were significantly better with a medical telephone prompt.’ I While promising, telephoning large numbers of physicians may be time-consuming and costly, especially when long-distance calls are required. We tested the effect of informing physicians of a forthcoming postal survey by mailing a simple ‘primer postcard’.
Method This study was part of a larger research project, investigating general practitioners’ attitudes to complementary therapies. The 1 1-page questionnaire was posted in mid-I997 to a random sample of 800 Australian general practitioners, using a sample generated by the Health Insurance Commission (HIC). The criteria used by the HIC to identify the reference sample were Victorian general practitioners who had had at least 1,500 consultations in the year 1995-96. The random sample was then selected using a technique called ‘Modified Synchronised Sampling’.’’ Five days prior, a primer postcard was sent randomly to the intervention group, using 400 computer-generated random numbers. The primer card explained the purpose of the main study and requested completion and return of the survey. The University of Melbourne Ethics Committee approved the study. Non-responders were followed-up with a reminder postcard at two weeks and if necessary another survey at four weeks. One person (MP), who was blinded to the intervention status of the respondents, completed data entry. The results of the postcard intervention are presented for the first two months after the original mailing of surveys. After this two-month period, urban non-responders with telephone numbers supplied by the HIC received telephone prompts from research assistants who were blinded to the intervention status of the non-responders. Three analyses were conducted. The first looked at return rate over time, prior to the
Correspondence to:
Dr Marie Pirotta, Department of General Practice and Public Health, The University of Melbourne, 200 Berkeley Street, Carlton, Victoria 3053. Fax: (03) 9347 6136; e-mail
[email protected]
AUSTRALIAN AND NEW ZEALAND JOURNAL OF PUBLIC HEALTH
1999 VOL. 23 NO. 2
Brief Report
Primer postcard improves postal survey response rates
use of tclcphone prompts, using a Kaplan-Meier survival curve. The second looked at overall response, by calculating an odds ratio to compare the response rates of the two groups. The third analysis looked at the cost per return for each group.
Results Of thc 800 surveys, 488 were retumed.Thirty-six subjects were excluded from the denominator ( 1 7 from the intervention group and 19 from the control group) - 24 had left their clinic with no forwarding address, three had taken extended leave, three had retired and six had either died, moved overseas or were very ill. Overall, a 64% (488/764) response rate was achieved. The demographics of the responders in the intervention and control groups were compared (age, sex, rurahrban, years as a general practitioner, year of graduation, solo practice) and none were found to be significantly different (lowest p=O. 12). The survival curve (Figure 1 ) shows that the intervention group had a consistently higher rate of return over the study period. The postcard resulted in a significantly more rapid response (the log-rank test p=0.046) and increased the overall response rate from 59.7% to 65.8% in the intervention group, although this latter effect did not quite achieve statistical significance. The odds ratio for response was 1.29 (0.96-1.73). There was a gender difference in response rates prior to the commencement of telephone prompting. While 26.0% of men had returned their surveys, only 17.7% of women had done so (odds ratio = 1.63, 95% C1 1.1 1-2.40).A significant difference in response rates by gender persisted at the end of the trial (OR 1.57, 95%CI 1.15-2.15). Using the actual costs (including artwork, printing, postage, stationery and labour for the primer and reminder postcards, initial and follow-up surveys), the difference per returned survey was minimal, being A$13.15 per control and AS13.55 per intervention. There was no significant difference between the rates of phone reminders (p=0.24), with the control group having a higher rate of reminder calls, as more of this group had not responded. This would tend to diminish any difference in response rates between the two groups.
..
0
20
40
60
Days
Figure 1: Non returns by days.
our study found that men's response rates were significantly higher than women's. A primer postcard appears to be a time and cost-efficient method to increase response rates in general practitioner surveys. It is also cost-effective, with an increased cost per returned survey of only 3%. The earlier studies8-" have not reported costs of the prompting strategies as an amount per returned survey, which limits comparisons with this study. Authors should consider including the cost per returned survey in their published reports to assist others planning surveys of health professionals.
References 1, Armstrong BK, White E, Saracci R. Principles of exposure measurement in Epidemiology. Oxford: Oxford University Press, 1994. 2. Sibbald BJ, Addington-Hall J, Brenneman D, Freeling P. Telephone versus
3. 4.
5.
6.
Commentshe commendat ions
7.
A primer postcard sent five days prior to the survey resulted in a more rapid response and increased the overall response rate from 60% to 66%. Previously cited Australian studies, which employed various response maximising strategies,9-" found no improvement in response rates in general practitioner surveys. These studies were limited by small numbers, which were only able to confidently detect differences of 10% to 15% between groups. However, Ward et aI.Il found that subgroups responded differently to interventions, so future studies would require sufficient sample sizes to enable sub-group analyses to be undertaken. Interestingly, in contrast to these previous Australian studies,
8.
1999 VOL. 23 NO.2
i
01
9.
10. 1 1,
12.
postal surveys o f general practitioners: methodological considerations. Br J Gen Pmct 1994; 44: 297-300. Dillman DA. Mail and telephone surveys: The total design method. New York: Wiley, 1978. HeberleinT, Baumgartener R. Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. Am Sociol Rev 1978; 43: 447-62. Parsons J, Warnecke RRC, Barnsley J. Kaluzny A. Factors associated with response rates in a national survey ofprimary care physicians. Eva1 Rev 1994; 18: 756-65. Deehan A, Templeton L, Taylor C, et al. The effect cash and other financial inducements on the response rate of general practitioners in a national postal study. Br J Gen P mct 1997; 47: 87-90. Goyder J. Further evidence on factors affecting response rates to mailed questionnaires. Am Sociol Rev 1982; 47: 550-3. Osborn M, Ward 1, Boyle C. Effectiveness of telephone prompts when surveying general practitioners: A randomised trial. Aust Fam Physician 1996; 2 5 : S41-S43. Gupta L, Ward J, D'Este C. Differential effectiveness of telephone prompts by medical and nonmedical staff i n increasing survey response rates: A randomised trial. Aust N Z J Public Health 1997; 21: 98-9. Temple-Smith M, Mulvey G, Doyle W. Maximising response rates in a survey of general practitioners. Ausf Fam Physician 1998; 27 Suppl: S15-SI8. Ward J, Bruce T, Halt P, et al. Labour-saving strategies to maintain survey response rates: A randomised trial. Aust N Z J Public Healfh 1998; 22: 394-6. Calcino G. Sampling from the HIC data set. Proceedings of the 1993 General Practice Evaluafion Progmm Work-in-Progress Con/erence. Canberra: National Capital Printing, 1993.
AUSTRALIAN AND NEW ZEALAND JOURNAL OF PUBLIC HEALTH
197
