Cross-cultural research: trying to do it better 1. Issues in study design
Rhonda Small, JaneYelland and Judith Lumley Centre for the Study of Mothers’and Children’s Health, School of Public Health, La Tmbe Universit)! Victoria
Pranee Liamputtong Rice School of Public Health, La Trobe UnivetsiN Victoria
D
esigning a cross-cultural research
’ study poses a number of methodo-
logical and practical challenges which need to be addressed in order to optimise study success. The particular sampling problems posed for research involving ethnic minority populations have been documented, both in Australia and elsewhere.’-3The difficulties include: defining ethnic it^,^,^ obtaining relevant population denominators, determining the most appropriate sampling and recruitment strategies, achieving meaningful sample sizes and assessing the representativeness of recruited samples.’ Standard random or community sampling techniques prove too time-consuming and expensive to be employed successfully in most instances. Researchers venture instead down the path of non-random sampling of one kind or another. Recently reported sampling strategies include snowball sampling from contacts within ethnic community organsiations6 or via the social networks of bicultural interviewers,’ sampling from geographical areas known to have high concentrations of a particular ethnic group or groups,* or from the patient lists of ethnic health providers9 and using surnames to identify and sample particular ethnic communities.I0 A reliance on non-random sampling techniques such as these is problematic, and the potential for bias due to unrepresentative sampling is very Where it is not possible to assess the representativeness of the final sample obtained by comparison with
reliable population datasets, then the nature of such sample bias remains unknown. Even once a sampling approach is determined, recruitment strategies and methods for retaining and following up participants can present further problems for achieving a representative sample in cross-cultural research. The selection, training and ongoing support of bicultural interviewers is also an area of particular importance when designing a cross-cultural research study. While many of the issues have elements in common with any research project employing interviewers,’ 1.12 they assume greater significance in a crosscultural study where language and culture can form very real barriers between the research participants and the researcher^.'^ In this situation, the study interviewer takes on a specialised ‘bridging’ role between the two groups.To perform this task well, interviewers need to be proficient in both English and their community language, to have a good understanding of both cultures (that of the participants and the culture from which the research stems) and they need to be able to make the links between the two so that the interview process is meaningful to participants and the researchers are able to make sense of the data collected. Researching the views and experiences of maternity care among women born in Vietnam, Turkey and the Philippines in the Mothers in a New Country (MINC) Study presented us with dilemmas in all these areas of study design. This paper describes the strategies employed to address them.
Correspondence to: Ms Rhonda Small, Centre for the Study of Mothers’ and Children’s Health, School of Public Health, La Trobe University, Bundoora, Victoria 3083. Fax: (03) 8341 8555; e-mail:
[email protected]
1999 VOL. 23 NO. 4
Abstract Objective:To discuss a range of strategies to address the methodological and practical challenges in designing cross-cultural public health studies. Method: The Mothers in a New Country (MINC) Study was an interview study of 318 Vietnamese, Turkish and Filipino recent mothers exploring their views of maternity care and the early months of motherhood. It was carried out in Melbourne between 1994 and 1997. Sampling, recruitment, retention and representativenessall pose problems for studies involving non-English-speaking background immigrant populations, as do selection. training and support processes for bicultural interviewers.These issues are discussed with reference to the strategies undertaken to tackle them in the MlNC Study. Results: In the MlNC Study, a systematic approach to sampling and recruitment, combined with a flexible and sensitive study protocol were largely successful both in achieving an adequate sample size and a largely representativestudy sample. Similarly, paying significant attention to the selection, training and ongoing support of the bicultural interviewers employed on the study contributed greatly to its successful completion and enhanced confidence in the study findings. Implications: Both researchers and funders need to take seriously the implications of the many methodological and practical issues involved in designing sound cross-cultural public health studies. In particular, there are major implications for study costs and timelines. However, the benefits to be gained are significant. (Aust N ZJ Public Health 1999;23:385-9)
Submitted: August 1998 Revision requested: February 1999 Accepted: May 1999
AUSTRALIAN AND NEW ZEALAND JOURNAL OF PUBLIC HEALTH
385
Small et al.
Methods Determining the sample size Determining sample size for studies with immigrant communities is often hampered by the lack of relevant population data. In MINC, sample size estimates were determined on the basis of findings from a previous population-based survey of women’s views of their maternity care and emotional well-being after birth (estimates for satisfaction with maternity care and depression after birth)I4-l6and data on births to Vietnamese, Turkish and Filipino women from the Victorian Perinatal Data Collection (in particular, rates of caesarean section). In the absence of data on satisfaction with care and depression for our specific country of birth groups, we used the previous survey results for the combined NESB category as the best surrogate data available. On the basis of these estimates, a sample size of at least 100 women from each of the three groups was needed to have adequate power to detect differences on our major outcomes of interest.
Selection of study groups and sampling strategy The decision to selectVietnamese,Turkish and Filipino women was based on overall numbers of women giving birth, variation in migration experiences, different degrees of facility with the English language and cultural distinctness. Data on the number and place of births in our study groups was available from the Victorian Perinatal Data Collection Unit, enabling us to determine the most feasible approach to sampling from a small number of hospitals where births to a significant proportion of women in our study groups occurred. We employed a process of quota sampling from Melbourne’s three maternity teaching hospitals where 57.6% of allVietnamese (total=2,447), 76.3% offurkish (total=797) and 32.7% of Filipino (total=810) women gave birth in Victoria during the study period.
Selection, training and ongoing support of bicultural interviewers The selection process
The interviewer positions in the MINC Study were advertised in the daily press as well as through community and ethnic organisations and via other cross-cultural research projects. A Reference Group had been established to advise the project with the view to broadening the research team’s expertise, and this group included advocates from the Vietnamese, Turkish and Filipino communities. These three community advocates and one invited community member, participated on each of the interview panels to select the bicultural interviewers for the study. We were able to benefit from community members’ views about cultural issues that might be relevant to the selection criteria (e.g. in assessing applicants’ degree of empathy and identification with the situation of women in their community) and in relation to specific interview questions and applicants’ responses. Language skills assessment
The need for excellent language skills in one of the three community languages and in English was crucial to the process of
Box 1: Components of the Bicultural Interviewers’ T‘raining Program. Introduction to and rationale for the proposed study. Background about the health issues at the heart of the research. Time to read and then discuss relevant papers. Skills development in recruiting and interviewing study participants, including participating -in mock interviews. Participation in discussion sessions focused on the relevance of cultural issues to the study research questions (childbirth, motherhood. mental health). _ _ ~ Involvement in the development of the interview schedules and the framing of questions. Detailed discussion of the interview schedules, including making explict the rationale behind each question, the prompts and probes to be used when following up incomplete answers, and the appropriate recording of responses. -__________ Discussion and agreement on protocols for handling a range of situations that might arise during interviews. _____ Participation in the piloting of interviews and the translated standardised instruments used in the study, followed by a review of issues and problems encountered. Time made available to develop resource lists of relevant community information likely to be helpful when interviewing mothers of young babies who were likely to have information needs and questions to pose the interviewers. Participation in briefing meetings for hospital staff at the three recruiting hospitals. ~~
~~
interviewing study participants in their community language and for subsequent translation of responses into English for the purposes of coding and analysing the data. For this reason, we incorporated language assessment into the selection interview process. Shortlisted applicants were advised about the proposed process for the selection interviews, including that their language skills would be assessed both orally and in the form of short written translations, from and into English. The community members on the panel administered this part of the interview and were then able to advise us about applicants’ oral and written competencies. The training program for interviewers
Almost as critical as the initial selection decision is the time and effort invested in training the successful applicants. In the MINC Study, a six-week training program was undertaken. Box 1 lists the main components of the program. Via the training program, the bicultural interviewers were encouraged to take an active part in the direction of the project. A central message of the interviewing skills training was that interviewing is a continual learning process. Despite training and piloting of the interview schedules, there were sure to be unforeseen issues to be discussed and sorted out during the study. With participants from different communities, with very different languages, migration histories and cultural backgrounds, it is difficult in initial interviewer training to encompass all the issues that might arise. Frank and timely communication about any problems the interviewers might experience during the study was emphasised. Only in this way did we feel we could together tackle
~
386
AUSTRALIAN AND NEW ZEALAND JOURNAL OF PUBLIC HEALTH
1999 VOL. 23 NO. 4
Cross-cultural research: trying to do it better. 1. Issues in study design
issues as they arose and thus ensure the best possible outcomes for the research participants and the interviewers, and thus for the study as a whole. To facilitate discussion and ensure regular communication about progress with the interviews, research team meetings were held fortnightly at the start of the study and then monthly. At each meeting, progress was discussed, enabling dialogue about issues that arose and a shared development of appropriate responses to previously unforeseen situations. We encouraged the interviewers to keep interview diaries recording any issues or queries they had following an interview for later discussion at team meetings. Interviewers were also encouraged to contact the project coordinator between meetings if any issues arose which they needed to discuss more urgently. The three interviewers also frequently contacted each other to talk about what was happening.
Recruitment The three trained bicultural interviewers used a systematic recruitment approach to invite women during their postnatal hospital stay to participate in the study. As births to Turkish mothers occurred almost exclusively at two of the hospitals, the interviewer recruited systematically from each of these on a week about basis, visiting three times a week.A similar process was undertaken between all three hospitals for the recruitment of Vietnamese mothers. Filipino women, with the lowest numbers of births at the recruiting hospitals, were recruited from all three hospitals every week, following a phone call by the interviewer to each hospital three times a week identifying whether any women born in the Philippines had given birth. For all three groups, all eligible women identified from ward records at each visit were approached to participate. On the rare occasion when more women were eligible than could be approached, a random sample was drawn from the list of names available. A brief interview at the time of recruitment gathered sociodemographic and background information. Women who had agreed to participate were then contacted by phone three months after the birth to maintain contact and ascertain their continued willingness to be involved in the study. Six months after the birth, women were phoned and an appointment time to conduct an interview at home was requested. Women chose which language the interview would be conducted in. If their choice was English, they were asked whether another member of the research team would be preferredacceptable. This was done in part to enable women to choose someone from outside their own cultural group if this was preferred (for confidentiality reasons), a strategy designed to increase retention in the study. In all, 50 interviews were conducted in English, 12 of which were with other membe; of the research team.
Assessing the representativeness of the final study participants First, the sample of women who completed the study was compared with those initially recruited on a range of demographic 1999 VOL. 23 NO. 4
and other factors. Second, the representativeness of the final sample was determined by comparison with Victorian Perinatal Data Collection Unit data on all women born in these countries giving birth during the study period, both at the recruiting hospitals and across the state of Victoria.
Protocol variations to facilitate participant retention There were a number of ways in which the study varied from the planned protocol. It proved difficult to organise a time for the home interview with Vietnamese women 7-10 days in advance to enable the posting out of one of the mental health measures used in the study. The women’s common response was that they could do the interview ‘tomorrow or the next day’, but were reluctant to make a time so far in advance. When a time was agreed well in advance, frequently the women would not be home. It was decided early into the project that it was more appropriate to conduct the interview at the first time women suggested, even if this meant a variation in the study protocol. It also became apparent that some Vietnamese women were reluctant to have the interviewer come to visit them at home, and several asked if they could do the interview over the phone. It is worth noting that a number of the women in our sample were doing piecework at home, often in very cramped and crowded circumstances and this may have influenced how they felt about having someone visit. In order not to lose participants from the study, the option of a telephone interview was given to women if they seemed reluctant about a personal interview at home. This resulted in 23 (22.1%) of the interviews withVietnamese women being conducted by ph0ne.A comparison with the rest of the sample of Vietnamese women interviewed in person revealed only one significant difference. Women interviewed by phone were more likely to be in paid work than women interviewed in person (12/23 vs. 22/81, OR=2.93 [ 1.02,8.45]).Women interviewed by phone did not differ from those interviewed in person in relation to their satisfaction with maternity care, nor whether or not they scored as depressed, two primary outcome measures in the study.
The bicultural interviewers The three interviewers chosen in the MINC Study were women with a wealth of experience in community work of various kinds: tenancy support work, welfare counselling in schools, and interpreting and translating in community, health and legal settings. None had previously worked on a research project. A very sobering aspect of the selection process was the realisation that at least one of our selection decisions would have been different if we had not had the benefit of formal language assessment. One otherwise very impressive candidate was insufficiently fluent in her original language to have performed well as a bicultural interviewer. Had we not developed a process for language assessment, making such a discovery once the study was under way would have been very costly for all involved, and could well have jeopardised the success of the study.
AUSTRALIAN AND NEW ZEALAND JOURNAL OF PUBLIC HEALTH
387
Small et al.
Table 1: Recruitment, refusals, withdrawals, loss to follow-up and final participants. Vietnamese Turkish Filipino
Total
Recruitment:
- aDDroached - refused - ineligible
173
190
154
517
8
49
6
63
16a
3
0
19
- recruited
149
138
148
435
~
~
Three month contact:
- uncontactable - contacted
___
-withdrew
-
- confirmed participation
14
8
2gb
51
135
130
119
384
4
4
4
12
131
126
115
372
5
5
5
15
126
121
117
364
Six month contact:
- uncontactable - contacted -withdrew
- direct refusal - d e facto refusal - interviewed - ineligiblec Final participants
4
4
4
12
16
7
3
26
5
5
7
17
105
1oa
107
320
1
1
0
2
104
107
107
31a
Notes: (a) 16 Chinese-Vietnamesewomen who were approached spoke insufficient Vietnameseor English to participate. (b) Seven of the 29 Filipino women uncontacfabdeaf fhfee months were contacted at slx. (c) Two women were excluded a?this stage because their babies had been
