High attrition rates seriously threaten the validity of follow-up studies of criminal justice populations. This study examines attrition from a follow-up study of drink-driving offenders referred 5 years earlier to a screening programme. The aim of the study was to determine which factors are most closely associated with: (1) inability to locate subjects, (2) subjects' refusal to participate; (3) the manner in which subjects refuse. Logistic regression models compared the following groups of subjects: located vs not located; interviewed vs not interviewed; type of refusal (direct vs indirect). Independent variables included gender, age group, ethnicity, whether the subject had a telephone, compliance with and completion of the screening programme, alcohol dependence or abuse diagnosis vs no diagnosis, breath-alcohol level (BAL) at the time of arrest, and whether the subject had an outstanding arrest warrant. Some factors (younger age, screening compliance, Mexican national ethnicity, and having an outstanding arrest warrant) predicted both inability to locate and type of refusal. Hispanic ethnicity and having a telephone predicted better success with locating subjects. Among refusers, non-Hispanic whites were more likely than other ethnic groups to refuse directly, and those with warrants were more likely to refuse indirectly. Non-compliance with the screening programme was also associated with differential follow-up rates. Neither arrest BAL nor alcohol diagnoses was associated with differential rates of follow-up. We conclude that alcohol diagnosis does not appear to influence successful follow-up in this criminal justice population. Rather, tracking and interviewing challenges differed among ethnic groups, suggesting a need for culturally sensitive recruitment strategies in these populations.
With increasing numbers entering the criminal justice system, researchers and policy-makers need to understand the factors that affect offenders' recidivism and reintegration into society. While follow-up longitudinal studies are important tools for identifying risk factors associated with favourable outcomes (Windle and Miller, 1990), their validity can be threatened by high attrition rates. A minimum follow-up rate of 80% is generally considered necessary to ensure that the interviewed sample is representative of the population under study (Stout et al., 1996). Even limited follow-up information on study subjects can be useful in evaluating comparability between the successfully followed sample and the intended study population. But, as pointed out by Stout et al. (1996), there is usually a group of participants for whom no follow-up information is available and bias may result.
Examples of attrition in longitudinal studies in criminal justice populations abound. A three-wave longitudinal study of 461 driving while impaired (DWI) offenders conducted over a 1.5-year follow-up period found that only 66% agreed to participate and completed at least one follow-up interview (Windle and Miller, 1990). A study which attempted to locate addicts arrested from 1.5 to 17 years earlier for possession, illegal use, or sale of narcotics in Baltimore, MD, USA, found that, of 36 persons in the sample, 64% were located (Nurco and Lerner, 1971). In another longitudinal study, 399 Florida detainees admitted to a juvenile detention centre were contacted for re-interview 4 months after admission. The completion rate was only 76.4% (Dembo et al., 1990).
The low follow-up rates may be due, in part, to the high rate of alcohol and drug problems in these populations. It has been suggested that the severity of the alcohol and drug use problem is directly related to difficulties in reaching study subjects (Robins, 1966; Cottler et al., 1987; Temple, 1993). Cottler et al. (1987) found that contact attempts with subjects who had alcohol problems required 20% more time to reach them, compared to subjects without alcohol problems.
At our centre, a follow-up study was recently conducted to determine the prevalence rate of alcohol and drug-related problems 5 years after offenders were convicted of DWI in Bernalillo County, NM, USA and entered a court-mandated screening programme. The screening programme was designed to support ongoing research efforts, so a great deal of baseline data were available by which to compare located and not-located subjects, and subjects who refused the interview. We undertook the present study to explore factors affecting attrition from the DWI offender follow-up study. In particular, we wanted to discover which factors were associated with the inability to locate subjects and with subjects' refusal to participate. To estimate the extent of potential bias in study outcome results, we also investigated whether subjects with suspected alcohol-related problems at screening were more or less likely to be located and to agree to be interviewed than subjects without alcohol-related problems.
SUBJECTS AND METHODS
Screening programme procedures
The study population consisted of individuals participating in a follow-up evaluation of a DWI screening programme, the Lovelace Comprehensive Screening Programme (LCSP). Details of the LCSP are presented elsewhere (Lapham et al., 1995). Briefly, the LCSP contracted with the Metropolitan Court of Bernalillo County, NM, to provide alcohol and drug screening services for convicted DWI offenders. The programme was conducted between April 1989 and October 1995. Services included an evaluation for alcohol- and drug-related problems, referral to community-based treatment programmes, and monitoring of client compliance with the screening and referral process. Client information from the interviews and data from the standardized tests were entered into a database, with procedures for maintaining confidentiality and quality assurance. The counsellor who interviewed the client at screening assigned this diagnosis, but only to those who completed the screening process. Therefore, this variable is missing for 22% of subjects. For those whose diagnoses were not known, the separate category ‘unknown diagnosis’ was created.
DWI offenders completed both standardized tests and a structured personal interview. The interviews were conducted by masters-degree-level counsellors who received training in the goals of the screening programme, interpretation of the test results, and resources for referral to treatment. The counsellors discussed the test results with their clients, obtained detailed histories regarding previous arrest and substance use, and sought additional information regarding any legal, social, family or medical problems related to alcohol or drug use (Lapham et al., 1995). The counsellor, using all the information available, including results of written tests and information obtained from the interview, then assigned an alcohol diagnosis (abuse or dependence) according to the Diagnostic and Statistical Manual of Mental Disorders, version III-R criteria (American Psychiatric Association, 1987). The diagnostic criteria questions were computer-prompted and counsellors entered responses directly during interviews. The counsellors' recommendations regarding referral to treatment programme were enforceable by the courts.
Clients referred for screening were classified into one of four screening status categories, depending on their level of compliance and completion of the programme: (1) Compliant: the client was compliant with all phases of screening (kept all appointments and fulfilled the mandatory, court-ordered requirements); (2) Not compliant: the client was not compliant with the screening process and consequently did not complete the programme; (3) Not compliant/complete: the client failed to pay fees or keep appointments, but did eventually complete the screening process; (4) Waived: the judge who sentenced the client to screening waived the sentence, and the client was no longer required to complete the screening programme.
Clients who did not comply were notified by letter of their non-compliance and were given 7 working days to correct the problem. If, within this period they did not contact the LCSP and were not reinstated to the programme, a letter was sent to the Bernalillo County Metropolitan Court notifying the Court of non-compliance. The client's LCSP file was updated to include the non-compliance notification. The court then issued a summons for the client to appear in front of his/her sentencing judge, and if the client did not respond, a bench warrant was issued. If sometime during this process the client responded and finished the screening process, she/he was classified as ‘not compliant/complete’.
The sample for the follow-up study included 1208 consecutive females and a sample of 1407 males referred to the LCSP from April 1989–March 1992 who initially reported to the programme. At the initial visit, clients provided demographic information, including home and work address. Fewer than 10% of referred offenders failed to report. These potential subjects were not included in the sample, as we had limited locating information for this cohort. Also, tracking information revealed 56 subjects (18 females and 38 males) who were deceased. These were removed from the study sample for the analyses.
Subjects were selected weekly over a 36-month period corresponding to the subjects' 5-year anniversary of their LCSP referral. Males were frequency matched to females by same date of referral to screening and ethnicity. Males were oversampled in the follow-up study, as a previous survey conducted in a New Mexico community (Samet, 1988) showed a higher refusal rate among males than females. The refusal rate in that study was 27.8% for males and 12.8% for females aged 18–34 years and 24.7% for males and 16.0% for females aged 35–44 years.
Locating and tracking clients
Efforts to contact and interview clients began on 25 January 1994, and continued until 30 June 1997. The primary data source for locating clients was LCSP record data. Motor Vehicle Department files, Polk City Directory, Haynes Directory, and a credit database were also used to track clients whose addresses and telephone numbers had changed since their enrolment in the LCSP. Interviewees were paid to complete the interview process. The Lovelace Institutional Review Board (Albuquerque, NM) approved the protocol.
Subjects were mailed a series of letters at 2-week intervals. A letter sequence included five identical letters (explaining the study and asking the client to participate) and one ‘pleading letter’ emphasizing the importance of the study. In addition, calls were made to all available telephone numbers and, when necessary, project staff visited homes to determine whether subjects lived at the listed addresses.
Locators, bilingual in English and Spanish, followed written procedures to convince clients to participate and were trained in techniques to reduce refusal rates. Client contacts centred around the need to have more information about health and psychological issues and the contribution subjects could make to this research. Those who refused were contacted by the principal investigator, who further attempted to convince subjects to participate. For those who agreed, an appointment was scheduled to complete the interview. Willing subjects provided written informed consent. The protocol was approved by an institutional review board.
The interview took an average of 1.5–2 hours. Clients were also asked to complete a series of questionnaires at home. Most interviews were done at the study's office site. Staff were directed to use their own judgement in offering home interviews. If the client seemed reluctant to come to the office or stated that the office location was inconvenient, the client was interviewed at home or in a neutral location. Out-of-town clients and those unwilling to participate in face-to-face interviews were interviewed via telephone. The participants were paid $75 for participation. This was augmented by offers of gift certificates to restaurants and $100 cash lotteries for those who participated.
The study's comparisons of interest are dichotomous. Subjects were either located or not. Once located, they were either interviewed or they refused. Those who refused either did so directly or refused indirectly. These groups are appropriately compared using logistic regression, whose dependent variable is dichotomous. Therefore, three logistic regression models were developed to determine the relationship between subject characteristics and follow-up status. The logistic regression models control for the effect of all covariables on the outcome variable. The first statistical model compared located vs not-located subjects. This model included all 2503 subjects selected for the follow-up study who were not known to be deceased. The remaining models had different sample sizes depending on the groups to be compared. The second model compared interviewed subjects with all refusers. The third model compared direct refusers with indirect refusers.
Of the 2559 selected subjects, 2062 (81%) were located, and 68% of those located were interviewed (Table 1⇔). Upon completion of the locating and interviewing phase of the study, subjects were classified into one of four status categories: (1) Interviewed: the client was located and agreed to be interviewed (n = 1396); (2) Refused indirectly: the client was located, but either did not schedule an appointment or failed to keep a scheduled appointment after being contacted (n = 239); (3) Refused directly: the client was located but explicitly and unambiguously refused to be interviewed (n = 427); or (4) Not located: the client was not located (n = 497).
Baseline client characteristics, by follow-up status
Located n (% of selected)
Interviewed n (% of located)
Refused directly n (% of refused)
Refused indirectly n (% of refused)
*Does not include deceased subjects.
29 and under
30 and over
Outstanding arrest warrant
Breath-alcohol level (BAL)
Variables included in the analysis are shown in Table 1⇔. These included demographic data (gender, age, ethnicity), whether the client had a telephone at the time of screening (an indicator of socioeconomic status), breath-alcohol level (BAL) at the time of arrest (an indicator of alcohol-related problems) (Lapham et al., 1997), whether or not the client had an outstanding arrest warrant at the time of follow-up, and alcohol diagnosis. Another variable included in the analysis was whether clients were compliant with or had completed the court-mandated screening process approximately 5 years earlier. Compliance was included in the models, because it was hypothesized that those who were less compliant with the original screening programme may be less willing to participate in the follow-up study.
Females were located more often than males (χ2: P < 0.001) and, once located, were more likely to be interviewed (χ2: P< 0.001). Only 60% of Mexican nationals were located, and 43% of the located Mexican nationals were interviewed. The Mexican national population has the highest indirect refusal rate of any group. Of the clients who did not have a telephone, only 67% were located. Outstanding arrest warrants had been issued for 9% of the population. (This information was obtained at follow-up from Bernalillo County Metropolitan Court, NM, records.) Of those, 64% were located, and 59% of those located were interviewed. The majority of the selected, as well as the interviewed subjects, were under the age of 30 years at the time of the follow-up interview.
Located vs Not located subject comparison
The first multivariate analysis was conducted to determine characteristics associated with inability to locate subjects. Logistic regression analysis revealed that males were statistically less likely than females and those aged 30 years or above were more likely than those under 30 years of age to be located (Table 2⇔). Compared to non-Hispanic whites, Hispanics had a higher location rate, and Mexican nationals had a lower location rate. The odds ratios for location were higher for those with telephones and lower for subjects with outstanding warrants, compared to those without these characteristics. In fact, those with outstanding arrest warrants were only about half as likely to be located as those without warrants. The odds of not being located were also lower among those non-compliant with screening. There were no significant associations between arrest BAL or alcohol diagnosis and location status.
The second comparison included all subjects who were located, and analyses were performed to compare characteristics of those who were interviewed with those who did not participate. Odds ratios (OR) for being interviewed were lower among males, those with Mexican national ethnicity, and those with outstanding arrest warrants, compared with the reference groups. That is, they had higher refusal rates (Table 2⇔). Conversely, subjects in the older age group and persons who had been non-compliant with screening but who eventually completed it (not compliant/complete) were more likely than their respective reference groups to be interviewed, once located.
Refused directly vs Refused indirectly subject comparison
Once located, the next problem was to convince subjects to complete the interview. The last logistic regression model included those who refused to participate in the follow-up study, and compared characteristics of those who refused directly with those who refused indirectly. Thirty-two per cent of located subjects refused to be interviewed. Subjects refused in one of two ways; directly, definitively stating their unwillingness to participate in follow-up, or indirectly, not keeping appointments or delaying the scheduling of an appointment. The odds of being a direct refuser were higher among older, than younger subjects, and were lower among Hispanics, Mexican nationals, and those in the ‘other race’ group, compared with the non-Hispanic white reference group (Table 2⇔). This was especially true of Mexican nationals, who had an OR for direct refusal of 0.14, compared with non-Hispanic whites. The odds of being a direct refuser also were lower for subjects with outstanding warrants and those who had been non-compliant, but completed the screening programme.
Two findings of this study are most noteworthy: (1) subjects with different ethnic backgrounds posed substantially different tracking and interviewing challenges compared with non-Hispanic whites; (2) alcohol diagnoses did not appear to affect the investigators' ability to locate or interview respondents. First, we found that Mexican nationals were located at a rate of 60%, the lowest location rate for any group. Although our sample included only 124 Mexican nationals selected for follow-up, their low location rate indicates differences between this and other populations. Possible explanations include the movement of the Mexican national population between the USA and Mexico and the possibility that many may be illegal immigrants.
A second intriguing finding was that Hispanics who were not Mexican nationals were the easiest population to locate (85%) and were 1.4 times more likely to be located than non-Hispanic whites. This phenomenon was also observed in a follow-up study of homeless alcohol abusers enrolled in an intervention programme in Albuquerque, NM (Lapham et al., 1996). Possible explanations include the strong family ties within Hispanic culture and their long history in New Mexico (Keefe and Padilla, 1987) leading to less emigration than other New Mexicans.
Finally, Mexican nationals and Hispanics, along with persons in the ‘other race’ group, were also statistically more likely than persons belonging to the other race groups to indirectly refuse the interview. One explanation is that many people within the Hispanic and Mexican national cultures are uneasy about disagreeing with someone in an authoritative position, in this case a researcher. Quesada (1976) states “Mexican and Latino cultural norms produce an accompanying ‘yes, yes mentality’”. In other words, it may be considered impolite or rude to refuse a reasonable request in this culture and the compromise is to agree but not follow through.
Other plausible explanations include less interest or investment in the research, privacy concerns, or lifestyles that make participation difficult. In the latter case, home interviews may increase participation rates, although offers to interview subjects in their homes for this study did not always yield positive results. Another possible contribution to this problem is that Mexican nationals may be in the USA illegally. However, other studies have shown that Mexican nationals who are illegal aliens are not any more likely than other Mexican nationals to refuse participation in research (Nurco and Lerner, 1971).
The other major finding of this investigation was the lack of association between alcohol diagnosis and attrition in this population. There were two indicators of alcohol problems in this population; diagnosis of alcohol abuse or dependence assigned by the counsellor at screening, and arrest BAL. Since about one-third of the offenders did not complete screening, the study is limited by incomplete data for the diagnosis variable. We attempted to address this by adding BAL to the statistical model. The rationale is that previous studies have demonstrated that persons with high BALs are rearrested at higher rates than those with low BALs (Epperson et al., 1975; Arstein-Kerslake and Peck, 1985; Lapham et al., 1997) and that a higher proportion is diagnosed with alcohol problems (Nichols and Reis, 1974; Lucker and Gold, 1995). However, high BAL has low sensitivity for detecting serious alcohol-related problems (Wieczorek et al., 1992; Lucker and Gold, 1995).
Despite its limitations, this study found no evidence of selection bias regarding the severity of alcohol impairment. Even so, it is not possible to determine whether, and what, biases were introduced by the high refusal rates. This finding does not support previously cited studies that found higher attrition rates among alcohol abusers than non-alcohol abusers. However, few such studies were conducted in criminal justice populations, where substantial proportions of subjects have alcohol and drug use problems (Cloninger and Guze, 1973; McCord, 1984; Bureau of Justice Statistics, 1992).
Other findings of this study are similar to those of other investigators. These include higher refusal rates among males than females (Cottler et al., 1987, 1996; Samet, 1988) and the association of young age with higher refusal rates (Robins, 1966; Cottler et al., 1987; Samet, 1988). Our data also indicate that people with outstanding arrest warrants and those who were non-compliant with the screening programme were statistically less likely to be located or to refuse than the reference groups.
Persons with criminal records would seemingly be the most difficult to locate in a longitudinal study (Dillman, 1978; Freeman, 1998) and it is, therefore, understandable that follow-up studies are particularly challenging among criminal justice populations. Persons with outstanding arrest warrants or who have history of non-compliance with the courts, and are vulnerable to arrest by law enforcement officers, may leave the state or be in hiding. They are more likely than others to be suspicious of attempted contact by strangers. Subjects with outstanding arrest warrants, fearing the possibility of police ‘sting’ operations, may have been more likely to refuse indirectly. Several of these ‘sting’ operations occurred during the period we were contacting participants for our study and these were publicized by the local media. This may have added to subjects' scepticism about the true intent of our project. Given these factors, interview staff members could improve their success rates by making a more convincing case that they are not connected with the law enforcement system. Of course, the best method for accomplishing this would be to anticipate a follow-up study and develop a rapport with subjects during the first contact (i.e. at initial screening).
Another finding was the lower location rate of persons without telephones. These subjects, who constituted 11.2% of the population, were located at a rate of 67%. This rate is low, compared to the location rate of those with telephones (82%) and probably would have been even lower without the use of multiple location methods (supplementing telephone calls with letters and visits to the clients' residences). Our rate of subjects without telephones was somewhat higher than the overall rate for people without telephones in Bernalillo County, NM. Data from the US Census Bureau show that 8.1% of Bernalillo County households, in which the head of the household was aged 15–59 years, do not have telephones (US Department of Commerce, 1990). With rates this high, it would seem that the use of multiple location methods is important in follow-up studies. Dillman (1978) demonstrated that the use of multiple location methods increased the location rate of his entire study population.
A prominent focus of the analysis was the characteristics of persons who directly refused vs those who indirectly refused to participate in a follow-up interview. Interestingly, the two refusal groups differed demographically in more ways than ethnicity. Whereas older subjects were more likely to participate in the interview than younger subjects, the older subjects who refused were statistically more likely than younger subjects to refuse directly. It is unclear why subjects over 30 years of age participated at higher rates than younger subjects. The most likely explanation is that older persons are more likely to have regular schedules and be more socially responsible than their younger counterparts.
The follow-up rate for this study was more limited by its high rate of non-participation than by inability to locate subjects. One reason for study non-participation that could not be evaluated in the present work — and that could account for much of it — was the clients' attitudes towards the court-mandated screening programme. The association of this research with the court and legal system may have negatively influenced subjects' willingness to participate in the follow-up interview. This may explain the finding that those who were initially non-compliant with screening but who completed (‘not compliant/complete’) were more likely to have refused indirectly. This does not explain, however, why members of this group were more likely to complete the interview, once located.
Subjects' experience of the original conviction and screening process may have also played a role. A study of factors associated with research attrition in a 4-year follow-up of alcohol treatment determined that the strongest predictors of refusal are all related to treatment participation (Stout et al., 1996). ‘Four years after treatment many participants had vivid memories of the index treatment, positive or negative, and these experiences were frequently cited as reasons for cooperating or refusing’. Our interviewers reported similar responses from subjects. It appears that the experience of having been convicted of DWI and mandated by the courts to attend the screening programme may have left some clients reluctant to participate in the follow-up interview, despite the monetary compensation.
Solutions to reduce refusal rates suggested by others include initially informing the subjects of the possibility of follow-up research, maintaining contact at regular intervals to facilitate the formation of positive relationships between clients and service providers, and finding more appropriate incentives for participation (Desmond et al., 1995; Cottler et al., 1996). It should be noted that some of the most difficult populations, those with outstanding arrest warrants and those without telephones, still had location rates of over 60% (Stinchfield et al., 1994). The fact that 81% of the subjects were located after 5 years indicates that it is possible to achieve high location rates for a criminal justice population, even after an extended period of time. However, as our study suggests, to reduce refusals even further, and to ensure that experimental validity remains as high as possible, researchers must expend extra effort and extend increased attention to ethnic minorities and to those who have continued to experience legal problems.
This study was supported by the National Institute on Alcohol Abuse and Alcoholism, Grant #RO1 AA09620.
↵* Author to whom correspondence should be addressed.
Arstein-Kerslake, G. W. and Peck, R. C. (1985) A Typological Analysis of California DUI offenders and DUI Recidivism Correlates, pp. 1–191. State of California Department of Motor Vehicles (Research and Development Office) Sacramento, CA.
Cottler, L. B., Zipp, J. F., Robins, L. N. and Spitznagel, E. L. (1987) Difficult-to-recruit respondents and their effect on prevalence estimates in an epidemiologic survey. American Journal of Epidemiology125, 329–339.
Dembo, R., Williams, L., LaVoie, L., Schmeidler, J., Kern, J., Getreu, A., Berry, E., Genung, L. and Wish, E. D. (1990) A longitudinal study of the relationships among alcohol use, marijuana/hashish use, cocaine use, and emotional/psychological functioning problems in a cohort of high-risk youths. International Journal of the Addictions25, 1341–1382.
Epperson, W. V., Harano, R. M. and Peck, R. C. (1975) Final Report to the Legislature of the State of California in Accord with Resolution Chapter 152, 1972 Legislative Session. Senate Concurrent Resolution 44-Harmer, Sacramento, CA.
Lapham, S. C., Skipper, B. J., Owen, J. P., Kleyboecker, K., Teaf, D., Thompson, B. and Simpson, G. (1995) Alcohol abuse screening instruments: Normative test data collected from a first DWI offender screening program. Journal of Studies on Alcohol56, 51–59.
Lapham, S. C., Hall, M., Snyder, J., Skipper, B., McMurray-Avila, M., Pulvino, S. and Kozeny, T. (1996) Demonstration of a mixed social/medical model detoxification program for homeless alcohol abusers. Contemporary Drug Problems23, 331–345.
Lapham, S. C., Skipper, B. J. and Simpson, G. L. (1997) A prospective study of the utility of standardized instruments in predicting recidivism among first DWI offenders. Journal of Studies on Alcohol58, 524–530.
Stout, R. L., Brown, P. J., Longabaugh, R. and Noel, N. (1996) Determinants of research follow-up participation in an alcohol treatment outcome trial. Journal of Consulting and Clinical Psychology64, 614–618.