Research has identified a number of effective components of treatment for alcohol and drug misuse and addiction. However, in order to ensure best practice and the maintenance of universal standards, continued monitoring and supervision of delivery is needed. This monitoring needs to be able to assess the extent to which effective methods are being used by therapists in routine practice. A number of instruments have previously been developed for treatment process evaluation (Madson & Campbell 2006). However, these tend either to focus on specific treatment types (e.g. Tober et al. 2008; Madson et al. 2005) or be too lengthy for use in routine practice (e.g. Carroll et al. 2000).
The aim of this study was to develop and validate a trans-theoretical scale that could allow for effective monitoring across a range of treatment types, and in a range of routine settings.
A literature review of existing scales designed to measure active components of effective treatments found 26 instruments with a total of 793 items. After thematic coding, eighteen ‘exemplar items’ were identified.
These ‘exemplar items’ were considered by 12 experts in the field of service provision and practice evaluation, using a Delphi survey (Powell 2003). After three rounds, consensus was reached on 12 items, which formed the first version of the scale. Item definitions providing guidance were also produced. Practitioners from four treatment sites were then consulted and minor amendments were made. Validity testing was conducted by the primary investigator by using the scale to evaluate four video-recorded sessions. Reliability testing was conducted by two researchers by independently rating a sample of video from these clinical samples.
The final scale was tested for validity with 80 pre-existing recordings of therapy sessions in routine practice. These had been created as part of three previous randomised controlled trials, and ranged from brief opportunistic intervention in hospital and primary care to hour-long sessions in specialist treatment settings. Providers included both NHS and third sector, and practitioners included psychiatrists, medical staff, general and psychiatric nurses, counsellors, and addiction therapists. Concurrent validity was based on the finding of significant correlations with ratings from two other instruments in previous research reports (Watson et al 2013a; Watson et al. 2013b). Pearson’s r = 0.678, p=0.01; r=0.805, p=0.01 respectively. Twenty recordings were randomly selected for double rating by two members of the research team to test for inter-rater reliability: individual item agreement ranged from 74% to 89%. Overall agreement per tape rated ranged from 83% to 100%.
In the absence of regular supervision and monitoring of practice, practitioners can lapse into ineffective interactions with service users (Martino et al. 2008). The Brief Addiction Therapist Scale was designed to promote the maintenance of effective practice by supporting both training and routine supervision. It can be applied with no additional costs to services, and can be used to evaluate standards at both individual and agency level. In each case, it can help assess the quality of the treatment being delivered.
The scale sets key standards for competence in treatment delivery, using Likert scale ratings to enable the tracking of improvements (or deterioration) in delivery. The quality of practice can, therefore, be better quantified and the effects on treatment outcome more consistently measured (Schoenwald & Garland 2013).
The BATS scale has acceptable psychometric properties and has been shown to be well understood and accessible to a range of practitioners. The scale has been adopted for peer supervision at a multi-site NHS addiction service in the UK, demonstrating its potential to impact on service delivery. It has been requested for use in specialist services in Estonia and Wales and implementation is being followed up. Feedback continues to be sought.
For further information contact Helen Crosby:
Email: Helen Crosby H.F.Crosby@leeds.ac.uk, BATS@leeds.ac.uk
Braun, V. and Clarke, V. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology. 3(2), pp.77-101.
Carroll, K.M., Nich, C., Sifry, R.L., Nuro, K.F., Frankforter, T.L., Ball, S.A., Fenton, L. and Rounsaville, B.J. 2000. A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and Alcohol Dependence. 57(3), pp.225-238.
Madson, M.B. and Campbell, T.C. 2006. Measures of fidelity in motivational enhancement: A systematic review. Journal of Substance Abuse Treatment. 31(1), pp.67-73.
Madson, M.B., Campbell, T.C., Barrett, D.E., Brondino, M.J. and Melchert, T.P. 2005. Development of the Motivational Interviewing Supervision and Training Scale. Psychology of Addictive Behaviors. 19(3), pp.303-310.
Martino, S., Ball, S.A., Nich, C., Frankforter, T.L. and Carroll, K.M. 2008. Community program therapist adherence and competence in motivational enhancement therapy. Drug and Alcohol Dependence. 96(1–2), pp.37-48.
Powell, C. 2003. The Delphi technique: myths and realities. Journal of Advanced Nursing. 41(4), pp.376-382.
Schoenwald, S.K. and Garland, A.F. 2013. A review of treatment adherence measurement methods. Psychological assessment. 25(1), p146.
Tober G, Clyne W, Finnegan O, Farrin A & Russell I in collaboration with the UKATT Research Team (2008) Validation of a scale for rating the delivery of psycho-social treatments for alcohol dependence and misuse: the UKATT Process Rating Scale (PRS). Alcohol and Alcoholism 43: 675-682.
Watson, J., Crosby, H., Dale, V., Tober, G., Wu, Q., Lang, J., McGovern, R., Newbury-Birch, D., Parrott, S. and Bland, J. 2013a. AESOPS: a randomised controlled trial of the clinical effectiveness and cost-effectiveness of opportunistic screening and stepped care interventions for older hazardous alcohol users in primary care.
Watson, J.M., Fairhurst, C., Li, J., Tober, G., Crosby, H., Lloyd, C., Godfrey, C., Mdege, N.D., Dale, V., Toner, P., Parrott, S. and Raistrick, D. 2015. ADAPTA: A pilot randomised controlled trial of an alcohol-focused intervention versus a healthy living intervention for problem drinkers identified in a general hospital setting. Drug and Alcohol Dependence. 154, pp.117-124.
