Performance measurement in alcohol and other drug treatment services

Abstract

In Australia, there is no standardised approach to measuring performance of alcohol and other drug (AOD) treatment services. Non-government organisations (NGOs) that provide AOD treatment in Australia manage contracts from multiple funding sources, with performance measures varying between funders. They have reported a high reporting burden and inconsistent expectations from different funders to demonstrate accountability. The primary aim of this thesis was to establish a list of performance measures that are acceptable to service users, treatment providers, and funders; explore approaches to improve implementation of performance measures; and make recommendations to funders of non-government AOD treatment. The research approach involved three phases: 1) an assessment by three independent raters of existing measures used in contracts against best practice; 2) focus groups with service users, treatment providers, and funders (n=10 focus groups) to identify the most important performance measures among diverse stakeholders and explore the challenges associated with implementation; 3) a Delphi process with a purposeful sample of service users, treatment providers, and funders to prioritise a core set of performance measures. Phase One found over 500 unique measures used in contracts for AOD treatment services, with most not adhering to best practice. Further, the majority were output and process measures. In Phase Two, focus groups identified that access, outcome, and experience measures were the most important measurement types across all stakeholder groups, with structural measures also important to service users. In Phase Three, 17 performance measures reached consensus. In contrast to Phase One, the final set were mostly outcome, access, and structural measures (n= 11/17) with only one measure each for output and process. Further, key findings from the focus groups highlighted that identification of measures is only part of a robust performance measurement system. Support systems for collecting, analysing, interpreting, and reporting performance data are also needed. At the policy-level, implementation of the final set of measures can improve accountability of public funds, and support the collection of standardised performance-related data to inform funding decisions and treatment planning. At the practice-level, the measures have the potential to reduce reporting burden, improve organisational efficiency, and inform quality improvement initiatives.