About outcome measurement

What is outcome measurement?

Put simply, an outcome is the result of an action or a process.

A consumer health outcome is basically a change in a consumer's health status between two measurement occasions.

Outcome measurement (OM) as used in Victoria, is the process of measuring health outcomes of consumers of mental health services by undertaking a series of ratings and comparing the results over time.

Why measure outcomes?

The purpose of measuring clinical outcomes is to see whether consumers of public mental health services in Victoria get better as a result of the services they receive.

This means that OM is not an administrative, statistical or research undertaking. Instead, it goes to the heart of clinical practice and consumer focus in mental health by addressing the vital question of clinical effectiveness.

Measuring the utilisation and performance of mental health services in the past often meant tracking inputs (such as funding levels or the number of staff), throughputs (such as bed occupancy rates or length of stay) or outputs (such as the number of community contacts). While each of these indicators has some ongoing utility, they yield no information as to the clinical effectiveness of the services provided.

The introduction of OM into routine clinical practice was a major undertaking with few international precedents. It has been described as arguably the most significant industry development endeavour since the beginning of de-institutionalisation in the 1960s. The routine use of OM requires clinical leadership, involves a cultural shift towards openness and transparency and embeds a focus upon consumers in clinical practice. It can empower consumers by giving them more opportunities to participate in decision-making about their treatment and care.

Outcome measurement in mental health factsheets provide more information:

Factsheet 1 - What's in it for me? (PDF file 622kb)
Factsheet 2 - What measures are used? (PDF file 634kb)
Factsheet 3 - What happens to the ratings? (PDF file 675kb)

How can we measure outcomes?

Measuring consumer health outcomes in mental health is not straightforward. As mental illness is often episodic in nature, a consumer’s levels of wellbeing, functioning and quality of life may vary over time. Furthermore, a consumer's progress may not always be attributable to the service they received.

In order to measure outcomes in mental health a range of instruments have been identified as both relevant and reliable. These instruments are designed to measure key aspects of a consumer's health status such as severity of symptoms or general functioning. Outcomes are not intended to be viewed in isolation. Outcomes can be derived by comparing these ratings over time within the context of service episodes and interpreted using additional clinical information, such as the diagnosis and legal status of the consumer and the focus of care during the episode. Some measures are designed to be completed by the treating clinician; others are completed by consumers who rate their own mental health.

Guidance to mental health sector on DHS requirements for OM

DHS has developed a program management circular on OM in collaboration with sector representatives to provide guidance to the sector on DHS requirements for OM. The document also provides guidance on a number of protocol areas requiring further clarification.

Outcome measurement in mental health factsheet provides more information:

Factsheet 4 Frequently asked questions – some sticky points (PDF file 885kb)

Leadership

National Advisory Bodies

At the national level, a number of advisory bodies auspiced by the Australian Government ultimately report to the Australian Health Ministers Advisory Council (AHMAC). The key engine for reform in the mental health area is the National Mental Health Standing Committee (NMHSC), which addresses strategic issues affecting mental health services and auspices a Mental Health Information Strategy Sub-committee (MHISS). The aim of MHISS is to strengthen the information infrastructure for mental health across the states and territories. MHISS hosts three OM Expert Groups (Child and Adolescent Mental Health, Adult Mental Health and Older Persons Mental Health). The format, structure, representatives, frequency of meetings and purpose are being reviewed by MHISS.

In 2004 the Australian Government established the Australian Mental Health Outcomes and Classification Network (AMHOCN) as a national resource to support the states and territories in the area of OM. The AMHOCN consortium comprises three components:

• A data bureau that will receive and process outcomes and classification data;
• An analysis and reporting function undertaken by the University of Queensland. This facilitates the development of OM reports that can be used for benchmarking purposes;
• A training and service development function undertaken by The New South Wales Institute of Psychiatry. This component develops training materials to support the states and territories in training and retraining staff in using the measures.

AMHOCN has a range of information available on their website.

Clinical Leadership

Successful implementation of clinical OM relies on senior clinicians in each health service assuming a role as advocates and supporters of the initiative. The pointers below suggest behaviours that might demonstrate clinical leadership in support of OM.

Scope for Ongoing Leadership Action

• Be an advocate for OM by emphasizing the rationale for and utility of OM.
• Support local change management processes, addressing any areas of resistance that may exist in your organisation.
• Develop and monitor an organisation-wide OM sustainability plan including processes, documentation, data requirements, training and clinical application of data.
• Ensure that adequate local arrangements are in place, including communication of the expectation for rater and refresher training in OM.
• Clarify, communicate and monitor local business rule requirements (e.g. who is responsible for completing outcome measures particularly when multiple teams are involved; data entry responsibility; production of reports)
• Monitor compliance with OM protocol and identify opportunities for improvement
• Communicate and facilitate integration of OM in clinical processes such as case review.
• Encourage staff to use OM data to reflect upon their individual and team practice.
• Encourage the professional development of OM 'specialists' in your organization.
• Integrate OM with existing research and quality improvement processes in your organization.
• Actively address emerging policy or practice issues (data entry, reporting, utility of measures,) with DHS, your local OM coordinator or Quality Manager.
• Work with colleagues, peers and DHS on developing and refining strategies for interpreting and using OM data.
• Provide ROMP and DHS with feedback on aggregate reports to enable further refinement of reporting.

Top of page