Learning organisations are…where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free and where people are continually learning how to learn together.

Evaluating Integrated Mental Health Care

Recovery evaluation a program such as Partners in Recovery requires more than measuring individual service outcomes. It also involves examining how organisations collaborate, how effectively people navigate different services, and whether the model is responding to the needs of people experiencing complex mental health challenges. A structured mental health program evaluation can bring these factors together to provide practical evidence for improvement.

Larter evaluated the Loddon Mallee Murray region’s Partners in Recovery program (LMMR PIR).  From our previous experience with other PIR programs, we know:

1. PIR stakeholders understand the consumer group well and they see the current service system failing these very vulnerable people. As a consequence, and on the whole, there is goodwill, trust and a strong commitment to finding ways to improve the outcomes for these consumers.
2. PIR is a great initiative but implementing and evaluating that initiative is COMPLEX!

Make learning and changing the primary goal

Larter consultants have participated in numerous evaluations through positions held in government departments and healthcare organisations.  Too often, the first time we were exposed to the evaluation findings is in a draft evaluation report along with the suite of recommendations to respond to these findings.  This approach inadvertently directs the focus to assessing the appropriateness or feasibility of these recommendations.

For the LMMR PIR Recovery evaluation, we engaged stakeholders in expansive thinking about the findings and harvested their collective aspiration for the program to inform our recommendations. Larter embedded a series of action learning workshops that engaged stakeholders with key findings before recommendations were made and with stakeholders working together developed the strategies for change to respond to the findings.

Collaborate

Collaborating working together fosters an environment of learning and change.  With LMMR PIR, every stakeholder brought a richness to the evaluation: the staff involved in delivering PIR had a deep understanding of program operations, clinical experience and their aspirations and goals for the program; consumer and carers had their lived experience of mental illness and their experiences of this and other mental health programs; Larter had evaluation knowledge and experience and an external view.

Understanding Service Coordination and Consumer Experience

Strong service coordination is an important component of recovery-oriented care. Evaluation can bring together data from service providers, consumers, carers, and other stakeholders to identify barriers to access, strengths in existing service pathways, and opportunities to improve the overall experience of care.

Be flexibly rigorous

While an apparent oxymoron, being flexibly rigorous is a necessary state of mind when evaluating an innovative and evolving program.  The flexibility is in the willingness to adapt your evaluation methods or project plan to respond to new ideas or changing circumstances; focusing on evaluation strategies that promote the primary goal, to learn and to change, rather than capital “R” research or a “pure” methodology.  At the same time you are rigorous in ensuring your evaluation methods will produce credible findings, managing the project with timeframes, scope and budget and keeping your outcomes clear.  Being flexibly rigorous means thinking like a flowing stream constantly finding ways to go around, over or under things in your path while always flowing forward toward your goal.

Communicate, communicate, communicate

As we developed the evaluation framework, it felt like every day there is a change in plan (or a rock in the stream, to continue the metaphor).  The project officers from Larter and LMMML were regularly on the phone, discussing and developing responses to these challenges.  Our conversations were peppered with the phrase “I just want to put this in your head so you can start thinking about it…(cue current or emerging problem)”.  These approaches not only developed trust and ownership, but they also built the capacity of PIR to create the results they truly desire.

Using Evaluation Evidence to Improve Mental Health Services

Recovery evaluation can help decision-makers understand which elements of a program are working effectively and where changes may be needed. By combining quantitative evidence with stakeholder perspectives, mental health services can use evaluation insights to strengthen planning, coordination, implementation, and future investment decisions.

Conclusion

The Partners in Recovery evaluation demonstrates how structured evaluation can generate practical evidence about program implementation, service coordination, and stakeholder experiences. By combining data with the perspectives of consumers, carers, and service providers, evaluation can help decision-makers identify improvements and strengthen the delivery of integrated mental health services.

Frequently Asked Questions