Introduction
The Commonwealth Evaluation Policy
The Commonwealth Evaluation Policy, launched in 2021 marks a significant milestone for monitoring and evaluation of Commonwealth-funded primary health care. It is now supported by an Australian Centre for Evaluation that commenced on 1 July 2023 within the Department of Treasury.
The policy applies to all Commonwealth departments and companies (such as the Department of Health and Aged Care, the National Disability Insurance Agency, among others) subject to the Public Governance, Performance and Accountability Act 2013.
However, it is envisaged that the Policy will influence evaluation practice and principles across other Australian and state/territory organisations (such as Primary Health Networks). From a health care perspective, the policy aims to enhance the value derived from public expenditure in health, aiming for better health outcomes for Australians.
Larter applauds the Commonwealth’s evaluation policy intentions and its focus on best practice in program evaluation. We recognise that the Policy’s principles are universally applicable to all evaluation practice that aims to best support community care initiatives in the primary health sector.
Evaluation in primary health care
Evaluation plays an integral role in shaping primary health care policy, financing and services. It highlights areas of success that could be extended or replicated, and invites reflection on what did not work so well and the opportunities for improvement.
In the context of the Commonwealth Evaluation Policy, rigorous evaluation mechanisms offer indispensable insights into how well these interventions meet the needs of the public, thereby informing decisions about the allocation of resources and strategic planning. For example, in recent years we have seen the publication of the evaluation of the Health Care Homes trial, that is influencing the design of the new MyMedicare program.
A principles-based approach to evaluation
The Policy provides a principles-based approach to evaluation. The Policy encourages organisations to build an evaluative culture, characterised by leadership that values and welcomes insights gained from performance monitoring and evaluation activities. Key principles underpinning the policy are that evaluations should:
- be fit-for-purpose
- be useful to achieve their intended goals, whether this be to inform continuous improvement and accountability or to inform decision making
- be robust, ethical, and culturally appropriate
- be credible, demonstrate integrity, and be independent
Relevance of the Policy for Primary Health Care
Intersection of Policy and Practice
The Policy applies to health by mandating regular and systematic evaluations of primary health care services. The Policy, along with its companion toolkit (Commonwealth Evaluation Toolkit), provides specific guidance to assess the delivery and impact of health programs and initiatives, determining their efficacy, effectiveness, and responsiveness to public health needs. The toolkit shares robust frameworks and guidelines for conducting evaluations, ensuring that they align with the principle underpinning the Policy.
Together, the Policy and the Commonwealth Evaluation Toolkit represent the strengthening and consolidation of previous work in this space, such as the Productivity Commission’s Indigenous Evaluation Strategy. Larter fully endorses the Commonwealth Evaluation Policy, and recognises the principles underpinning it as universal for good practice in evaluation to support the primary health sector.
Success stories – impactful evaluations
There are many examples of evaluations that have been transformative, and aligned with the Policy’s principles. We provide some examples below of some that Larter has delivered within the PHN sector. Our hope is that the Commonwealth Policy leads to new evaluation success stories into the future, and that governments support and prioritise genuine evaluation practice that delivers robust and actionable insights.
- Evaluation of the early intervention speech pathology program
The Early Intervention Speech Pathology Program, funded by the Central and Eastern Sydney Primary Health Network (CESPHN) since 2017, aims to provide speech pathology services to vulnerable children and their caregivers in central and eastern Sydney. Three health providers deliver the program’s distinct models, catering to specific populations such as Aboriginal and Torres Strait Islander children. In 2022, CESPHN commissioned Larter to conduct an evaluation of the program to assess implementation, and changes in access to services for consumers.
Larter conducted a strengths-based evaluation, collecting data from service providers, CESPHN, and early learning centre staff. Through extensive interviews, we explored program implementation, outcomes, successes, and barriers with key stakeholders. We analysed program implementation data, to identify access trends, and used a program logic approach to accurately describe and assess the unique approaches taken by each provider. The evaluation findings were presented in a detailed report for CESPHN, accompanied by performance reports for each service provider. The insights generated met CESPHN’s expectations, resulting in acceptance of the evaluation recommendations and continuation of services for children and families under the program.
Our evaluation team excels in conducting participatory evaluations, collaborating with stakeholders to gain unique perspectives and generate actionable insights. Their expertise in data collection, design and analysis informs investment decisions, ensuring positive changes and enhancements to existing programs or services.
- Evaluation of the Healthy Ageing Service Response program
Larter delivered an evaluation of the Healthy Ageing Service Response (HASR) Program for Eastern Melbourne PHN (2021-23). HASR addresses the service gap for older persons’ mental health and wellbeing, and is aimed at people aged 65 years and over, living with or at risk of, mild to moderate mental illness in Residential Aged Care Facilities (RACFs) or in the community. Our 15-month mixed methods program evaluation answered questions about the extent to which HASR achieved the Quintuple Aim for primary care delivery.
We engaged with RACFs across the EMPHN catchment, engaging with staff including registered/enrolled nurses, care managers, GPs and residents and/or their carers to fully understand the program’s impact on care delivery, access to care, and patient and clinician experience among other themes. The evaluation also included a comprehensive cost benefit analysis, and provided a 5 year projection for return on investment (in dollar terms), based on reduced ED presentation, hospital bed days saved, and reduced carer, resident, and RACF costs.
The final report and knowledge products (videos, graphics) met the PHN’s needs, and recommended changes from evaluation were incorporated into the service.
Larter’s approach to evaluation
Larter takes an innovative and comprehensive approach to evaluation that is in line with the Australian Commonwealth Evaluation Policy. Larter specialises in delivering utilisation focused participatory evaluations. We utilise evaluation methodologies that leverage both qualitative and quantitative data, recognising the inherent value of diverse data sources in painting a complete picture of health service delivery. Larter’s evaluation team has deep expertise in delivering evaluations across diverse contexts, and at all levels of the commissioning cycle, including economic evaluation, and rapid evidence reviews.
We prioritise consumer and provider involvement at all stages of the evaluation process, ensuring that evaluations are both informed by and responsive to the needs of the community and of providers that deliver services. Furthermore, we emphasise the importance of using evaluation findings to drive strategic decision-making and improve health outcomes.
Talk to us if you would like to explore how we can support your organisation in evaluating your primary care programs and initiatives, incorporating Commonwealth Evaluation Policy principles to measure outcomes and generate learnings.