What Is Indigenous Evaluation?

Indigenous evaluation is an approach to assessing policies and programs that considers the perspectives, priorities, knowledge and experiences of Aboriginal and Torres Strait Islander people. Effective Indigenous evaluation should involve Aboriginal and Torres Strait Islander people throughout the evaluation lifecycle, including planning, design, data collection, interpretation and reporting. The Australian Government’s Indigenous Evaluation Strategy provides a framework for improving the quality and usefulness of evaluations affecting Aboriginal and Torres Strait Islander people.

Evidence is critical to designing and improving policies and programs that improve the lives of Aboriginal and Torres Strait Islander people. Yet we know that, nationally, Aboriginal and Torres Strait Islander people have historically had limited input into the processes of evaluation planning, implementation and reporting.

Professor Maggie Walter, quantitative Indigenous researcher, University of Tasmania: “… evaluations only superficially reflect Aboriginal and Torres Strait Islander people’s perspectives because input is only sought at the participant level. By that time, important decisions have already been made about the evaluation process what to evaluate, how to evaluate, the evaluation logic, key outcomes and objectives. Indigenous voice is denied and the current policy mindset is perpetuated. We need Indigenous voice up-front and helping to make decisions from the outset”. (sub. 112, p. 2)

Recognising the link between evaluation and better outcomes, the Australian Government has recently committed to improve the evaluation of policies and programs affecting Aboriginal and Torres Strait Islander people.

As a result, a new Indigenous Evaluation Strategy (2020) has been developed to be used by all Australian Government agencies for policies and programs affecting Aboriginal and Torres Strait Islander people. The Strategy is designed to improve the lives of Aboriginal and Torres Strait Islander Australians by having policy and program decisions informed by high‑quality evaluation methods, processes and evidence that are relevant and meaningful.

The strategy focuses on:

1. Delivering better evaluations: How can we improve the quality of evaluations to make them more credible?
2. Delivering more useful evaluations: How can the findings from evaluations be used more effectively to improve policies and programs?

Culturally Appropriate Evaluation for Aboriginal and Torres Strait Islander Communities

Culturally appropriate evaluation requires more than applying standard research methods to Aboriginal and Torres Strait Islander communities. Evaluation planning should consider cultural context, community priorities, ethical general practice support and how information will be collected, interpreted and shared. The Indigenous Evaluation Strategy emphasises meaningful involvement of Aboriginal and Torres Strait Islander people and communities across evaluation activities, helping ensure that evaluation findings are relevant, credible and useful.

At its core, we need to continue to improve our methods to embed Aboriginal and Torres Strait Islander people and perspectives at the centre of evaluation, and throughout its life cycle, from planning and design through to implementation and reporting.

We need to ensure that we plan and design approaches that work with and alongside Aboriginal and Torres Strait Islander people when deciding what to evaluate and how to conduct an evaluation. For example:

• Asking questions that address issues that are important and meaningful to Aboriginal and Torres Strait Islander people/communities.
• Measuring things that are important to Aboriginal and Torres Strait Islander people.
• Ensuring that research methodologies are always ethical, acceptable and culturally safe.

As an external evaluator who is commissioned to conduct evaluations, Larter Consulting always strives to use best practice in designing our methods and approaches. We are always looking at ways to improving the quality and usefulness of our evaluations, and we welcome this new strategy as a reminder and motivator.

Some of the key issues from a consulting or evaluation perspective that can sometimes be missed in designing methodology for evaluations that affect Aboriginal and Torres Strait Islander people (for reasons of efficiency or time constraints, for example), include:

• Ensuring decisions about methodology, data planning, collection and use are undertaken with Aboriginal and Torres Strait Islander, including issues of data governance and ownership, and communication of data and results.
• Ensuring that results are synthesised, translated and disseminated in (a) in accessible and meaningful ways; and (b) in forms that can be used by Aboriginal and Torres Strait Islander people to support decision making in community, in funded organisations and by funders.

Indigenous Data Governance and Evaluation

Data governance is an important consideration when evaluating Aboriginal and Torres Strait Islander health programs. Decisions about data collection, ownership, management, interpretation and communication should be considered early and in partnership with Aboriginal and Torres Strait Islander people. Clear processes can support culturally safe evaluation and help ensure findings are communicated in ways that are accessible and useful to communities, organisations and decision-makers.

Since its publication we have used the Indigenous Evaluation Strategy to develop our own Best Practice Checklist to use throughout the evaluation lifecycle when working with Aboriginal health programs or activities.

Conclusion

Effective evaluation can help organisations determine whether policies and programs are achieving meaningful outcomes for Aboriginal and Torres Strait Islander people. However, the quality of an evaluation depends not only on the methods used, but also on how Aboriginal and Torres Strait Islander perspectives are incorporated throughout the process.

A strong approach considers community priorities, cultural context, ethical research practices, data governance and the way findings are communicated and used. Involving people and communities early can help make evaluation processes more relevant and support findings that are useful for decision-making.

Larter applies a practical and evidence-based approach to evaluation, supporting organisations to assess programs, strengthen evaluation methods and use findings to inform better health and community outcomes.

Frequently Asked Questions