Pillar 3: Increasing capability

We invest in staff capability as the foundation for effective evaluation.

3.1. Evaluation maturity

Principle 3.1: We understand our current evaluation maturity and target areas for growth.

This strategy is supported by a maturity matrix outlining the characteristics of an evaluation practice at 4 levels of maturity: beginning, developing, embedded and leading. The below table illustrates these levels of maturity at a high level. The detailed maturity matrix supports a consistent understanding of departmental evaluation maturity.

At the start of this strategy, the department's overall evaluation maturity is at the 'developing' stage, with aspects of 'embedded' practice. We identified this level based on department-wide evaluation processes and a survey of departmental monitoring and evaluation capability.

Levels of evaluation maturity

Adapted from Australian Evaluation Society seminar, *Developing and implementing an effective evaluation maturity model* (May 2023)
Beginning	Developing	Embedded	Leading
Evaluation is not well understood. Evaluative practices are underdeveloped. Evaluation is ad hoc and not planned. When evaluation is done, it delivers limited benefit to the department or its stakeholders.	There is a general understanding of the role of evaluation. Evaluative practices are growing, but inconsistent. There are examples of good practice, but the department and its stakeholders do not get the full benefit of evaluation.	Evaluation is largely integrated in business functions. Evaluative practices are established and consistent. The department commissions and conducts evaluation well and strategically builds and uses its evidence base.	Evaluations present evidence and insights on impact and change. Evaluative practices are exemplary. The department and its stakeholders benefit greatly from our evaluation activity. Others regard the department as a leader in this field.

Our strengths

Our staff understand the value of evaluation. They understand its role in delivering efficient and effective initiatives that build a better future for all Australians in line with our departmental purpose.
Some monitoring and evaluation processes are embedded in stages of the policy and program lifecycle.

Our challenges

APS requirements change over time. The department will remain flexible in our approach to respond to shifts in the broader evaluation landscape.
Evaluation can be resource-intensive. The department will plan fit-for-purpose evaluation activities and manage our resources strategically.
Change is constant, and building evaluation capability is an ongoing exercise.

Increasing our maturity and capability

Developing and maintaining evaluation maturity is an ongoing process that is balanced with other organisational objectives. It requires continued investment.

Given our finite resources and current maturity, we need to lift our efforts to develop staff capability. We must move towards a widespread understanding and appreciation of the role and value of evaluation throughout the department.

We will assess our maturity across the 3 pillars every year. This will provide insights about our strengths and challenges and ensure our efforts adapt to the changing needs of staff and the department.

By 2028 we want to be at a ‘leading’ level of maturity. This is an ambitious target. Reaching an exemplary level of evaluative practice and fully embedding evaluative thinking into everything we do is a cultural shift. It will take time and ongoing commitment from staff and leadership.

'Leading' is not just about having a consistent framework for evaluation – it is about how:

evaluation is used and resourced in practice
its value is perceived
it is communicated both inside and outside the department
the evidence from evaluation is strategically and systematically applied to inform organisational learning.

3.2 Supporting the department to build the right skills

Principle 3.2: Evaluation is most effective when the whole organisation understands its value and embraces shared responsibility for integrating evaluative practices.

Increasing the department's evaluation capacity and capability requires all areas of the department to implement and support evaluative practices. To support this strategy, the whole department will need to:

commit to undertaking evaluations scheduled for the initiative
ensure decisions on evaluation activities are made in line with the strategy and the Commonwealth Evaluation Policy
apply evaluative practices throughout the life of an initiative
share and implements lessons learned from evaluation.

To do this, our people need the right skills and support.

The Evaluation Unit provides ongoing support and advice on evaluating policies and programs. To further build the department’s skills, the Evaluation Unit will deliver several capability-building activities, including:

raising awareness of the available guidance and support
developing an online self-service toolkit for staff to access guidance and templates
developing internal training sessions and workshops and identifying relevant external training
setting up a community of practice to share knowledge and encourage collaboration.

There may be opportunities to use emerging technologies (such as AI) as tools in conducting evaluations, for example through streamlining evidence synthesis. The Evaluation Unit will continue to monitor and engage with potential opportunities to use improved technologies over the coming years. It will do this in line with relevant government guidelines and consider potential ethical implications.