Pillar 3: Increasing capability

We invest in staff capability as the foundation for effective evaluation.

3.1. Evaluation maturity

Principle 3.1: We understand our current evaluation maturity and target areas for growth.

This strategy is supported by a maturity matrix outlining the characteristics of an evaluation practice at 4 levels of maturity: beginning, developing, embedded and leading. The below table illustrates these levels of maturity at a high level. The detailed maturity matrix supports a consistent understanding of departmental evaluation maturity.

At the start of this strategy, the department's overall evaluation maturity is at the 'developing' stage, with aspects of 'embedded' practice. We identified this level based on department-wide evaluation processes and a survey of departmental monitoring and evaluation capability.

Levels of evaluation maturity
Adapted from Australian Evaluation Society seminar, Developing and implementing an effective evaluation maturity model (May 2023)
­Beginning Developing Embedded Leading
  • Evaluation is not well understood.
  • Evaluative
    practices are underdeveloped.
  • Evaluation is 
    ad hoc and not planned.
  • When evaluation 
    is done, it delivers limited benefit to the department or its stakeholders.

 
  • There is a general understanding of the role of evaluation.
  • Evaluative practices are growing, but inconsistent.
  • There are examples of good practice, but the department and its stakeholders do not get the full benefit of evaluation.

 
  • Evaluation is largely integrated in business functions.
  • Evaluative practices are established and consistent.
  • The department commissions and conducts evaluation well and strategically builds and uses its evidence base.

 
  • Evaluations present evidence and insights on impact and change.
  • Evaluative practices are exemplary.
  • The department and its stakeholders benefit greatly from our evaluation activity.
  • Others regard the department as a leader in this field.

 

Our strengths

  • Our staff understand the value of evaluation. They understand its role in delivering efficient and effective initiatives that build a better future for all Australians in line with our departmental purpose.
  • Some monitoring and evaluation processes are embedded in stages of the policy and program lifecycle.

Our challenges

  • APS requirements change over time. The department will remain flexible in our approach to respond to shifts in the broader evaluation landscape.
  • Evaluation can be resource-intensive. The department will plan fit-for-purpose evaluation activities and manage our resources strategically. 
  • Change is constant, and building evaluation capability is an ongoing exercise. 

Increasing our maturity and capability

Developing and maintaining evaluation maturity is an ongoing process that is balanced with other organisational objectives. It requires continued investment. 

Given our finite resources and current maturity, we need to lift our efforts to develop staff capability. We must move towards a widespread understanding and appreciation of the role and value of evaluation throughout the department.

We will assess our maturity across the 3 pillars every year. This will provide insights about our strengths and challenges and ensure our efforts adapt to the changing needs of staff and the department.

By 2028 we want to be at a ‘leading’ level of maturity. This is an ambitious target. Reaching an exemplary level of evaluative practice and fully embedding evaluative thinking into everything we do is a cultural shift. It will take time and ongoing commitment from staff and leadership. 

'Leading' is not just about having a consistent framework for evaluation – it is about how:

  • evaluation is used and resourced in practice
  • its value is perceived
  • it is communicated both inside and outside the department
  • the evidence from evaluation is strategically and systematically applied to inform organisational learning.

3.2 Supporting the department to build the right skills

Principle 3.2: Evaluation is most effective when the whole organisation understands its value and embraces shared responsibility for integrating evaluative practices. 

Increasing the department's evaluation capacity and capability requires all areas of the department to implement and support evaluative practices. To support this strategy, the whole department will need to:

  • commit to undertaking evaluations scheduled for the initiative
  • ensure decisions on evaluation activities are made in line with the strategy and the Commonwealth Evaluation Policy
  • apply evaluative practices throughout the life of an initiative
  • share and implements lessons learned from evaluation.

To do this, our people need the right skills and support. 

The Evaluation Unit provides ongoing support and advice on evaluating policies and programs. To further build the department’s skills, the Evaluation Unit will deliver several capability-building activities, including:

  • raising awareness of the available guidance and support
  • developing an online self-service toolkit for staff to access guidance and templates
  • developing internal training sessions and workshops and identifying relevant external training
  • setting up a community of practice to share knowledge and encourage collaboration.

There may be opportunities to use emerging technologies (such as AI) as tools in conducting evaluations, for example through streamlining evidence synthesis. The Evaluation Unit will continue to monitor and engage with potential opportunities to use improved technologies over the coming years. It will do this in line with relevant government guidelines and consider potential ethical implications.