Evaluation complements, and can inform, or be informed by, several other assurance activities. These assurance activities are some of the key mechanisms to help ensure that policies:
- are well defined and informed
- are executed successfully
- are delivered within constraints
- achieve their desired effects.
However, there are some key differences in scope, purpose and process.
Policy impact analysis
Policy impact analysis gives decision makers information on the potential economic, social and community impacts of a policy. It must demonstrate the proposal will deliver overall benefit to the community. It helps ensure policy options are well-designed, well-targeted and fit-for purpose.
Impact analysis (IA) can use a range of quantitative and qualitive tools, including cost–benefit analysis, stakeholder engagement and compliance costs calculations. Some large programs may also require a post-implementation review.
IA is required for 'any policy proposal or action of government, with an expectation of compliance, that would result in a more than minor change in behaviour or impact for people, businesses or community organisations' (Office of Impact Analysis 2023).
The Office of Impact Analysis manages this process and determines the level of IA required for each proposed policy. More information is available in the Australian Government Guide to Impact Analysis. DISR's Analysis and Insights Division also supports IA.
IA will inform future evaluations, with a specific requirement to set out how the policy will be monitored and evaluated. An evaluation may assess if the objectives and impacts set out in the IA were realised or if there were other impacts or consequences. The IA will identify the metrics for measuring success and any data gaps (and how they will be closed to support future evaluations).
Whole-of-government impact assessment
Policy impact analysis (described above) is one type of whole of government impact that must be assessed for all new policy proposals (NPPs). In addition to IA, which is focused on economic, social and community impacts, all NPPs must include a whole-of-government impact assessment. This assesses the potential impacts of a proposed policy on other cohorts or areas of focus, including gender equality and First Nations peoples. It also assesses legal, digital and financial impacts.
This assessment looks at what might happen, rather than what has occurred. Further detail is in the Cabinet Handbook.
Whole-of-government impact assessment also informs future evaluation. It helps to:
- determine the timing and scope of an evaluation by identifying risks and sensitivities
- identify particular cohorts to focus on as part of assessing performance.
Audits
An internal audit is an independent review of polices and operation that helps improve governance, risk management and control processes. Internal audits often focus on operations or systems, rather than the policy intent or impact of an initiative. These may be undertaken in response to a performance audit or be requested by senior executives (referred to as ‘management initiated reviews’).
A performance audit is undertaken by the Australian National Audit Office. It is an independent assessment of all or part of an entity’s operations and administrative support systems. It identifies where improvements to public administration can be made. An audit may make recommendations on how to improve cost-effectiveness, efficiency, effectiveness and compliance.
Evaluation can complement the audit program, including managing risks that may reduce the need or scope of a future audit. The findings of an audit can inform an evaluation.
Performance measurement and reporting
In our department, performance measurement generally means the entity-level measurement and reporting obligations under the Public Governance, Performance and Accountability Act 2013 and Commonwealth Performance Framework.
Our portfolio budget statements and corporate plan set out our performance measures each year. DISR reports on these through the annual performance statements in our annual report.
Data collected by monitoring the progress of an initiative can inform performance measurement and reporting.
Initiative Support Model and Program Advisory Service
The Initiative Support Model (ISM) was introduced in DISR in 2023 to ensure robust program management and improve risk management. The ISM involves risk tiering the department’s projects, programs and initiatives.
The Program Advisory Service sets expectations for appropriate governance and assurance based on levels of risk and complexity. It then ensures that the right kind of support is provided for successful delivery.
The Program Advisory Service will prompt initiative owners to consider monitoring and evaluation and seek advice from the Evaluation Unit where appropriate.
Gateway reviews
Gateway reviews are managed by the Department of Finance and strengthen governance and assurance practices. They are used in high-risk proposals to provide independent assurance and advice to improve program delivery and implementation. As part of developing an NPP, the Department of Finance will determine whether a proposal will be subject to a gateway review.
A gateway review may provide input into an evaluation and may be a factor in determining the scope and timing of an evaluation.
External reviews
Occasionally the government may commission a review of a policy or program. This may consider its effectiveness, whether it remains fit for purpose and whether changes may be required. The department may also seek funding through the original NPP process to conduct an external review at certain points of program delivery. Parliament may also commission this type of review.
Reviews may be undertaken by the department, the Productivity Commission, an individual or a panel of experts.
The scope and tools used by the review vary, but normally draw on monitoring and reporting. They may include cost-benefit analysis and economic modelling.
A review complements evaluation. The form, scope and timing of an evaluation will also be informed by a review that has been undertaken.
Figure 1 shows how evaluation fits with other assurance activities in the department. Principles 2.2 and 2.3 provide more detail on how the interaction between evaluation and assurance activities applies to individual policies and programs.
