Regulatory system information: Public Records

Description of the regulatory system

System objectives/purposes

The public records regulatory system aims to:

  • enable the government to be held accountable by:
    • ensuring that full and accurate records of the affairs of central and local government are created, maintained and accessible
    • providing for the preservation of, and public access to, records of long-term value both before and after their transfer to Archives New Zealand
    • enhancing public confidence in the integrity of public and local authority records
  • provide a framework for records management by public offices and local authorities
  • develop and support government recordkeeping practice
  • ensure that the disposal of public records, and certain local authority archives, occurs only with the Chief Archivist’s authorization, the granting of which is a statutorily independent function.

The regulatory system should deliver, to government and the community:

  • effective and responsible stewardship of information
  • the creation and maintenance of records that can be used to hold government to account and provide evidence of rights and entitlements, now and in the future
  • the preservation of records with long-term historical or cultural value, or which contribute to New Zealanders’ sense of their national identity.

Key statutes

  • Public Records Act 2005 (PRA)
  • Official Information Act 1982, Local Government Official Information and Meetings Act 1987: provide for the Ombudsman to notify the Chief Archivist of a complaint related to the inability of a regulated party to find information or make it available. 
  • Commercial and Contract Law Act 2017: allows the Chief Archivist to approve the retention in electronic form of records originally created in non-electronic form.

Key regulations

  • None currently made under PRA.

Brief description of what the system does

This regulatory system is a key component of the constitutional framework.  It supports democratic and accountable government and the rule of law by ensuring the creation of, and accessibility to, a full and accurate record of government.  It therefore requires:

  • public offices and local authorities to create and maintain full, accurate and accessible records, as part of normal, prudent business practice, to enable the government to be held accountable
  • the preservation of, and public access to, records of long-term value to enable the government to be held accountable.

To support the purpose of enabling government accountability, the regulatory system requires: 

  • the Chief Archivist to act independently of the Minister and chief executive when exercising professional judgement in core regulatory functions, notably authorizing records disposal, issuing standards, commissioning audit, and monitoring and reporting 
  • public offices to gain prior authorization of the Chief Archivist for records disposal
  • public offices to transfer 25-year-old records to Archives New Zealand, unless this default setting has been brought forward or deferred with the agreement of the Chief Archivist
  • public offices to classify and provide access to 25-year-old records retained
  • the Chief Archivist to commission independent audits of each public office’s recordkeeping practices at least every 10 years
  • annual reporting of the result of audits and on the state of government recordkeeping
  • local authorities to manage and dispose of gazetted protected records under the Chief Archivist’s direction.

Archives New Zealand develops and implements a system to:

  • maintain oversight of the appraisal and evaluation of the public record
  • provide disposal authorization to enable effective management and preservation of public records
  • develop mandatory and discretionary standards about how regulated parties should meet the outcomes sought by the PRA
  • implement the mandatory transfer to Archives New Zealand of records for long-term preservation
  • provide general guidance and point advice for regulated parties
  • monitor and audit regulated parties’ performance
  • examine specific issues identified by sector knowledge, advice from regulated parties, Ombudsman referral and complaints from the public, and follow these up with advice, instructions or, potentially, prosecution
  • align with and support related regulators, complaints bodies and functional leads
  • work to build systems together with regulated parties, sector groups and, as appropriate, the vendors of information management services.

Agencies involved in the public records regulatory system

  • Department of Internal Affairs policy and delivery responsibilities under the public records regulatory system, primarily through the Archives New Zealand business group. 
  • Other Department of Internal Affairs system lead roles:  Government Chief Digital Officer; Government Chief Privacy Officer; the regulatory and legislative frameworks that govern the local government sector.
  • Externally: Office of the Ombudsman; Privacy Commissioner; Government Chief Data Steward; State Services Commission; Association of Local Government Information Managers.   

The Chief Archivist and Archives New Zealand staff participate in:

  • the Information Group, within which system leadership takes a system-wide perspective on how government can better unlock the value of information it holds on behalf of citizens
  • the Functional Lead Collective, which seeks to work collaboratively across disciplines supporting the public service to increase cohesion and alignment, reduce duplication in work programmes and approaches to stakeholders
  • general and case-based collaboration with the Office of the Ombudsman.

Regulated parties and non-government stakeholders

Main regulated parties:

  • public offices, defined as the legislative, executive, and judicial branches of the Government of New Zealand, including state and integrated school boards of trustees, numbering approximately 2,400
  • local authorities, defined by the Local Government Act 2002
  • non-government entities and individuals when subject to the PRA, including its offence provisions. 

Key stakeholders:

  • New Zealanders using or seeking access to public and local authority records
  • Minister of Internal Affairs
  • Iwi Māori, particularly where there are Treaty Settlement-derived relationships and to fulfil the legislative requirement to ensure there are processes for consultation with Māori in exercising Chief Archivist functions
  • information management professionals, ICT professionals and information architects in regulated parties
  • executive sponsors in regulated parties (nominated by their chief executives)
  • New Zealand Government Information Managers Group
  • Association of Local Government Information Managers
  • Archives and Records Association of New Zealand
  • advisory groups: Archives Council (advises the Minister) and Te Pae Whakawairua (advises the Chief Archivist)
  • private sector information management and ICT service providers, vendors and advisors.

Engagement between system agencies and regulated parties

Archives New Zealand engages with regulated parties by:

  • promulgating standards and guidance under the PRA
  • making standards, policy statements, guidance and general advice available through the Records Toolkit and providing specific advice through the rkadvice@dia.govt.nz service
  • tailoring engagement processes with the sector and the public on specific proposed regulatory changes
  • a programme of engagement with executive sponsors
  • delivering or participating in sector networks and events
  • briefing and seeking feedback from the advisory groups.

Fitness-for-purpose assessment

Reviews/assessments of the public records regulatory system

In 2017 the regulatory system was assessed against the four criteria of: effectiveness, efficiency, durability and resilience, and fairness and accountability.  The assessment was based on internal knowledge of the regime, an assessment of the recently-completed first audit programme and some survey data from the regulated sector.  A select external reference group from among regulated parties tested and largely confirmed the assessment prior to finalisation.  While the PRA was assessed as broadly fit for purpose, potential non-legislative improvements identified included:

  1. 1. a proposed programme to improve regulated party information management and a move to fully-digital record of government
  2. 2. the development of a monitoring framework to refresh understanding by the regulator and regulated parties      
  3. 3. examining information asset registers as a tool to improve awareness of information holdings by regulated parties and the regulator
  4. 4. further development of executive sponsor roles both in public and local authority sectors to raise awareness at a strategic level
  5. 5. explore better links with the Office of the Ombudsman. 

Work continued on items 2-5 during 2017 and 2018 within current resources, with a proposal to fund item 2 not able to be met from within the Department’s baselines for 2018/19.  Item 1 led to the 2018 public engagement process to test proposed components of a programme to improve key aspects of the regulatory system.  The engagement process used an open online engagement tool and gained worthwhile responses, mainly from regulated parties.  As a result, the work programme is now focusing on:

  • developing and trialling a new relationship management model
  • designing, testing and rolling-out the monitoring framework (including audit)
  • designing and implementing a new model for disposal authorities and
  • a technology-focused research programme. 

Proposals to enable this work to continue effectively during 2019/20 and beyond, subject to funding being available, were also developed during 2018.  It is also intended to take steps to examine how information management can better support Māori-Crown relations, for example, by seeking to create consistently applied metadata that allows information to be created and used in a way that enhances access under a Māori world view. 

Review/assessment findings

Effectiveness

The extent to which the system delivers the intended outcomes and impacts

The purpose and principles of the regulatory system are clearly articulated in the PRA and key documents.  Archives New Zealand’s organisational approach supports the purposes of the PRA.

The current absence of a monitoring framework means that assessments of effectiveness rely on individual cases and issues identified by other regulatory systems, rather than by directly relevant and comprehensive performance and compliance data.  The overall effectiveness of the regulatory system in delivering the outcomes sought cannot be fully assessed and patterns of improvement or deterioration cannot be clearly seen. 

The tools and systems for creating and communicating information are often adopted by regulated parties without the accompanying components to allow effective information management.  Regulated party compliance with core requirements is unlikely to be improving significantly for this reason, which increases the size of the legacy of information that will require retrospective management. 

The benefits of the regulatory system are usually invisible to the beneficiaries.  When government services are successfully delivered or individual rights are protected, the part played by full, accurate and accessible records is seldom recognised.  Information and records management failings are often described in terms of their effects, for example, as privacy breaches.   

Efficiency

The extent to which the system minimizes unintended consequences and undue costs and burdens

The lack of systematically collected performance information means that a full assessment of efficiency is not possible.  However, the regulatory system’s principles are sound.  These encourage active information management based on a clear strategy.  As public bodies, regulated parties should accept information management compliance as an inherent part of undertaking public business.  The PRA’s design is compatible with upholding the unique status and needs of some bodies within the mandate, for example the independence of the Courts.

The costs of non-compliance are usually diffuse and seldom have an attributed financial value.  An example of costs would be the size of teams dedicated to servicing official information requests because ineffective information management means regulated parties cannot readily find information to fulfil requests.  This also undermines the transparency and effectiveness of processes of accountability for current decisions.  Full and accurate records support intergenerational accountability processes, such as the Royal Commission of Inquiry into Historical Abuse or Treaty Settlement processes.

The lack of archival repository space in Wellington is preventing physical archival transfer from some public offices.  Only a few public offices are ready to make transfers of born-digital records, and Archives New Zealand’s ability to receive them is sometimes restricted by aging current systems.

Durability and resilience

How well the system copes with variation, change and pressures

The PRA encompass all information formats.  The standard setting, advice and monitoring provisions allow flexible evolution of the regulatory system.  A differentiated regulatory approach, by factors such as risk or information value, is also supported, as would be standards to support data access or algorithmic transparency. 

The regulatory system needs to address the results of not having information management capability and capacity built into the design, procurement and implementation of regulated parties’ information technology solutions.  This has created a legacy of under-managed information.  Better monitoring would reveal these gaps so that remedial action can be taken. 

The speed of change in information technology heightens the regulatory challenge, but there is no reason to consider that the current regulatory approach to ensuring the integrity and accessibility of the record of government is the wrong design.  Similarly, the regime appears adequate but is largely untested in regulating information needed to support joined-up services that delivered by multiple regulated parties.   

Some PRA tools have not yet been tested to their fullest extent.  For example, consistent and public performance reporting has not yet been achieved.  Lessons from the first audit programme indicate that the next programme will be more useful if it is more searching and better focused, with more specific feedback for and follow-up with regulated parties. 

Fairness and accountability

How well the system respects rights and delivers good process

Procedural fairness and partnership are built into regulatory transactions.  Regulated parties and complainants are appropriately involved when cases of possible non-compliance are being considered.  Key regulatory documents are developed in cooperation with regulated parties, notably disposal authorities, which are granted by the Chief Archivist and empower lawful disposal of records by either transfer to Archives New Zealand or destruction.  Proposed disposal authorities are available online for public comment for at least 30 days.  Email lists are maintained to notify stakeholders of publication.  Targeted communication occurs where important sector stakeholders are unlikely to be otherwise aware of the process. 

The PRA also requires consultation on any mandatory standards, with non-mandatory standards and other guidance often being developed in cooperation with regulated parties or in light of their comment.  Archives New Zealand also seeks to involve other stakeholders in these processes on a case-by-case basis.  The mandatory Information and records management standard and policy statements, guidance and advice are readily accessible through the Records Toolkit

Archives New Zealand seeks views on important issues from existing stakeholders and the general public.  Notable recent examples include the 2017 consultation on the Archives 2057 Strategy and the findings of the 2018 Regulatory Programme Engagement to test proposed components of the work programme. 

Plans for regulatory and operational improvements

Amendments to the PRA are not currently being considered.  It is a useable statute that is broadly fit for purpose.  Its regulatory tools can be tested further to deliver on its purpose. 

Several initiatives are underway to improve the effectiveness of the regulatory system as part of the Archives New Zealand Transformation Programme.  The continuation of these initiatives in 2019/20 and beyond is subject to funding being available.  The main initiatives are:

  • Archival Integrated Management System (AIMS) – This key project will provide a fit-for-purpose system to support the description, preservation and accessibility of archives held by Archives New Zealand.  AIMS will also be the tool for hosting and applying disposal authorization.  The project will replace the current Archway system, which carries a risk of failure that could close down access to archives in the worst case.
  • Preserving the Nation’s Memory (PtNM) - The development of proposals that include additional archival storage space to allow the resumption of physical archives transfers in Wellington has proceeded during 2017/18 and 2018/19.  The current transfer suspension puts records that should have been transferred at risk if they are held in storage not of archival standard.  It is unlikely that physical transfers will resume until 2024.
  • Monitoring framework – Archives New Zealand needs a monitoring framework that supports public reporting on how well government is managing information.  Consistent, regular, long-run monitoring data will help to identify regulated parties that need to lift performance and inform the selection of regulatory tools.  Performance improvement will be stimulated by the public availability of better performance information, for example, through the annual report on audits and the state of government recordkeeping.  The framework includes the next recordkeeping audit programme.  Development is to be completed by mid-2020, with audits commencing in 2020-21.  The development of the framework is a commitment under New Zealand’s Open Government Partnership National Action Plan
  • Disposal transformation – Authorized disposal of records is a central component of the regulatory system that supports accessibility to current information and the preservation of and access to information that should be available long-term.  A project is being scoped with the goal of increasing the coverage, implementation and effectiveness of disposal authorization.  AIMS and more useable disposal processes are mutually supportive, and a resumption of physical transfers in Wellington rests on PtNM. 
  • Relationship management – a conceptual framework for more effective and deliberate relationship management to support work with regulated parties has been developed and is awaiting implementation planning. 
  • Ministers’ recordkeeping – Archives New Zealand is contributing to the Department’s work to improve the processes and support available to Ministers and their offices to ensure that they can readily meet their statutory responsibilities relating to information management in an inherently complex environment.  Archives New Zealand intends to develop updated disposal authorization for Ministers’ public records in 2019.
  • Māori-Crown relations – work has commenced on how information management can better support Māori-Crown relations, for example, by seeking to create consistently applied metadata that allows information to be created and used in a way that enhances access through a Māori world view.