[ Previous | Contents ]

Part 3

Guidance Material

View printable pdf version of this section: EOI v2 Part3 Sections8-9,Appendix (.pdf) 410k*

* This document is in Adobe Acrobat (.pdf) format. You need to have the Adobe Acrobat Reader installed on your computer. You can download a free version from the Adobe site.


In this section:

8 Service Delivery Phase

9 Monitoring and Evaluation Phase

9.1 Continual improvement of EOI processes

9.2 Monitoring and evaluation approaches

9.3 Step 1 – Develop Monitoring and Evaluation Plan

9.3.1 Monitoring processes and performance indicators

9.3.2 Collection of data/information

9.3.3 Appropriateness of monitoring

9.4 Evaluation processes

9.4.1 Designing evaluation processes

9.4.2 Issues for evaluation

9.5 Step 2 – Ongoing monitoring and evaluation

9.5.1 Frequency of monitoring and evaluation activities

9.5.2 Changing monitoring processes

9.6 Step 3 – Amend EOI processes

Notes

Governance Group representation

Acknowledgement

Copyright

Referenced documents

Joint Australian/New Zealand standards

New Zealand legislation

Other

Related websites

Latest revisions

Review of standards

Information on New Zealand-issued documents

Appendix A - EOI ‘primary’ documents/records referenced in this Standard


8 Service Delivery Phase

Figure for Service Delivery Phase


Apart from EOI processes that are required before a service can be delivered to an individual, the Service Delivery Phase is outside of the scope of this Standard. See 7 for guidance on the operation of EOI processes.


9 Monitoring and Evaluation Phase

This section provides guidance to assist agencies to develop a formal EOI process Monitoring and Evaluation Plan.

The purpose of the Monitoring and Evaluation Phase is to ensure that, once implemented, each agency’s EOI business processes and associated outcomes remain consistent with their objectives. Agencies need to modify their EOI processes if monitoring and evaluation results indicate that:

9.1 Continual improvement of EOI processes

Just as the context and nature of the risks dictate the types of risk analysis and risk evaluation carried out, context also dictates the design and implementation of appropriate monitoring and evaluation procedures.

When carrying out EOI process evaluation, agencies SHOULD:

Monitoring, evaluation, and process improvement is an ongoing and iterative process. In essence, monitoring processes provide this information and evaluation processes analyse it in order to identify any required improvements. The following are practical examples that demonstrate the iterative nature of the monitoring, evaluation, and process cycle:

Figure 9 contains a generic monitoring, evaluation, and process improvement cycle.

Figure 9 – Monitoring and evaluation cycle

Figure 9 Monitoring and evaluation cycle



9.2 Monitoring and evaluation approaches

Identity-related risks vary between services. Monitoring and evaluation processes should be tailored to the individual contexts of each agency.

This Standard does not prescribe maximum or minimum intervals between monitoring and evaluation cycles. These decisions are the responsibility of each individual agency. However, agencies SHOULD ensure they document the basis for their approach to monitoring and evaluation and SHOULD maintain up-to-date documentation for auditing and quality assurance purposes. It is recommended that agencies review their EOI process monitoring and evaluation practices at least every two years.

9.3 Step 1 – Develop Monitoring and Evaluation Plan

A Monitoring and Evaluation Plan SHOULD be completed as part of the Design and Operation Phase before an EOI process becomes operational. If this is not practical, information about monitoring and evaluation processes SHOULD be incorporated into the business and risk management documentation that relates to the particular service.

9.3.1 Monitoring processes and performance indicators

Selecting appropriate performance indicators to measure the effectiveness of EOI processes is extremely important. Performance indicators need to inform assessments about the degree to which the EOI processes meet the agency’s EOI-related objectives.

An agency will need to factor in the following considerations when choosing its performance indicators:

Table 16 provides a list of types of performance indicators that could be used for monitoring EOI processes. It is not an exhaustive list. Agencies SHOULD select performance indicators most relevant to their specific risks and desired outcomes, and ensure that they only monitor performance against a manageable number of indicators.

Table 16 – Performance indicators

Performance indicator Example of measure

Emergence of any new EOI process risks

Analysis of discovered fraudulent activities to determine whether new modes of operation are being used to misuse and abuse identity fraud in relation to agency services.

Quality of administrative activities

Measurement of the proportion of processing errors found within EOI processes (e.g. individuals’ identities were established without the required EOI documentation being confirmed by a staff member).

Compliance with the Privacy Act

The proportion of EOI processes for identifying individuals that are found during an audit to have been in breach of the Privacy Act due to the actions of staff involved with the service.

Staff training and performance

Measurement of the results of tests of staff ability to correctly identify authentic and non-authentic documents of the type used for the EOI processes they administer.

Cost and effort associated with EOI processes

Measurement of the cost, time or other measure invested in the design and operation of EOI processes compared with the outcomes of those processes in relation to identity abuse rates or customer satisfaction.

Feedback from other agencies

Analysis of the number of instances where other agencies have reported the use of identity-related documents issued by the agency that contain errors or which are stolen or counterfeit.

Alignment between EOI processes and objectives

Measurement of any increase in the number of false identities detected per annum through implementation of an EOI process designed to better detect false identities.



The examples of performance indicators contained in Table 16 highlight the importance of identifying specific cause and effect relationships between the achievement of business objectives and the indicators being measured. If a strong cause and effect relationship exists, changes in the results of data collection will indicate corresponding changes in the achievement of business objectives. Each agency’s choice of performance indicators SHOULD enable the agency to remain informed about the degree to which its business objectives are being met.

9.3.2 Collection of data/information

Various methods can be used to collect monitoring data/information. These range from simply gathering feedback or descriptions of success or failure, to systematically gathering qualitative and/or quantitative data for statistical analysis. Collection methods include:

9.3.3 Appropriateness of monitoring

The types and amount of monitoring chosen SHOULD provide agencies with information in a timeframe and format that allows decisions about the suitability of the EOI processes to be assessed.

If an unacceptable departure from EOI processes or a mismatch between the EOI processes and objectives is detected, the monitoring processes SHOULD allow process changes to be designed and implemented before significant problems arise.

9.4 Evaluation processes

The following aspects of evaluation MUST be taken into account during the development of an evaluation plan.

9.4.1 Designing evaluation processes

Evaluation processes SHOULD allow assessment of the adequacy of EOI processes and identification of appropriate improvements. As a general rule, an agency’s EOI processes SHOULD be changed if the processes’ criteria for success are not met and the expense and/or effort required to improve the outcomes is justified. Evaluation processes SHOULD be tailored to the specific situation in which they are being carried out. These processes SHOULD be designed in conjunction with the monitoring processes for agency EOI processes.

When carrying out evaluation processes, agencies MUST document the:

Documenting these factors is important as it will assist any external review of the appropriateness of the EOI processes and helps the agency to implement and maintain appropriate EOI processes.

The design of evaluation processes is particularly important if specific interventions are to be assessed. For example, if a new business process is introduced in order to reduce the number of processing errors, the effect of the new process on the number of errors will need to be measured to gauge the success of the new process. Information will also need to be retained about the previous process so improvements can be measured. Where such changes to process are expected to affect another agency, the changes SHOULD be agreed on by all affected agencies before implementation.

9.4.2 Issues for evaluation

Table 17 provides examples of the types of issues that an agency may evaluate in relation to its EOI processes. This is not an exhaustive list. The issues an agency chooses for evaluation will need to take into account the agency context and objectives within which particular services operate.

Table 17 – Issues for evaluation

Issue Evaluation example

Solutions to address identity-related risks

Evaluation of whether new EOI checks on applications for welfare benefits resulted in an increase in the discovery rate of identity-related benefit fraud.

Implications of ongoing initiatives, such as communication with agencies about downstream effects to existing operational procedures

The discussion with other agencies of possible measures to counter EOI process concerns that have been raised. A cost/benefit analysis which considers the following general points SHOULD be carried out before the introduction of any new EOI checks:

the expected benefits that the changes would bring to the affected agencies

the expected costs and operational changes that would need to be incurred with the new EOI checks.

Performance indicators relating to those EOI checks, such as the number of discovered false identity events per year, would need to be identified and the resulting information periodically evaluated.

Possible solutions to EOI process problems identified (e.g. breaches of the Privacy Act)

Evaluation of the effectiveness of staff training courses aimed at improving staff’s compliance with the Privacy Act by measuring the reduction in the number of breaches of the Act by staff.

Adequacy of staff training and performance

The performance of staff measured by regular assessments of the quality of their EOI process-related activities (e.g. quality of document verification checks and customer liaison activities).

Possible improvements to EOI processes (e.g. efficiency gains)

An agency might identify patterns of EOI-related concerns (e.g. attempts to use non-authentic EOI documentation). As a result, the agency might aim to reduce its processing costs while maintaining or even improving its level of success in meeting its business objectives by channelling future EOI processing to staff members who have received more comprehensive training in document recognition.

Monitoring the processing costs and number of attempts to use non-authentic EOI documentation provides the agency with the information required to evaluate whether the initiative has achieved the desired outcome.

Agencies are encouraged to share the results of their evaluations if the results of their evaluations are appropriate to share and may be of use to other agencies or the Custodian of this Standard (e.g. if an agency’s evaluation activities identify changes in the nature of methods being perpetrated by individuals for the misuse or abuse of identity).

9.5 Step 2 – Ongoing monitoring and evaluation

Once an EOI process becomes operational, the monitoring and evaluation processes SHOULD commence.

9.5.1 Frequency of monitoring and evaluation activities

Monitoring and evaluation will be undertaken at different frequencies depending on the particular context within which a service exists. Monitoring and evaluation can be undertaken:

The underlying rationale for the choice of frequency is the need to keep pace with the rate of change in the data or information that is being measured, so that any unacceptable deviations from desired performance of the EOI processes are avoided wherever possible, or resolved where this has not been possible.

The frequency of monitoring and evaluation that an agency adopts needs to be influenced by both the rate at which the identity-related risks can change and the extent to which any changes are important. In many cases, a change in one of these factors also affects the other.

The intervals between evaluation cycles can be increased if the circumstances being evaluated have not materially changed since the last evaluation. Each agency SHOULD revise its evaluation processes in light of experience gained from previous monitoring and evaluation cycles.

NOTE – Episodic evaluation can be either in addition to or instead of periodic evaluation, depending on the extent to which outcomes may deviate from business objectives.

9.5.2 Changing monitoring processes

The type of monitoring SHOULD be changed if the type of monitoring currently used by an agency does not allow a timely and adequate assessment of how well EOI processes meet the agency’s EOI-related objectives.

More monitoring SHOULD be undertaken if additional monitoring of the same type is expected to yield additional information that justifies the additional expense or effort.

Until an agency establishes a good understanding of its exposure to identity-related risk and the extent to which its EOI processes address it, changes the agency makes to its monitoring regimes will usually involve changes to both the types of monitoring and the overall amount of monitoring that is undertaken. Once the agency has a good understanding of its exposure to identity-related risk and the extent to which its EOI processes address it, the agencies is likely to be more efficient at adjusting their monitoring and evaluation activities to maintain the identity-related risks associated with their services at an acceptable level.

9.6 Step 3 – Amend EOI processes

Agencies MUST consider amending EOI processes if evaluation processes indicate that EOI processes are not sufficiently reducing identity-related risk or meeting objectives.

Any amendments to an agency’s EOI process MUST be subject to the same consideration as the initial design (see 7.7) and SHOULD be fully tested before becoming part of the ongoing operation.

Notes

Governance Group representation

The following agencies/organisations are represented on the EOI Standard Governance Group:

Department of Internal Affairs

Department of Labour

Inland Revenue

New Zealand Transport Agency

Ministry of Education

Ministry of Health

Ministry of Social Development

New Zealand Bankers’ Association (Observer status)

New Zealand Police (Advisory status – identity fraud prevention)

Office of the Privacy Commissioner (Observer status)

NOTE – Some Governance Group agencies committed resources, in the form of subject matter expertise, to the establishment of a Virtual Working Group. The Virtual Working Group informed revisions to this iteration of the Standard. The State Services Commission also contributed to this process and the EOI Standard Governance Group prior to the integration of Government Technology Services (GTS) into the Department of Internal Affairs.

Acknowledgement

The Department of Internal Affairs gratefully acknowledges the contribution of time and expertise from all those involved in developing this Standard.


Copyright

Creative Commons Copyright


This work is licensed under the Creative Commons Attribution 3.0 New Zealand licence. You are free to copy, distribute and adapt the work, as long as you attribute the work to the Department of Internal Affairs and abide by the other licence terms. To view a copy of this licence, visit http://creativecommons.org/licenses/by/3.0/nz/.

Please note that no departmental or governmental emblem, logo or Coat of Arms may be used in any way which infringes any provision of the Flags, Emblems, and Names Protection Act 1981 or would infringe such provision if the relevant use occurred within New Zealand. Attribution to the Department of Internal Affairs should be in written form and not by reproduction of any such emblem, logo or Coat of Arms.

Referenced documents

Joint Australian/New Zealand standards

AS/NZS 4360:2004 Risk Management (Australian/New Zealand Standard)

www.standards.co.nz

SAA/SNZ HB 231:2004 Information Security Risk Management Guidelines (Australian/New Zealand handbook).

www.standards.co.nz

SAA/SNZ HB 436:2004 Risk Management Guidelines – Companion to AS/NZS 4360:2004 (Australian/New Zealand Standard)

www.standards.co.nz


New Zealand legislation

Adult Adoption Information Act 1985

Arms Act 1983

Births, Deaths and Marriages Registration Act 1995

Citizenship Act 1977

Civil Union Act 2004

Electoral Act 1993

Electronic Transactions Act 2002

Human Rights Act 1993

Immigration Act 1987

Land Transport Act 1998

Land Transport (Driver Licensing) Rule 1999

Ombudsmen Act 1975

Passports Act 1992

Protection of Personal and Property Rights Act 1988

Privacy Act 1993

Public Records Act 2005

Sale of Liquor Act 1989

Social Security Act 1964

Tax Administration Act 1994

Other

Border Sector Work Programme. May 2009. Identity Information Management Principles


Bradner, S. March 1997. Key Words for Use in RFCs to Indicate Requirement Levels (RFC 2119). www.ietf.org


Department of Internal Affairs. 2004. Evidence of Identity Framework. www.dia.govt.nz


ISO/IEC FDIS 19792:2009(E), Information Technology — Security Techniques — Security Evaluation of Biometrics


Office of Management and Budget. 2003. E-authentication Guidance for Federal Agencies (M-04-04). www.whitehouse.gov


Office of the Privacy Commissioner. 2002. Privacy Impact Assessment Handbook. www.privacy.org.nz


Office of the Privacy Commissioner. 1998. Fact Sheet No. 5, Information Matching. www.privacy.org.nz


State Services Commission. 2006. Authentication Key Strengths Standard. Version 1.0.

www.e.govt.nz


State Services Commission. 2006. Data Formats for Identity Records Standard. Version 1.0. www.e.govt.nz


State Services Commission. 2006. Guidance on Multi-factor Authentication. www.e.govt.nz


State Services Commission. 2006. Guide to Authentication Standards for Online Services. Version 1.0. www.e.govt.nz


State Services Commission. 2008. New Zealand E-government Interoperability Framework (NZ e-GIF). Version 3.3. www.e.govt.nz


State Services Commission. 2006. Password Standard. Version 1.0. www.e.govt.nz


State Services Commission. 2006. Security Assertion Messaging Framework. www.e.govt.nz


State Services Commission. 2004. Authentication for e-government: Best Practice Framework for Authentication. www.e.govt.nz


State Services Commission June 2008. New Zealand Security Assertion Messaging Standard. Version 1.0. www.e.govt.nz

Related websites

www.dia.govt.nz

www.legislation.govt.nz

www.privacy.org.nz

www.ssc.govt.nz

www.travel.state.gov/visa/recipocity/index.htm

Latest revisions

This Standard is to be reviewed from time to time by the Governance Group, so that it keeps up to date with changes in the sector.

Users should ensure they access the latest revisions of the NZ e-GIF authentication standards, including amendments (if any). These can be found at www.e.govt.nz. Users should also access the latest revisions of the documents included in the list of referenced documents set out in this Standard.

Review of standards

Suggestions for improvement of this Standard are welcomed. They should be sent to the Department of Internal Affairs (the EOI Standard Custodian) at:

Email: eoistandard@dia.govt.nz

Information on New Zealand-issued documents

Additional guidance material, including factsheets on specific New Zealand-issued documents, can be found on the Public Sector Intranet (PSI) subsite:

https://psi.govt.nz/evidence/default.asp



Appendix A - EOI ‘primary’ documents/records referenced in this Standard

Table A1 provides an overview of the EOI objectives that various documents can be used to satisfy. Documents/records that are used to meet Objective C (presenting person links to identity) MUST be used in conjunction with either an in-person or a trusted referee verification that the photo in the document is that of the claimant of the identity. Table A1 includes a column that indicates which of the documents can be used to provide evidence of a name change.

Table A1 – Documents used for EOI processes

Document Issuing Agency Objective10 Name Change
A B C D E

New Zealand Passport

Department of Internal Affairs (Identity Services)

red tick


red tick




New Zealand Emergency Travel Document

Department of Internal Affairs (Identity Services)

red tick


red tick




New Zealand Refugee Travel Document

Department of Internal Affairs (Identity Services)

red tick


red tick




New Zealand Certificate of Identity (issued under the Passports Act 1992)

Department of Internal Affairs (Identity Services)

red tick


red tick




New Zealand Certificate of Identity (issued under the Immigration Act 1987)

Department of Labour (Immigration)

red tick


red tick




New Zealand Firearms or Dealer’s Licences

New Zealand Police

red tick


red tick




New Zealand Birth Certificate

Department of Internal Affairs (Identity Services)

red tick





red tick

New Zealand Citizenship Certificate

Department of Internal Affairs (Identity Services)

red tick






New Zealand Death Certificate

Department of Internal Affairs (Identity Services)


red tick





New Zealand Driver Licence

New Zealand Transport Agency



red tick


red tick


18+ Card

Hospitality Association of New Zealand



11 red tick


red tick


Community Services Card

Ministry of Social Development





red tick


Electoral roll record

Enrolment Centre of New Zealand Post





red tick


IR Number

Inland Revenue





red tick


New Zealand Marriage Certificate

Department of Internal Affairs (Identity Services)






red tick

New Zealand Civil Union Certificate

Department of Internal Affairs (Identity Services)






red tick



10 A That the identity exists

    B Identity is living

    C The presenting person links to the identity

    D Presenter is sole claimant of the identity

    E Use of the identity in the community

11 The witness must not be a relative or part of the family group of the applicant; be a partner of the applicant; or live at the same address as the applicant. The witness must have a known the applicant for at least 12 months (or since birth for children under 12 months); be 16 years or over; and be a holder of a valid New Zealand Passport or from one of these groups: lawyer, teacher, minister of religion, police officer, kaumātua, registered medical professional, justice of the peace, applicant’s employer.

[ Previous | Contents ]