practice note 2: peer review - microsoft · pdf file3.2 liability and insurance for peer ......
TRANSCRIPT
Contents
1. About this Practice Note 1
1.1 Purpose 1
1.2 Development 1
2. Peer review 3
2.1 The value of peer review 3
2.2 Reviews considered to be peer reviews 3
2.3 Who can undertake a peer review? 4
3. Guidance for peer reviewers 5
3.1 Establishing the scope and objectives 5
3.2 Liability and insurance for peer reviewers 5
3.3 Ethical obligations of peer reviewers 5
3.4 Peer review and confidentiality 6
3.5 Informing the design engineer their work is being reviewed 6
3.6 Engineering judgement 6
4. Types of Peer Review 7
4.1 Concept or Strategic Peer Review 7
4.2 Regulatory Peer Review 7
4.3 Specific Peer Reviews 8
4.4 Forensic Peer Review 8
5. Conducting a Peer Review 9
6. References 11
7. Legal information 11
26 July 2017 Version 2 Draft 1
1. About this Practice Note
1.1 Purpose
A peer review is an independent assessment of an engineered asset or an engineer’s design, design process, materials
choice or assessment against set criteria. Whether as the design or assessing engineer or the peer reviewer, engineers
in any field may find themselves being part of a peer review.
Peer review plays an increasingly important role in the engineering design process, as a means of managing risk,
maintaining design quality and ensuring compliance. Peer reviews may also be requested on assessments made on
existing structures or engineered assets. Some fields of engineering, such as those involved with construction, have
well developed peer review requirements and processes. Other fields of engineering use peer reviews to a lesser
extent.
This Practice Note provides guidance to engineers, clients and other interested parties on what a peer review is and
how to undertake one. Differentiating a peer review from other types of review is important, as is differentiating
between types of peer review. The Practice Note sets out the various kinds of peer review used in New Zealand with
the aim of creating common understanding of the terms used. This guidance is generally applicable to all fields of
engineering where peer reviews are undertaken, but it does not attempt to reflect details in every circumstance.
This Practice Note differentiates between the following types of peer reviews:
Concept (or strategic) Peer Review – a preliminary stage review, which takes place before detailed design or
engineering work begins
Regulatory Peer Review –to confirm compliance of a complex or unique aspect of a design with a statutory code or
standard
Specific Peer Review – to confirm the suitability of specific aspects of a completed or partially completed design or
to review an assessment of a specific aspect of an existing structure or asset
Forensic Peer Review – usually required as a result of a failure of a completed design.
This Practice Note does not extend to:
in-house review processes that may be part of an engineering firm’s quality assurance processes
compliance reviews that may be undertaken by staff from a Building Consent Authority (BCA) to review aspects of
a design solely for compliance with the Building Code
second opinions, where an alternative design is required without review of another engineer’s work
competence reviews, where an engineer’s competence is reviewed
expert reviews required as part of a legal or disciplinary process1.
1.2 Development
The Institution of Professional Engineers New Zealand (IPENZ) has facilitated the development of this Practice Note.
This version supersedes Practice Note 02 (ISSN 1176-0907), which was published in 2003.
1 An IPENZ Practice Note on the Role of the Expert Witness is being developed.
Jul 2017 Version 2 Draft 2
The Practice Note has been prepared in accordance with standard IPENZ Practice Note procedures, which include
reporting on progress to the Engineering Practice Advisory Committee, peer review and general membership review.
This review and reporting process ensures the delivery of a robust good practice document.
Documents referred to or drawn on during the development of this Practice Note are summarised on page 11.
Jul 2017 Version 2 Draft 3
2. Peer review
A peer review is an independent assessment of an engineered asset or an engineer’s design, design process, materials
choice or assessment against set criteria. It is carried out by another engineer competent in the subject of the review
criteria. It is a professional opinion based on sound engineering analysis and assumptions, good practice, appropriate
codes and unbiased judgement. A peer review is often desirable in the design process for complex or unique aspects
of the design or where the impact of the design will be significant. It is not intended as a process for resolving design
issues, although the output can be a key source for such resolution.
Many engineers in certain industry areas, such as construction, are finding they are increasingly engaged to carry out
peer reviews. This increasing need originates from regulators who do not have the necessary in-house technical
expertise, and from clients who seek verification of compliance and their own quality assurance procedures.
For the purposes of this Practice Note we refer to:
the person who undertook the original design or assessment as the “design or assessing engineer”
the person undertaking the peer review as the “peer reviewer”
the person or organisation commissioning the peer review as the “client”.
(In many cases the client will commission the peer review for submission to a regulator. The client will also often be
the party who commissioned the original work or design.)
2.1 The value of peer review
Peer review is a powerful tool in maintaining and enhancing design quality and is an important feature of a self-
regulating profession such as engineering. Peer review helps ensure suitable processes, assumptions, decisions and
designs have been taken or made. This helps strengthen the quality of work engineers do and gives New Zealanders
confidence in the work of engineers.
A peer review will mean an engineer’s work will be scrutinised by another. When design and assessing engineers
willingly and actively participate in the process, the quality of the profession’s overall output remains at a high
standard. Wider benefits, including reduced conflicts and quicker completion, can be achieved if the peer review is
regarded as a collaboration between designer or assessor and reviewer.
In some cases a peer review may identify situations where design decisions could have been made differently. The
design or assessing engineer, and most likely their engineering firm, has the chance to hear and understand
alternative approaches to particular design challenges, making peer review an opportunity for ongoing professional
development.
When a peer reviewer is engaged early in the design of larger and more complex projects the value of a peer review is
significantly increased. Inadequate or incorrect decisions or assumptions picked up early by the peer reviewer can
save significant re-design effort.
2.2 Reviews considered to be peer reviews
This Practice Note differentiates between the following types of peer reviews:
Concept or Strategic Peer Review
Regulatory Peer Review
Specific Peer Review
Forensic Peer Review.
Each type of review is discussed in section 4.
Jul 2017 Version 2 Draft 4
In-house review processes that may be part of an engineering firm’s quality assurance processes, second opinions and
competence reviews are not considered to be peer reviews.
2.3 Who can undertake a peer review?
Before agreeing to undertake a peer review, an engineer may need to confirm their competence to the client who
wishes to engage them. This confirmation can include a CV, references, assessments made by a professional
engineering body, or a portfolio of relevant experience.
Ideally there should be no form of dependent relationship between the peer reviewer and the design or assessing
engineer. The peer reviewer should declare they are independent of the design or assessing engineer and the design
or assessing engineer’s organisation, and have no financial or other interest in the outcome of the peer review.
In some practice fields or disciplines where there are small numbers of engineers but a significant demand for peer
reviews, independence can be difficult. Design or assessing engineers may end up being the peer reviewer for other
design or assessing engineers who, in turn, peer review their work. The same situation may occur in certain locations
where local knowledge is required, or preferred, but the pool of suitable engineers is small. In these cases, the peer
reviewer should clearly identify any potential conflicts of interest prior to being engaged and consider their ethical
obligations to maintain honesty, objectivity and integrity. In some niche areas where the necessary expertise is not
available in New Zealand the client may need to engage an overseas peer reviewer.
For regulatory peer reviews, it is important that the peer reviewer is considered acceptable by the Regulator. For
Building Consent, it is advisable that the peer reviewer is approved by the relevant Building Consent Authority before
carrying out review work.
Jul 2017 Version 2 Draft 5
3. Guidance for peer reviewers
This section sets out the matters engineers in particular should consider when they are appointed to the role of peer
reviewer. Besides technical expertise, peer review relies on parties effectively agreeing the scope and objectives of the
work, acting ethically and being aware of their liability.
3.1 Establishing the scope and objectives
Peer reviews may be limited to parts of a design or may relate to an entire design or design process. Equally, if a post-
design assessment or a review of such an assessment is to be carried out then this may be limited to certain criteria
related to the built structure or asset.
The objectives and scope of a peer review need to be clearly determined and agreed by the client and peer reviewer
before the peer reviewer begins their work. There should be a contract in place between the peer reviewer and the
client, supported by a brief setting out what is and is not to be covered in the peer review. The scope should ideally be
focused on complex and unusual aspects of the design. Keeping the scope focused on key aspects where possible is
the most effective choice, especially when demand for peer reviewers is high.
For regulatory reviews, peer reviewers should be engaged, in particular, to consider alternative solutions used to show
compliance to the Building Code.
It is at the discretion of the peer reviewer to add relevant comments over and above the brief supplied by the client.
The peer reviewer may think it appropriate to provide additional comments on both positive and negative aspects of
the design. However, it is not considered appropriate for peer reviewers to comment on personal design or material
preferences.
3.2 Liability and insurance for peer reviewers
Engineers engaged as peer reviewers should consider limiting their level of liability in contract or in tort and make this
clear in the contract with the client. It is recommended that engineers limit their liabilities in accordance with industry
guidelines and by discussing appropriate levels of liability for certain high cost projects with their insurer. Under the
Consumer Guarantees Act, engineers need to be aware that there is no limit of liability where the work relates to
personal, domestic or household use.
In the event of an upheld claim related to the design work or other aspects of the design process, the peer reviewer
will be liable for relative damages apportioned by the court.
For regulatory peer reviews carried out to verify Building Code compliance, peer reviewers will have to complete a
Producer Statement (PS2 Design Review). The IPENZ/ACENZ/NZIA PS2 form states a maximum amount of damages
payable to the BCA in relation to the work done by the peer review firm. This amount can be altered but should only
be altered for certain high cost projects.
3.3 Ethical obligations of peer reviewers
Ethical considerations can arise during peer review, particularly as the design or assessing engineer’s information is
being shared with the peer reviewer and there is the potential for differences of opinion between the design or
assessing engineer and the peer reviewer.
Every rule in the CPEng and IPENZ Codes of Ethical Conduct is relevant to all stages of peer review, including accepting
peer review work.
The IPENZ Practice Note Engineers and Ethical Obligations (2016) provides guidance to IPENZ members and Chartered
Professional engineers on how to interpret the relevant Code of Ethical Conduct.
Jul 2017 Version 2 Draft 6
Engineers concerned about their ethical obligations as the peer reviewer (including concerns about health and safety
and competence) can contact IPENZ.
3.4 Peer review and confidentiality
When reviewing another engineer’s work the peer reviewer must agree to maintain confidentiality of that design
work or any information relating to it that may be available as part of the review process. When considering disclosure
of information obtained during a peer review, peer reviewers must take account of their ethical obligations to
maintain confidentiality under the Chartered Professional Engineers’ Code of Ethical Conduct as well as any additional
clauses on confidentiality set out in the contract.
3.5 Informing the design engineer their work is being reviewed
It is no longer considered an ethical obligation under the 2016 Chartered Professional Engineers’ and IPENZ Codes of
Ethical Conduct for the peer reviewer to inform the design or assessing engineer they are reviewing their work.
However, it is strongly recommended that peer reviewers inform the design or assessing engineer they are reviewing
their work as a professional courtesy, provided that confidentiality obligations are maintained.
3.6 Engineering judgement
All engineering design and engineering assessment includes elements of engineering judgement – a unique personal
perspective on what is appropriate or acceptable derived from personal and professional experiences and skills.
Judgements will vary from engineer to engineer just as personal and professional experiences vary.
Codes and standards used for compliance purposes determine the minimum criteria a design must satisfy. In some
circumstances, for example, when showing compliance to the Building Code using an Alternative Solution, guidelines
are used as a basis for showing compliance. Similarly, engineering assessments may be based on guidelines where
standards or codes don’t exist. Minimum criteria set out in these standards or guidelines may not be sufficient to
meet the client’s broader requirements for the entire building or process as judged by design or assessing engineers
and peer reviewers.
Design or assessment engineers and peer reviewers may differ on the point at which those broader requirements are
met. A peer reviewer may feel their role is to err on the side of caution to a greater extent compared to the design or
assessment engineer. However, peer reviewers are encouraged to maintain a fair and reasonable approach when
judging the design, process or assessment choices made by the design or assessment engineer.
Jul 2017 Version 2 Draft 7
4. Types of Peer Review
The following reviews are widely recognised and understood as peer reviews within the design community. Some
reviews may be required at specific checkpoints in the design process of certain projects, while others can be called
for when needed irrespective of the stage of the project. It is important that there is a common understanding and
shared expectations between the client and the peer reviewer of the output, the process and the end use of the
review.
4.1 Concept or Strategic Peer Review
A confirmation of direction or concept peer review typically occurs relatively early in the design process. It may be
requested by the client or another engineer to confirm the proposed design or direction is appropriate. This is more
likely to be required when the design problem has complex elements. This review may be a lengthy process as there
may be significant social, cultural or environmental impacts to be analysed.
The purpose of a Concept Review is to check the overall concept is well conceived and understood. It will also provide
an opinion on design option identification and evaluation, risks and the conclusions drawn by the design engineer (or
team of design engineers). The review will include comments on the proposed design methods and how compliance
with pertinent regulations and laws and design codes will be achieved.
As a general guide, a Concept or Strategic Review could cover:
client objectives
social, cultural and environmental impacts
relevant regulatory framework and standards
design inputs and assumptions considered in concept drawings and a preliminary Design Features Report
load paths or processes
proposed analytical methods
proposed QA processes
Risk Register
health and safety considerations
aspects of the design that should be subject to further review.
4.2 Regulatory Peer Review
A Regulatory Peer Review is an opinion based on reasonable grounds on whether certain aspects of the design
submitted by the design engineer comply with the pertinent regulations, codes and standards. It is important to
differentiate between a formal Regulatory Peer Review and an in-house compliance check, undertaken by a BCA, for
example.
A Regulatory Peer Review is not intended to be, and ought not to be used as, an independent verification of the
details of every calculation. It should also not be used as a means to verify that design documents, such as Producer
Statements, satisfy contractual obligations.
Regulatory reviews may be requested or triggered by the regulator for several reasons including:
complexity, unusual form, use of new technology or material, or unusual risk of project (including compliance via
an alternative solution that is not well understood)
the design engineer not meeting the regulatory body’s required level of knowledge or skills criteria in the relevant
area of design
concerns about aspects of the design or design methodology.
Jul 2017 Version 2 Draft 8
A Regulatory Peer Review may also be requested by the client as a matter of course prior to applying for consent in a
complex or higher risk project.
For building consents, Regulatory Peer Reviews may be required for a number of Building Code clauses.
In addition to what is covered in a Concept Peer Review, a Regulatory Peer Review may cover:
load assessments
modelling assumptions
software model inputs and outputs
design of primary and selected secondary elements
drawings and specifications
performance specifications.
4.3 Specific Peer Reviews
A Specific Peer Review is a technical review that can be undertaken at any part of the design process to review the
suitability of one or more aspects of a completed or part-complete design, or a review of an assessment on an already
as-built structure or engineered asset. The criteria triggering a specific peer review can include cost effectiveness,
‘buildability’ or complexity of an assessment
Within the design process, a Specific Peer Review is usually requested by the client in the event of some concern
about the design or a new factor, such as a change in legislation or a physical (geological) event that has arisen since
the original design was undertaken. The scope of the review will usually be limited to certain elements or aspects of
the design. A Specific Peer Review of an assessment of an already built structure or engineered asset can be requested
by the client if the structure is complex or there have been differences in opinion in previous assessments.
The methodology of a Specific Peer Review is similar to a Regulatory Peer Review except it is not necessarily being
undertaken for compliance purposes. In addition to what is covered in a Regulatory Review, a Specific Peer Review
may cover:
economic viability
buildability
environmental impact
innovation risk
seismic capacity.
4.4 Forensic Peer Review
A Forensic Peer Review may be part of an investigation and is typically requested when an examination of the
performance of a completed or as-built project is needed following an engineering failure or dispute. A Forensic Peer
Review could be requested by a client, a third party, a regulatory body or a professional body and can be carried out
by either a single engineering “expert” or a panel of experts to examine the failure of the structure or built asset.
The focus of this type of peer review is more on the likely performance and compliance of the completed work rather
than the design analysis and calculation. It may consider a repeated design that occurs in several existing structures or
built assets.
The peer reviewer(s) in this type of review will not be determining culpability for the failure or dispute, but the review
they produce may be used as a basis for litigation or disciplinary action.
Jul 2017 Version 2 Draft 9
5. Conducting a Peer Review
This section outlines the sequential steps for establishing, undertaking and completing a peer review.
1. The client identifies the need for a peer review and the broad scope of the review. The client then identifies a
competent peer reviewer, ensuring the peer reviewer is acceptable to the Regulator where a Regulatory Peer
Review is being undertaken.
2. The peer reviewer makes the client aware of any potential conflicts of interest.
3. The client and peer reviewer agree on a brief that sets out:
o the purpose of the peer review
o objectives for the work (those given to the design or assessing engineer)
o scope of the peer review
o cost and expected timeframes for the peer review
o lines of communication between the peer reviewer and the client
o reporting schedule (where interim reviews are expected)
o any barriers that may prevent aspects of the peer review being carried out, such as known
limitations in obtaining necessary documentation
o the limit of the peer reviewer’s liability
o recipients of the review.
4. The client and peer reviewer enter into a contractual agreement, which includes the agreed brief.
The Short Form IPENZ/ACENZ contract should be suitable unless the peer review is complex. Other
agreements such as the American Society of Civil Engineers’ E-581 Agreement between owner, design
engineer and peer reviewers for peer review of design may be appropriate where the IPENZ/ACENZ SFA is not
sufficient.
5. The peer reviewer undertakes the peer review in accordance with the brief agreed with the client. As a
professional courtesy, it is recommended that the peer reviewer informs the design engineer at this stage
that they have been contracted to undertake a peer review of the engineer’s work. The peer reviewer may
also want to ask the design or assessing engineer for relevant documentation and agree how further requests
for information and feedback of matters should be managed between them.
6. During a Concept or Strategic Peer Review, Regulatory Peer Review or a Specific Peer Review it is common for
there to be considerable communication between the Peer reviewer and the design engineer. There may be
little or no communication for a Forensic Peer Review depending on any specific directives, such as legal
constraints.
It is recommended that the peer reviewer maintain a log of queries and responses between themselves and
the design or assessing engineer. Maintaining a log may be a requirement for some regulatory bodies. The log
should be maintained until each matter raised has been resolved and retained according to the engineer’s
internal QA procedures.
7. Upon completion of the peer review, the peer reviewer reports to the client and should send a courtesy copy
of their report to the design or assessing engineer having obtained the client's consent. (Providing a courtesy
copy to the design engineer may not be possible for a Forensic Peer Review due to legal constraints.) The
review report should:
o specify who is entitled to rely on the report and under what circumstances
o state the scope and purpose of the report
o describe what has and hasn’t been reviewed
Jul 2017 Version 2 Draft 10
o provide key findings
o include disclaimers and qualifying statements for
work not undertaken
matters requiring further investigation
reliance on information provided by others
assumptions made.
8. A Producer Statement (PS2 Design Review) should also be completed where a regulatory review has been
undertaken. Care must be taken to clearly define the scope of the design review when completing the PS2.
Jul 2017 Version 2 Draft 11
6. References
The following resources were used in preparation of this Practice Note, or referred to in it:
American Society of Civil Engineers (2011). E-581 Agreement between Owner, Design engineer, and Peer reviewers for
Peer Review of Design. Available at http://www.asce.org/templates/contract-document-product-detail.aspx?id=6672.
Coleman, J (2016). Govt initiated inquiry into Havelock North water: Beehive. Available at:
https://www.beehive.govt.nz/release/govt-initiated-inquiry-havelock-north-water
Institution of Professional Engineers New Zealand (2016). Code of Ethical Conduct: What you need to know. Available
at: https://ipenz.nz/home/professional-standards/ethical-conduct/code-of-ethical-conduct
Institution of Professional Engineers New Zealand (2014) and Association of Consulting Engineers New Zealand.
Practice Note 1: Guidelines on Producer Statements. Available at: https://www.ipenz.nz/home/news-and-
publications/news-article/practice-note-guidelines-on-producer-statements-2014
Institution of Professional Engineers New Zealand and Association of Consulting Engineers New Zealand (2016). Short
Form Agreement for Consultant Engagement. Available at: https://ipenzproduction.blob.core.windows.net/cms-
library/docs/default-source/news-publication/engineering-practice/short-form-agrmt-apr-
2016a8a300e1c0486f9aa31eff00000785bd.docx?sfvrsn=4
Institution of Professional Engineers New Zealand, Association of Consulting Engineers New Zealand and New Zealand
Institute of Architects (2013). Producer Statement – PS2 – Design Review. Available at:
https://ipenzproduction.blob.core.windows.net/cms-library/docs/default-source/news-publication/engineering-
practice/ps2-producer-statement-2013-revised70af00e1c0486f9aa31eff00000785bd.pdf?sfvrsn=2
Registration Authority for Chartered Professional Engineers (2016). Code of Ethical Conduct. Available at:
http://www.legislation.govt.nz/regulation/public/2002/0389/latest/DLM6896801.html (Part 3 of Chartered
Professional Engineers of New Zealand Rules (No 2) 2002)
Smith, N. (2016). MBIE investigation into building performance: Beehive. Available at
https://www.beehive.govt.nz/release/mbie-investigation-building-performance
Wellington City Council (n.d.). Building control and peer review Wellington City Council Guidance Document.
Wellington: Wellington City Council.
7. Legal information
Practice Notes offer guidance to practising engineers by exploring issues of importance to the profession and setting
out good-practice methodologies. They are written by practitioners and subject to peer review by IPENZ members.
While every care is taken in their preparation, these documents are not intended to be exhaustive and are not offered
as formal advice. Practices, systems and advice may vary depending on individual circumstances, and practitioners
must exercise their own professional skill and judgement. IPENZ accepts no liability arising from their use and nothing
in the Practice Note binds IPENZ in determining the outcome of any future complaint.
Practice Notes are copyright to IPENZ and cannot be reprinted without permission.
© The Institution of Professional Engineers New Zealand Inc.