Best PracticesProject Review Process

Timothy Briscoe Cristian Baccineli James Chambless

Executive Summary
Quality Assurance (QA) establishes a process by which projects are evaluated to determine whether they are compliant or at risk. The QA process uses an approved and published Project Methodology as a baseline for this evaluation. In addition, QA recommends specific action steps for projects at risk of noncompliance. QA also implements a procedure for performing a follow-up review. This review process occurs within different time frames based on project criticality and complexity. Specific procedures must be followed to ensure timely and effective review of deliverables. Quality Assurance addresses these issues by implementing a disciplined review approach that correlates to the project life cycle. The review is a layered approach, which allows the Review Team to provide advice and counsel to project teams after citing findings during face-to-face-interviews. Project Managers (PMs) assign subject matter roles to team members. During the Review Interview, team members will be responsible for answering subject matter questions based on their respective roles. Project Managers may choose not to assign roles to their team members; in this case the PM would answer all review team questions personally. QA will assign a team to conduct the review process. The roles will include the QA Review Team Leader and the QA Analyst. The organization will become involved in the review process to the level it considers appropriate. Each project is subject to two types of reviews: Standard and Special: Stakeholder Requested Review. A standard review process will consist of three phases: Initiation, Research, and Report. Two standard reviews are planned for each project. A special review maybe held if the project is non-compliant or at the request of a stakeholder. The elements of the final Report include: (1) Project Determination: Compliant or at risk; (2) Executive Summary: a one-page brief summarizing the findings; (3) Project Metrics: a report that compares the projects metrics to the PMO standards; (4) Findings: a summary of the issues found during the review process; and (5) Recommendations: actions to be taken to correct findings.

Introduction
Objective
The objective of this document is to communicate the scope, organization, and implementation detail of the QA Project Review Process (Review Process) to the QA Project Review team members, QA Project Review Leader (PRLs) and the QA Analyst (Analyst).

Terms and Definitions

Cycle of Reviews Each project will have a minimum of two Standard Reviews. Any special reviews will be held after the review process. Review Process The Standard Review has three phases: Initiation, Review, and Report. During the Initiation Phase, the review date, time, and location are scheduled with the projects Project Manager. Analysis of the projects deliverables by Review Teams QA Analyst is the primary activity in the Research Phase, which lasts approximately two weeks. The face-to-face interview will last one to four hours. A final report will be written during the third phase. Face-to-Face Interview During the Research Phase, a face-to-face interview is held that will last one to four hours depending on the complexity of the project being reviewed. The PRL leads the face-to-face interview through a list of questions. The questions and corresponding responses are stored in a Lotus Notes review database. The Review Team will then determine whether the project is labeled compliant or at risk.

Roles
The following Roles will be allocated within the Project Review Process. The Project Manager will determine who will fill these roles for the project team. One person can fill several roles or each role can be assigned to one person. QA Project Review Team Project Review Leader (PRL): Conducts the review, analyzes documents, creates interview notes to evaluate findings and makes recommendations to address risk exposures.

Proceedings of the Project Management Institute Annual Seminars & Symposium September 716, 2000 Houston,Texas, USA

 QA Analyst (Analyst): Assists the PRL with deliverable analysis and comparisons to ESTABLISHED project standards. Recorder: Responsible for taking notes during the interview. This role will likely be fulfilled by the Analyst to ensure that key ideas and discussions that may be relevant to findings, issues, or final review determination will be captured. Project Team Project Manager: Designated spokesperson for the project. Responsible for ensuring the required deliverables are in the designated location during the projects life. Testing Coordinator: Designated spokesperson for testing questions raised during the face-to-face interview. Key members of the Project Team: Designated spokespersons for other questions to be raised during the face-to-face interview.

Responsibilities
Project Team The Project Team is required to bring supporting documentation to the face-to-face interview. This documentation needs to demonstrate the rationale for their decision-making process. For example, the Project Team will need to explain the criteria used when they selected their Project Route Map, Testing Strategy, Project Plans, and other deliverables. The Project Team is required to provide precise, comprehensive and factual responses during the question and answer portion of the face-to-face interview. If questions are answered in an evasive or vague manner, the project may be evaluated as RED. The Project Team needs to ensure that the project deliverables are stored and the Project Management deliverables are stored securely throughout the projects life cycle. Project Management deliverables must be available to the Review Team. The Project Manager will be responsible for notifying project management and the project team of the review. The Project Manager will assign and schedule team members participation in the review. Once participants have been assigned, the Project Manager will inform the Review Team of the participants names and their corresponding roles. The Project Manager will ensure that all project-related materials, including the Project Plan, are current and ready for review. After the review is completed, the Project Manager must take action to address the findings and recommendations documented in the report if the project was labeled Non-Compliant. Project Review Team The Review Team will distribute the PM Project Review Guidelines and Document Review Checklist to the Project Manager once the review has been scheduled. These documents will give the Project Manager an understanding of the review process as well as a list of deliverables. This list of deliverables will identify the focus of the review analysis. The Review Team will notify the Project Manager of the review by e-mail, which will specify the confirmed date, time, and location.

The Review Team will review the project teams deliverables during a two-week period prior to the scheduled face-to-face interview. During this process, the Analysts will evaluate whether the review deliverables meet required standards. The Review Teams analysis will also evaluate whether the projects documentation is an accurate representation of the evolving deliverables. For example, If TPD (Technical Procedure Design) has just been completed, does the project plan reflect that progress in a timely fashion? During the two-week analysis period, the Review Team will be comparing project documentation and deliverables against the established Standard for Testing, as well as M Methodology for Project Management. Prior to the face-to-face interview, the PRL will analyze the information in the project review packets to identify issues for discussion during the face-to-face. The PRL will use these issues as a starting point from which to build questions for each face-toface interview. The PRL will direct the face-to-face interview. The PRL will publish a review agenda prior to the face-to-face interviews. The agenda will specify the deliverables and the discussions time frame. During the face-to-face interview, the PRL will identify issues and lead the discussion. The review will follow the distributed agenda, which consists of the PRLs questions. If findings exist, the PRL will document those findings and ask questions to clarify the underlying issues. The Review Team will inform the Project Manager of the findings to verify issues. The Project Manager will have one business day to clarify the findings. The intent of this process is to ensure that issues are clearly stated; it is not designed as a forum for negotiation, discussion, or iteration. After the review has been completed and the Review Team has declared their findings to the Project Manager for clarification, they will create their report. The report will make a determination of Red, Yellow, or Green for each project reviewed. The Analyst will distribute the Project Review Report. It will be distributed to the Project Manager and the Project Management Office. A copy will also available in the Review archives five days after the review is completed.

Description of Review Process

Introduction
The Review Team initiates the review process by scheduling reviews with the Project Manager. The Project Review Leader (PRL) and the Project Manager establish the review date. It is important to schedule the review so that it can be conducted in a manner that is not disruptive to the project itself. This is not always possible; however, the disruption a review will cause should be taken into account. The Project Review process is part of the PM methodology and as such is an integral component of effective project management.

Proceedings of the Project Management Institute Annual Seminars & Symposium September 716, 2000 Houston,Texas, USA

A Project Review Report will be generated from the project review process. Based on these findings, the project will be categorized as Red, Yellow, or Green. The projects status will indicate whether the project complies with project management standards. A second review will be scheduled for all projects. How soon and to what depth will be determined by the findings. An action plan to address risk exposures must be put in place for Yellow and Green projects. Thus, the project review process helps to identify those areas that require attention to make the project team and the project manager successful. The project reviews will be conducted in three phases that are described below in detail. Initiation Phase The Project Review team compiles an Entry Packet, which contains materials to help the PM understand the Project Review Process and that which the Project Manager needs to make available for the project review. The Project Review Leader will deliver the Entry Packet to the Project Manager in the review kickoff session. Subsequent meetings will be scheduled with the Project Manager and project team members. The entry packet will typically contain the PM Process Overview document, the PM Checklist and sample questions. Project Managers should ensure that documents are stored in their designated areas. The Project Manager will identify the Projects participants and their respective roles. Research Phase The Research Phase is divided into two activities: Documentation Review and Interview. During the Document Review, documents are retrieved for analysis to ensure that required project deliverables are in their designated location. Steps conducted in the research phase include: 1. The Analyst will create a document packet from project documentation. 2. The Review Team will conduct informal exploratory interviews with Package Coordinator, team leads, Planner, Testing Coordinator, Administative Support Assistant for the project, etc. 3. The deliverables and documentation are analyzed, and then compared against the approved Project Methodology. 4. Interview preparation activity is performed. 5. The PRL will interview Project members. 6. The Project Review will follow the PRLs agenda and review questions. 7. The review will last between one and two business days, depending on the size, scope, and complexity of the project. 8. The project is evaluated. 9. The Analyst will consolidate notes and create a preliminary set of findings and issues. If practical at this time, recommendations for preliminary findings can also be documented. 10. The Projects deliverables and documentation will be analyzed against existing standards. 11. The Review Team will document their findings during the face-to-face interview.

Report Phase A Project Review Report will be provided to the Project Manager. This briefing is to ensure that there is no miscommunication between the Project Manager and the Review Team. If differences of opinion exist, the PRL will make the final decision about what the Project Review Report includes. The Review Team will formalize the information collected during the pre-review analysis, review, and post-review findings. The PRL will determine the projects status. Recommendations will be made to the Project Manager. The Project Manager then implements the recommend changes. If the project has been determined to be at risk a follow-up review will be scheduled. The PRL will write the Project Review Report. The Project Review Report will be distributed to the Project Manager and the Project Management Office. A copy will then be stored in the Review archives. If additional information is requested by the organization due to the complexity and/or severity of the findings, the PRL will brief the organization on the results of the review.

Types of Reviews
There are four types of reviews conducted during the course of a project. The reviews are to be conducted after control points are triggered.

First Project Review

The first project review will determine if the project meets the Requirement defined as necessary to initiate a project. Project phases included in this review could include; Project Planning and Initiation (PPI), System Requirement Analysis (SRA), PM System Acceptance Testing (SAT), and Transition TRA

Completion Review
The completion review will determine that the project met all requirements, sign-offs, and deliverables defined by the project scope and that the project management process met all required standards. The Completion review will determine that all project technical, financial and contract closure events have been completed correctly.

Special Reviews
Special reviews will determine that the project meets all requirements, sign-offs, and deliverables defined by the project scope to be completed at the point in time the special review is conducted. It will determine that all project management processes are in place and meet all required standards. The special review will determine that all project technical, financial and client closure events due at the time of the review have been completed correctly. Finally, the special review may focus on a subset of activities, conditions or work products that are highlighted as at risk.

Proceedings of the Project Management Institute Annual Seminars & Symposium September 716, 2000 Houston,Texas, USA

Exhibit 1
Review Name Initial plans review Interim progress reviews Completion review Special review: non-compliant follow-up review Special review: stakeholder requested review Frequency Once As needed Once As Needed As Needed Timing of Project Review After the completion of the requirement analysis phase After the completion of each major phase After project close Within one month of any project with a negative previous review Per stakeholder request

Non-Compliant Follow-Up Review A Non-Compliant review will be held within one month of a previous review, if the previous project review found the project at risk. The type of Non-Compliant review, (R1, R2, or R3) will determine what the (RS) Special Review will focus on. If the noncompliant review were a (R1)Initial Project Review, the subsequent (RS)Special Review would examine the Project Deliverables specified in R1. Stakeholder Requested Review A stakeholder in the project (including a Project Manager) can request an ad hocor Special Review (RS). The rational for an ad hocreview can include any of the following: (1) change in Project Managers, (2) a significant issue that may effect the projects ability to deliver the required solution, and (3) project that has been classified as Non-Compliant Projectand requires a specific finding or issue addressed. A Stakeholder Requested Review will be conducted to examine the identified issue. The format of this review is similar to the standard Project Review. The stage in the projects life cycle will determine what deliverables will be reviewed.

rated between 57 for criticality with two scheduled reviews, composed of an Initial Review and an Interim Review. Category 3 projects are testing projects that have criticality levels of 810; they will have one scheduled review called an Interim Review.

Review Frequency
Review frequencies of reviews are shown in Exhibit 1. For umbrella programs that have subprojects, PMRs may be conducted on the overall program or the individual projects within the program as determined by the Quality Assurance Team.

Report
Project Status
After the review process the project will be labeled green, yellow, or red. Projects with a red label will be considered at risk.Projects with a green label will be considered compliant.Yellow project may be considered compliant or at risk, depending on the finding(s). All Major findings will result in a red label and the project will be considered to be at risk. A green label may have one or two Minor findings, but in general the project is compliant.

Review Timing and Frequency

Timing of Reviews
The types of reviews and their timing are determined by the projects attributes. The three project attributes are (1) the testing or non-testing attribute, (2) project phase, and (3) criticality. These three attributes determine the projects status. The project status determines the timing and composition of project reviews. The chart above identifies the scheduled reviews. The Special reviews are not included in the chart because conducted on an ad hoc basis. Scheduled reviews are the (1) Initial, (2) Interim, or (3) Completion Review. A Testing Project rated a 14 in criticality, that is Category 1, will have all three scheduled reviews. A NonTesting Project will have one scheduled review that will be an Interim Review. A Category 2 project will be a testing project

Recommendations
If the project has findings, the PRL will add suggestions on how the project can correct the projects findings. If the findings are Major, the recommendations will need to be implemented. Major findings will require a project to return in one month to the SpecialNon-Compliant Review. At Non-Compliant Review the PRL determine if the recommendations were implemented.

Findings
This is a list of the issues gathered during the review process. The findings will be marked major or minor.

Next Steps
The actions the Project Manager will take are after the review process is based on the projects color status, as shown in Exhibit 2.

Proceedings of the Project Management Institute Annual Seminars & Symposium September 716, 2000 Houston,Texas, USA

Exhibit 2

Project Status
Green Yellow Red None

Action Required
Depending on the nature of the findings, a follow-up review may be held within a month. A follow-up review will be held within a month to determine if the issues and findings were resolved.

Types of Reports
The Project Management Review Report will be completed and distributed within five business days of the review. It will be distributed to the Project Manager. The Project Manager may distribute copies to significant stakeholders. A copy of the report will be stored in the Quality Assurance archive; the report will be available to approved readers. This document will be marked Confidential and is intended for internal use only. The PRL may need to conduct a follow-up session with to review findings and recommendations, especially if the project has significant recommendation items that require management attention. Compliant If the project is not at risk, the abbreviated format can be used: Project Determination: Compliant: No Major Findings exist. Executive Summary: A one-page briefing that summarizes the Review Teams findings and the basis for their decision. At Risk If the project is at risk, the extended report format will be used: Project Determination: Non-Compliant: Major Findings exist. Executive Summary: A one-page briefing that summarizes the Review Teams findings and the basis for their decision. Project Metrics: A one-page report that compares the projects metrics to the published standards. Issues and Findings: A report between one and three pages in length that summarizes the issues and findings that have been found during the review process. The length of this section will be determined by the amount of issues and/or the severity. Recommendations: A report between one and three pages in length that summarizes the recommendations to be made to the project team by the PRL. Special Review: Non-Compliant Review: The next review date will be included in the Project Management Review Report. The Project Manager must take appropriate actions to address the findings. The Review Team will perform a Special Review: Non-Compliant if requested by a stakeholder. A Special Review: Stakeholder Requested Review can also be performed to address a specific issue or finding.

Return To Menu
Proceedings of the Project Management Institute Annual Seminars & Symposium September 716, 2000 Houston,Texas, USA