November 2006 - Suggested Risk Assessment Procedure for Construction Contracts
posted by Larry J Smith
Introduction
In the "August 2006 Article of the Month - MILCON Transformation" I introduced Risk Based QA as something that the Resident Engineer will use to determine which QC inspections their staff will attend based upon "risk based analysis."
We are constantly reminded that the QA process is one of evaluating and checking on the effectiveness of the contractor's CQC system through the process of selective verification inspections of the actual work.
The basic concept is to perform QA inspections before anything is covered up or completed. Too often contractor's fail and we ask ourselves "how could this have happened? Worse, the client asks "where was the QA when all of this unacceptable work took place?
MILCON Transformation
My colleagues remind me of the goals for Milcon Transformation ("MT"):
"The number one MILCON Transformation goal - cut 30% off the overall facility acquisition cycle. Using a standardized RFP and fast track design-build process will easily achieve this goal-provided that we don't adopt a "business as usual" attitude.
Using "commercial specifications", allowing the industry to provide the design and construction approach within a known funds limitation, then streamlining the execution methods, should also allow us to meet the second goal of MILCON Transformation - reducing facility acquisition cost by 15% below what it would otherwise cost.
Using performance specifications in lieu of locking in one design solution allows innovative ways to meet schedule, cost and quality goals."
"FASTER, BETTER, CHEAPER" is often associated with the goal for MT. The use of performance specifications and/or commercial specifications and "faith" in the D-B process and the D-B Contractor along with "partnering" are further defined as the keystone for success.
Some have debated that MT could be a way to reduce construction management costs. Others, me included believe MT will only increase construction management costs and also have potential to increase our risk for failure.
A document “Risk Assessment Procedure and QA Responsibility Analysis for Construction Contracts” spells out general guidelines for risk assessment. This "Article of the Month" questions the guidelines and theory that "QA Responsibilities" should be based on CCASS ratings or past performance evaluations of key personnel.
General Guidelines:
1. "Resident Engineers are responsible for providing the appropriate level of resources to enable the most efficient execution of duties, which in many cases will result in less than traditional levels of technincal input, client interface and contractor oversight."
2. "Depending on the financial risk, the schedule, and safety risk, the Resident Engineer would determine what tasks must be performed by the QA personnel."
3. "Construction QA Teams, lead by the Resident Engineer, can incorporate a process of documenting a risk assessment approach towards Quality Assurance (QA) and analyze the risk of not performing certain responsibilities."
4. "The contract risk assessment method utilizes 'Contractor Past Performance' and 'Contract Risk' ratings. These ratings are developed using Tables 1 and 2." The methodology includes using the CCASS system to acquire past performance ratings from the contractor that will be constructing the project.
5. "QA responsibility would be accomplished base on acceptable risk" and recommended results are "included in the QA Plan."
What's Wrong with the Guidelines?
In the introduction I begin with a simple fact - "contractor's fail and we're asked how could this have happened? The acquisition strategy for MT and D-B Construction typically utilizes "best value" source selection. Contractors are typically evaluated on "past performance and key personnel."
Anyone that has been a member of a source selection board knows contractors will submit exquisite resumes for key personnel and all past performance will be "Outstanding." Anything short of this results in a lower technical score or a returned submittal asking for clarification on meeting the RFP. Eventually corrections are made and seemingly all personnel are qualified and all past performance is stellar. Contractor's only submit their best ratings. Any rating below "Outstanding" is simply not included for the source selection panel or the Resident Engineer managing the subsequent D-B Contract after award. Pre-award determinations do not always flush out every rating or the source selection board did not have time to evaluate every rating.
As a Resident Engineer I've seen hundreds of CCASS evaluations evaluations that do not always address individuals by name, they do not address the team work exhibited (good or bad) and they are often "politically" influenced resulting in a "higher rating."
"Contract Risk" ratings as a function of past CCASS ratings is a good suggestion but I wouldn't bet the farm on the final determination of risk to the QA Planning process.
I'm troubled with the idea of "not performing certain responsibilities." I'm also troubled about contractors' who fail to provide the same personnel included in the "winning proposal."
I currently work for a District Engineer that expects "boots on the ground." This means both contractor and COE boots. I look at risk by asking "if this gets screwed up who will get the blame." The answer is always the Resident Engineer because he/she is in "responsible charge" and is always the first person the customer sees every day. Therefore the "Contract Risk Assessment Rating" on every contractor is "High" and every duty in the "Responsibilities Matrix" must always be performed until the contractor proves performance.
With MT I'm further reminded "that at some point, we are going to have to start adjusting our attitudes of distrust and start partnering with industry in design-build to succeed."
By assessing every contractor as "high risk" until performance is proven is a way to influence the Resident Engineers attitude.
"Faster, Better, Cheaper" is only in the eye of the beholder. Faster means something gets eliminated from the process. Some examples are UFGS Specifications, Government Review, and the very thing that the Air Force has been critical of - building commissioning. Better means quality and facility life cycle. A recent series of articles in SAME publications have studied that improving facility life cycles are a function of the contract specifications and the materials used. Cheaper or less expensive again implies something gets left out. Some examples - contractor quality control is suplemented by ICC Code inspections and DOR certifications.
Conclusion
Risk Based QA is the wrong approach to successful project delivery. The customer is paying the COE a fair amount of S&A. For that money we should not be taking any additional risk based upon the contractor's past performance or our desire to deliver projects faster or at less cost.
"Boots on the ground" are the answer. Begin by fully staffing your office to evaluate and check the effectiveness of the contractor's CQC system through day-to-day verification of every step taken by the contractor. Time and time again it's proven that absent the COE QA Representative the Contractor's CQC system will not function properly. That's that simple and only conclusion I can share about "Risk Based QA."
Comments
Please submit your comments by e-mail to Larry J. Smith, P.E. . All comments received will be posted to this article.
11/13/2006 Comments by: Manuel Fano, P.E.
1. The Suggested Risk Assessment Procedure/Risk Based QA is not mandatory. It was provided as another tool for the Resident Engineers to have available. The words used on this process gives you the indication that it’s not mandatory; i.e., “suggested”, “can incorporate a process”, “would” not shall.
2. The Resident Engineer makes the determination if he wants to use this process or not. In my opinion, the only limit the RE has on the amount of QA his office performs is the amount of S&A funds he/she has available.
3. In regards to material reviewed by Source Selection Boards, it is very important that the KO print from CCASS all evaluations for each contractor being considered, and copies be provided to the board members. The KO or the board should not make a selection based on this incomplete information provided by the Contractor. If this is happening, this process needs to be fixed. The SSB members should say something if they are not getting this information.
4. In regards to the validity of the CCASS ratings, I will say this. Garbage in, garbage out. If the Contractor Evaluations are not complete and truthful, and/or “politically influenced”, then we are hurting ourselves. This is the system we have available to get feedback on how a contractor performed in past projects. If we as government employees, are not providing truthful evaluations, then in the long run we will suffer the consequences and we will only have ourselves to blame.
5. You state that “I’m troubled with the idea of not performing certain responsibilities.” Well, do perform them all. As I mentioned before, the only limit is the S&A funds available. Again, Risk Based QA is not mandatory, just another tool available. If the RE wants to use it, but he/she wants to make some changes to it; they can. They can revise the Word file as best fits their needs.
6. You also stated that “I am also troubled about contractors’ who fail to provide the same personnel included in the winning proposal.” If this is allowed, the Contractor should be required to provide personnel of equal or better qualifications of those provided in their proposal.
7. The Corps of Engineers will always get blamed for everything that goes wrong. I don’t think this will change.
8. If you elect to use the Risk Assessment process, but always rate all the Contractors as a High Risk and that all the QA responsibilities must always be performed, then there is no need to use this process. You already had the answer without using the tables and the matrix. Again, this is not mandatory.
9. For MT to work, we will need to trust the contractor more as partners. But, trust is something that is earned. If you will rate all contractors as high risk, maybe you do need (as you say) an attitude adjustment, or a change of venue. I can’t believe you do not have Contractor’s that you can trust somewhat and give them at least a “moderate risk” rating.
10. I do not agree that “Faster means something gets eliminated from the process.” There are faster methods of construction that are considered by giving the D-B contractors the flexibility of coming up with their own design.
11. I have to disagree that “the contractor quality control is replaced by ICC Code inspections and DOR certifications.” They are not. The CQC is being augmented by Code inspections and DOR certifications. Contractor QC is still required to follow the 3-phase inspection system, perform daily inspections and provide a daily report, review and certify compliance of submittals and shop drawings, etc. There are no changes that I have seen where the Contractor would only inspect their project 5 to 15 times (based on IBC inspections).
12. You conclude that “Risk Based QA is the wrong approach to successful project delivery.” I will not agree nor disagree. The RE must form their own opinion and use this tool if they feel inclined or the need for it. You should pass on your opinion to the Navy since they use Risk Based QA on all their projects.
13. I agree that the customer is paying the COE a fair amount of S&A. Let’s use it to the extent we are comfortable with the amount of QA we need to provide the customer the best product.
Thanks for the opportunity to comment.
Manuel Fano, P.E.
Senior Regional Construction Engineer
Southwestern Division