Procurement
Management
Survey Report

JOHNSON SPACE CENTER

February 27 – March 10, 2006

OFFICE OF PROCUREMENT
HEADQUARTERS
WASHINGTON, D.C.

PREFACE

The NASA Headquarters Office of Procurement conducted this procurement management survey at Johnson Space Center (JSC) under the authority of NASA Handbook 1101.3, The NASA Organization. The survey was conducted February 27 through March 10, 2006. The report contains the survey strengths, weaknesses, and considerations.

An exit briefing was held at JSC on March 10, 2006 to discuss the survey findings. This report serves as a basis, in part, for fulfilling internal control requirements in accordance with the Federal Manager’s Financial Integrity Act of 1982 (P.L. 97-255).

Jerry P. Edmond
Survey Program Manager
Analysis and Assessment Team
Office of Procurement, NASA Headquarters

CONTENTS

SECTION I        OVERVIEW

SECTION II       ORGANIZATION — MANAGEMENT

SECTION III        PRE - AWARD

SECTION IV        POST - AWARD

SECTION V       PRICING - FINANCIAL - AUDITS

SECTION VI       GRANTS, COOPERATIVE AGREEMENTS, SIMPLIFIED ACQUISITIONS, AND OTHER ISSUES

SECTION VII       SMALL AND DISADVANTAGED BUSINESS (SDB) UTILIZATION — This portion of the report has not been completed. It will be posted once it is available.

SECTION I

OVERVIEW

The Procurement Organization at the Johnson Space Center is providing meaningful support to their program customers. The Procurement Management Survey Team interviewed randomly selected technical and program representatives to ascertain any issues or concerns with the current processes. A significant degree of customer satisfaction is apparent from the interviews with the technical community. Additionally, interviews with numerous acquisition professionals at all levels of the JSC organization regarding the effectiveness of the procurement office were also conducted. The consensus from the procurement staff was also positive.

The interviews of technical and acquisition personnel are given roughly equal survey emphasis with a review of contracting actions focused on compliance with procurement statutes, regulations, and procedures. The thrust of the compliance portion is directed towards systemic procurement processes, as opposed to focusing on individual file anomalies. Attention was also directed to current procurement innovations, both Agency-wide and Center specific.

The results of the compliance reviews and the interviews are detailed as strengths, weaknesses, and areas of consideration. Also, to promote the exchange of successful lessons learned and innovative procurement methodologies between Centers, the team sought to identify JSC processes or initiatives that might benefit other Centers and, likewise, looked to other Centers for suggested approaches that might be exported to JSC.

The exit conference at the conclusion of the survey consisted of a direct exchange of observations and ideas between the participants. To emphasize Center ownership of the resolution of any identified weaknesses or considerations, the survey follow-up process will focus on the corrective actions or initiatives undertaken by the Center. At an appropriate interval (approximately six months after this report is issued) the JSC Procurement Officer will brief the Deputy Chief Acquisition Officer/ Assistant Administrator for Procurement and the survey team leader on Center achievements in these areas.

Below is a list of team members and the areas reviewed by each:

The survey could not have been accomplished successfully without the support from Johnson Space Center, Kennedy Space Center, and the NASA Management Office at the Jet Propulsion Laboratory, and the following individuals:

Return to Contents

SECTION II

ORGANIZATION - MANAGEMENT

1. Organization -- Management

The Johnson Space Center Office of Procurement (OP) is providing valuable support to their program customers. This is significant, since it comes at a time when the Center is experiencing program uncertainty resulting from major Agency challenges and an increase in the number of large dollar value Source Evaluation Boards (SEB). In addition, the OP has effectively responded to numerous complex requirements with professionalism. The organization is clearly designed to focus on meaningful customer service while attempting to be responsive to the demands of a dynamic workplace.

In September of 2005, the JSC OP added an Acquisition Division in order to support the Headquarters (HQ) Exploration Systems Mission Directorate. There are now five operations groups responsible for planning, awarding, and managing acquisitions and associated activities for customers at JSC. The five operations groups are the Space Shuttle Procurement Office, Space Station Procurement Office, Projects Procurement Office, Institutional Procurement Office, and the Exploration Systems Mission Directorate Procurement Office. In addition to the four offices directly supporting JSC customers, the Procurement Policy and Systems Office continues to provide guidance and focus to JSC contracting personnel on a wide range of procurement regulations, issues and concerns, with more recent focus on Source Evaluation Boards (SEB) and emphasis on the implementation of the Contract Management Module (CMM).

Since the previous survey there have been several management positions added to the Office of Procurement including positions at the team lead, deputy division manager, and division manager level. Also, the Pricing Team Lead was promoted to Deputy Division Manager in the Procurement Policy and Systems Office. Based upon this a Pricing Team Lead rotational opportunity was created in order to continue to provide valuable service to the SEBs ongoing at the Center. This phenomenon allows the Office of Procurement to have more promotional opportunities and rotational opportunities than in the past.

In the previous survey there were concerns that the Procurement Officer and Deputy Procurement Officer did not know employees below the team lead level. Based on information attained in the Procurement Management Interviews (see below) this perception no longer exists. Therefore, the Procurement Officer and Deputy Procurement Officer should be commended for taking the steps to alleviate this perception.

The number of SEBs at JSC has increased since the previous survey. During the 2004 Survey there were 7 SEBs ongoing at the Center in comparison to 12 as of this report. This increase can be attributed to expiring institutional contracts and new requirements from the HQ Exploration Systems Mission Directorate (ESMD). From information obtained during the COTR interviews (see below), the JSC Technical Offices are pleased with the level of service they receive from the Office of Procurement, even during this period of significant increase in SEBs.

Since the previous survey the Procurement Officer has begun several initiatives in order to improve the management, efficiency, and effectiveness of the Office of Procurement. These initiatives are as follows:

• Joint Leadership Team - a center-wide effort to improve the usefulness of Center management metrics;
• Contractor/Civil Servant Procurement Roundtable - this team provides a forum for training, communication and relationship building that enhances the complete NASA JSC procurement process;
• Team Lead Roles and Responsibilities - a team formed in OP to better define the role of Team Leads in the OP management process;
• Award Fee Standardization Team - a team formed to make the Award Fee process at JSC streamlined, efficient and effective;
• Career Development Roadmap - a team formed to outline a roadmap for each 1102 position to be used as a guideline for career development;

The JSC Procurement Officer is commended for her effort to improve the management, effectiveness, and efficiency of the JSC Office of Procurement through the implementation of the aforementioned initiatives.

As of this survey there are 134 contracting personnel in the 1100 classification series (including individuals classified as 1102, 1105, and 1106) compared to 130 two years ago.

*New contract awards, modifications, contract change orders, and purchase orders over $25,000.

**Excludes 3 White Sands Test Facility 1102's (as of FY05 they are part of the Projects Procurement Office.)

Overall in FY04 there were 143 employees and in FY05 there were 146 employees, including those in the 300 series.

STRENGTHS:

(1) The JSC Office of Procurement is commended for implementing their work in an effective and efficient manner in light of increasing technical requirements and demands.

(2) The Procurement Officer is commended for implementing initiatives to improve the effectiveness, efficiency and management of the Office of Procurement.

2. Procurement Management Interviews:

The Survey Team program manager interviewed a broad cross section of employees throughout various grade levels and branches in the Office of Procurement to gain insight into their perspectives on the effectiveness of the organization. While interview candidates were selected randomly, an effort was made to assure that both long time employees and newer hires were in the sample, so as to get a more representative picture of the organizational health.

The majority of the employees interviewed believed that morale was good. However, there were several employees who believe the morale is average or just above average. Most often the employees who believe morale is average usually gave two reasons lack of communication below the team lead level or high work load.

The majority of the employees interviewed believed that work was distributed fairly or in a manner that ensured the work would be executed in a timely manner. The majority of employees believed their management was supportive of them. Employees also expressed that managers valued the use of Individual Development Plans (IDPs) to determine career enhancement and training needs. Employees believed they were able to get training when needed whether it was CON courses or JSC center training. Also, the majority of employees believed there are rotational opportunities throughout the Procurement Organization.

Employees were also appreciative of the fact that managers give continuous feedback on their work products. The employees expressed that it helps them to better understand what they are doing well and what they need to improve upon.

The majority of employees interviewed believed they have learned to cope with SAP but were looking forward to the implementation of the Contract Management Module (CMM) and are hopeful that it can deliver the level of service projected.

The entry level employees interviewed appreciated their team leads as work process mentors but voiced a concern that they would like to have an opportunity to have a mentor to help them navigate their career as soon as they begin working at the Center. The JSC Center Mentor Program is only available to employees who have been at the Center for one year. The JSC Office of Procurement should consider implementing a mentor program similar to the one at Marshall Space Flight Center to help new employees navigate the JSC Office of Procurement career path.

The employees who believed morale was good overwhelmingly also believed that communication was good throughout the organization. These employees believed their team leads did a good job of informing them of issues that flow down from the Procurement Officer.

The majority of the employees interviewed appreciated the Procurement Forums. They felt that the forums were the right length and provide training that was appropriate for their jobs. They also expressed that they were appreciative of the fact that the presentations remain on-line for future reference. Finally, the employees also viewed the Forums as a form of monthly all-hands meeting since the Procurement Officer used it as an opportunity to make a brief statement at the beginning discussing issues important to the Office of Procurement.

STRENGTH:

The JSC Office of Procurement is commended for the high level of support given to their employees, the high level of communication throughout the organization, and for the emphasis it places on giving employees feedback on their work products.

CONSIDRERATIONS:

(1) The JSC Procurement Officer should ensure that Team Leads in all offices are effectively communicating with their employees.

(2) While Team Leads are effectively helping new employees understand how to do the work, the JSC Procurement Officer should consider implementing an internal Office of Procurement Mentoring Program similar to the one at MSFC in order to help new employees navigate their careers. The Career Roadmap would be an excellent tool in conjunction with a mentor to help new employees navigate their careers in Procurement at JSC.

3. Technical Customer Interviews:

The Survey Program Manager interviewed a number of employees from various project and programs offices regarding the effectiveness of acquisition support and the level of customer satisfaction.

Overall, the technical community communicated that they are satisfied with the level of service they receive from the OP. The technical community believes the OP accomplishes their work in an effective, responsive, and timely manner.

Several COTRs interviewed had just participated on SEBs. These COTRs were very satisfied with the level of service rendered by the OP during the process. However, several did voice concerns about the SEB Review Process. The technical community feels that the review process should be more coordinated. They often gave examples where the Procurement Review Team (PRT) would tell them to execute a work product one way and then it would get over turned by higher levels of management. They feel it would be helpful if upper management would express to the PRT what they needed before hand in order that work products would be approved in a more timely fashion.

The majority of the COTRs interviewed believe the staffing level for their procurement support was appropriate. However, the majority expressed that with the increase in technical requirements on the Center; there may be a future need for more personnel in the OP. One Technical Customer went as far to express that “given the level of responsibility on a Contracting Officer it would seem that they would be at a higher grade level”.

Most COTRs interviewed stated they had completed their yearly refresher training; when specifically asked about the training being required yearly at JSC as oppose to the Agency requirement of every 4 years, all expressed that they appreciated the yearly training as long as it focused on changes in the previous year in the acquisition process. The majority of the COTRs believed that the yearly training had improved since a JSC person was conducting the training. The technical community at JSC appreciates someone from the center conducting the training because they are more familiar with the center’s acquisition processes and needs than a person from the outside.

Some of the ideas expressed to improve the COTR training are as follows:

• Identify more on-line resources during the training.
• Present more discussion on what a COTR can do, not what they can’t do.
• Add a COTR mentor program to help new COTRs transition into their new COTR responsibilities by shadowing an experienced COTR.
• Hold classes in a small setting with someone with JSC experience conducting the training.

All the COTRs interviewed understood their responsibilities as a COTR and all received their NF-1634 COTR Delegation Form.

STRENGTH:

The Office of Procurement is commended for their high level of customer satisfaction given the high number of SEBs and changing requirements at the Center.

CONSIDERATION:

Office of Procurement senior management needs to develop a consistent SEB review process in order to make the review process timelier.

4. Legal Interviews:

The Procurement Survey Manager conducted the legal interviews with JSC’s Office of Chief Counsel. Legal counsel expressed that the relationship with procurement is excellent. They were impressed with how well the OP handles their SEBs and other high valued procurements.

General Counsel’s office expressed great respect for Senior Procurement Management. General Counsel stated that, “they are an exceptional management group and great people to work with.” They believed the higher level people were exceptional to work with and that they have a very collaborative relationship and “usually come to the same answer at the same time”.

However, they did express concerns with the less experienced people in the organization. General Counsel understood why they are being pushed along faster due to lack of resources, but they expressed that maybe some more training can be given to the less experienced personnel in the organization. The office expressed that the quality of work products were directly related to the level of experience of the procurement personnel. The office also expressed that there may be a need to review JOFOCs at a lower level than is currently required by the review process. (See JOFOC write-up)

Legal counsel stated that “the acquisition system is stressed because of lack of people and not having enough time.” They understood that procurement is trying to manage their organization in an appropriate way by trying to give less experienced people more work than in the past but the office stated that “the workload is getting ahead of the growth and they don’t see the level of mentoring as in the past.”

STRENGTH:

Both the Office of Chief Counsel and the Office of Procurement are commended for the strong collaborative relationship that they have developed.

CONSIDERATION:

JSC Office of Procurement should continue efforts (e.g. Procurement Forums and Mentoring Program) to improve the quality of work from their less experienced personnel.

Return to Contents

SECTION III

PRE - AWARD

Nine noncompetitive contract files were reviewed for compliance with FAR Parts 5.2 and 6.3 and NFS Part 1806.3. Dollar values ranged from $300K to $35M. All used statutory authority (c) (1) - only one responsible source. Three JOFOC’s had estimated values greater than $500K. All three contained all of the information required by FAR 6.303-2, specifically followed the FAR content listing from FAR 6.303-2(a)(1) through (a)(12), and provided sufficient rationale. Two were reviewed and approved at the appropriate level. One JOFOC greater than $10M was approved by the Center Competition Advocate instead of the Center Director as required by JSC policy.

All six JOFOC files with estimated values of less than $500K had FAR compliance issues. The issues included the following: market research was not mentioned in the JOFOC; market research was cited in the JOFOC but not included in the file; a synopsis was not included in file; a synopsis or synopsis results were not mentioned in the JOFOC; one JOFOC was approved more than one month prior to release of the synopsis; many JOFOC’s were approved prior to the end of synopsis response period, with some approved the same day or one day after release of the synopsis with statements that “no sources expressed an interest in writing; incorrect authorities were cited, for example, some cited the FAR instead of the statute and one had an incorrect FAR cite; when substantial duplication of cost was cited no further details were provided; and one JOFOC was approved two years prior to release of the solicitation with no explanation in the file.

It was noted that the JSC Procurement Homepage guidelines includes the Virtual Procurement Office (VPO) JOFOC under the standard formats/templates. It was also noted that the JSC Coordination and Approval Matrix requires Legal Office review of contracts valued over $100K and JOFOC’s valued at $500K or more.

WEAKNESS:

The JSC Office of Procurement should add more scrutiny to the JOFOC’s valued at less than $500K to ensure that the JOFOC’s adhere to the requirements for content listed in FAR 6.303-2; adhere to the publicizing and response times in FAR 5.203; and are not approved until synopsis response times have elapsed, and any expressions of interest are resolved and the results are included in the JOFOC. The JSC Office of Procurement should also encourage use of the VPO JOFOC template for JOFOC’s valued at less than $500K to help ensure all FAR requirements are covered.

CONSIDERATION:

(1) The JSC Office of Procurement should consider lowering the Legal Office review of JOFOC’s to $100K and above to be consistent with the contract reviews in order to add more scrutiny to JOFOC’s valued at less than $500K, and also consider lowering the Procurement Office Manager review of JOFOC’s to $100K and above.

(2) The Office of Procurement should provide a more rigorous review of JOFOCs that do not go through the legal review process. Also, the Office of Procurement should provide more training to its newer employees on writing a sufficient JOFOC. The Procurement Forum would be an excellent outlet to provide this training.

2. Cost Estimates

The last two surveys found that the files lacked sufficient cost estimates. A review of the cost estimates included in the current noncompetitive contract files again finds the basis of the estimates in the files to be sorely lacking. Only one file contained sufficient backup data as the basis of the estimate, while most only contained a total estimated amount with no basis. JSC Form 1219 – JSC Acquisition Summary was used in some of the files. This form includes an item for the “estimated cost/price”. The 2004 survey suggested revising the form to add the “basis for the estimated cost/price” as a reminder of the need to document the file with that information. This does not appear to have been done. Based on the review of the noncompetitive contract files, the lack of cost estimates is presumably more widespread, and the weakness from the last two surveys has not been resolved.

WEAKNESS:

The JSC Office of Procurement should ensure that sufficient government cost estimates are established and documented in the files. (Repeat Finding)

CONSIDERATION:

In response to the findings from the 2002 survey, JSC stated that training regarding preparation and documentation of cost estimates was provided at the 9/12/02 Procurement Forum. The JSC Office of Procurement should consider repeating this training. In addition, if the JSC Form 1219 is still allowed/required to be used in the contract files, the Office of Procurement should consider revising it as suggested in the 2004 survey, as a reminder to include a detailed basis for the estimated cost in the files.

3. Acquisitions in Process

The primary emphasis of this review was to determine if the awards were in compliance with current procurement regulations and policies. In general, the procurement files selected for review contained the appropriate level of information, were well documented, and followed the appropriate procedures. This was particularly true for larger acquisitions, including those utilizing SEC/SEB procedures. With the exception of files utilizing simplified acquisition procedures, all files reviewed utilized the NASA Form 1098, Checklist for Contract Award File Content, which helped to ensure that all required documentation was both included in the award file and filed in the appropriate location.

For acquisitions of all sizes, the reviews indicated that the overall source selection process was effective and resulted in timely awards of procurements. For non-competitive actions, the sole source documentation was generally thorough and followed the required FAR format. For large competitive acquisitions, source evaluation teams were appointed prior to receipt of proposals. Evaluation documentation was also found to be very thorough and comprehensive. When not performed by the buyer, pricing support for competitive actions was provided by dedicated pricing support analysts in support of these activities. Additional supporting documentation for large competitive acquisitions indicated that policies and procedures had been followed and that each step was thoroughly documented. This documentation provided an acceptable audit trail from acquisition planning through final award. In addition, source selection statements, which considered the findings of the board, were thorough, well documented, and provided clear rationale for the basis of selection.

Reviews of the files selected indicated no systemic problems, regardless of acquisition type or level of complexity. These reviews indicated that buyers were, in general, familiar with the policies and procedures governing the performance of their job responsibilities.

Non-systemic and isolated instances relating to a lack of proper documentation were noted, though these instances were often mitigated by other documentation provided elsewhere in the contract file. Other minor non-systemic findings included occasional lack of VETS100 certifications, lack of a D&F for provision of Government-furnished property, lack of market research documentation in a sole source justification, lack of CCI consultation, lack of debarred list consultation, and an incorrect Certificate of Current Cost and Pricing Data.

4. Market Research

FAR Part 10 requires Agencies to conduct market research before soliciting offers for acquisitions with an estimated value in excess of the simplified acquisition threshold. Latitude is granted in that the extent of market research may vary, depending on various factors such as urgency, estimated dollar value, complexity, and past experience. While no specific format is mandated for the documentation of market research activities, a template is contained within the VPO that can be used for this purpose and is highly recommended in order to standardize this documentation and the associated processes.

Reviews of numerous contract files indicated that some form of market research had been performed for each acquisition. However, it was also evident by these reviews that some files provided inadequate documentation of the market research activities that had been performed. It was also evident that the types of market research conducted and the documentation provided varied depending on the type and dollar value of the procurement.

In most cases, the acquisition files contained detailed Market Research Memorandums utilizing the VPO template. One file included a memorandum entitled, “Uniform Methodology for Determination of Small Disadvantaged Subcontracting Goals”, which contained a comprehensive discussion of the market research activities that were performed. However, two files for construction revealed that market research had obviously been performed but had not been documented under the appropriate tab. References to this research were instead made in the respective Pre-negotiation Position Memorandums. Though coordinated with the Small Business Administration, these files provided no rationale as to how the firms participating in the presentations for eventual award had been located and selected for participation. Other non-systemic items included a reference within the market survey documentation of the sources sought synopsis with no copy of the notice in the file, and a reference in a sole source document to a sources sought synopsis without reference of the notice in the market research documentation.

CONSIDERATION:

The JSC Procurement Officer should encourage the use of the market research template provided in the VPO in order to standardize the documentation supporting the research conducted.

5. Affirmative Procurements

A sampling of several recently awarded or in-process contracts were selected for review in order to determine compliance with applicable guidance. Two contracts selected for review were construction contracts and therefore had environmental impacts by the nature of the requirement. In addition, other Center support services contracts were also reviewed to determine whether environmental clauses were incorporated when required or appropriate.

The review indicated that the contracts contained the required and appropriate FAR Part 23 clauses. In addition, the Contracting Officers on the contracts with specific environmental impacts were interviewed to determine whether environmental compliances were technically being managed properly in accordance with the contract. These interviews provided sufficient evidence that the environmental impacts were indeed well managed by the Contracting Officers and COTRs. Where applicable, the appropriate management of the compliances was evident by comprehensive surveillance plans that were tied to environmental Performance Work Statements (PWS) requirements and approved safety and health plans.

All contracts reviewed contained one or more of the following environmental clauses: 52.223-3, Hazardous Material Identification and Material Safety Data, 52.223-5, Pollution Prevention and Right to Know Information, 52.223-9, Estimate of Percentage of Recovered Material Content for EPA-Designated Products, 52.223-10, Waste Reduction Program, and 52.223-11, Ozone Depleting Substances, and 52.223-12, Refrigeration Equipment and Air Conditioners.

In addition to reviews of contract files, the continued implementation of PIC 01-27, "Applicability of Affirmative Procurement," was discussed with the JSC Procurement Policy and Systems Office. This discussion indicated that the steps taken to implement the requirements of the PIC have been successful. As part of this implementation, JSC Procurement Policy and Systems Office established a "Request for Waiver (for EPA-CPG Items)" Form JF 1121, revised February 7, 2005, which has been utilized successfully since the last survey. In addition, the NF 1707, Certification and Special Approval for SAP Purchase Request, includes a certification of the affirmative procurement in accordance with FAR Part 23.4 and NPR 8530.1, "Affirmative Procurement Plan for Environmentally Preferable Products". The JSC Environmental Office is also conducting discussions with NASA Headquarters regarding the implementation of additional clauses of benefit to the Agency.

The Affirmative Procurement Program has been a training topic at the "monthly procurement forums". JSC also continues emphasis on the use of recyclable materials and presents relevant environmental issues to the logistics organizations as well as part of their risk-based management acquisition planning for service procurements exceeding $15M.

6. Government Furnished Property

Eight contracts were reviewed for compliance with regulations and procedures applicable to providing Government property to contractors. The review focused specifically upon the use of appropriate clauses, compliance with 1018 reporting, and review of requirements by the Supply and Equipment Management Officer (SEMO). In addition, files were reviewed to ascertain whether a Determination and Findings (D&F) required by FAR 45.302-1(a)(4) and the contracting officer rationale required by 1845.302-1(a)(iv) were included in the file.

All files included evidence of review by the SEMO or representative. All contracts reviewed included the appropriate property clauses with the exception of one, which appeared to be missing 52.245-4, Government-Furnished Property (Short Form) or 52.245-2, Government Property (Fixed-Price Contracts), even though a small amount of GFP was listed and provided and financial reporting and other GFP related activities appeared to have been properly performed.

No D&F or detailed explanation of why continued furnishing of the facilities is in the best interest of the Government could be found in the files of two contracts providing GFP, as required by FAR 45.302 and NFS 1845.302-1. The associated acquisition plans mentioned that GFP would be provided, but no rationale or justification was included.

WEAKNESS:

JSC Procurement Office should ensure that contract specialists be reminded of the requirement to include a D&F, or for follow-on contracts, a detailed explanation, when property is provided under a contract. (REPEAT FINDING)

Return to Contents

SECTION IV

POST - AWARD

1. Task Order Contracts (Competition under Multiple Award and Delivery Order Contracts)

The survey team reviewed a sample of contracts where multiple awards were made in the area of A/E and construction. Some new construction contracts were in place and even though no orders were yet placed against them, the process for placing orders ensured adequate competition.

Determinations to make delivery order placements without competition were limited to existing A/E contractors in which delivery orders were placed in accordance with FAR 36. All requirements of the solicitation were in accordance with 36.602-3, 36.602-4 and NFS 18-36-602-4. The files were well documented.

In reviewing orders placed against GSA contracts, adequate price competition is being accomplished. Prior to placing orders, quotes were received from three GSA sources. One order utilized sole source procedures in accordance with the simplified acquisition procedures in FAR 13.106.

2. Cost Plus Award Fee/Incentive Fee

Six contract files were reviewed for compliance with FAR Subpart 16.4 and NFS Subpart 1816.405-2 regarding CPAF contracts. The six CPAF contracts reviewed included 14 performance evaluation periods since last JSC Procurement Survey. Compliance areas reviewed included: proper consideration for contract type and evidence of justification and approval for use of award fee; method of determination of maximum fee for contract (competition vs. profit/fee analysis); timeliness of fee determination and payment; proper use of interim or final fee determinations (including “rollover” provisions); proper use and weighting of evaluation factors; and development of and adherence to performance evaluation plans.

All files reviewed included comprehensive award fee plans that addressed technical, cost and schedule considerations in accordance with the guidelines set forth in the NFS. All of the award fee evaluations reviewed followed their corresponding award fee plans and were very comprehensive and provided well-documented analyses. Some of the files contained letters identifying areas of emphasis which were provided to the contractors prior to the start of the evaluation periods. Where maximum fee amounts were not determined by competition, a profit/fee analysis on NASA Form 634 was performed.

JSC continues to maintain an award fee tracking system. A review of the tracking system database for all award fee contracts with evaluation periods ending since January of 2005 revealed that sixty-five percent (65%) of FDO ratings were completed and the contractor notified within 45 calendar days from the end of the award fee period with and average time of 43 days; and 61% of award fee payments due the contractor were made within 60 calendar days from the end of the award fee period with an average time of 57 days. While average notifications and payment times are within the prescribed limits, a significant number of individual notifications and payment times didn’t meet the standard.

The survey team also reviewed a sample of fee determinations. The evaluation process appeared consistent between periods and contracts and was documented thoroughly from the contractor's self-assessment to the completion of the fee determination. JSC has implemented several evaluation process improvements since the last survey as part of their “Award Fee Standardization Guidance and Best Practices” including “FDO templates” and “Award Fee Standardized Scorecards” that appears to contribute positively to the process.

The contract files were also reviewed for compliance with NFS 1816.405-270 (a), which states, “Use of an award fee incentive shall be approved in writing by the procurement officer. The procurement officer's approval shall include a discussion of the other types of contracts considered and shall indicate why an award fee incentive is the appropriate choice.” While all contract files included evidence of the Procurement Officer approving the use of Award Fee through the Acquisition Strategy Meeting process and approval of the RFP, only two had any significant discussion on why a CPAF contract best suited the particular procurement.

STRENGTH:

JSC continues to effectively manage their award fee contracts. Files are well-documented and letters with areas of emphasis sent prior to the evaluation period are effective in maintaining contractor focus. Recent process improvements appear to add consistency and rigor to the performance evaluation process.

WEAKNESS:

JSC should take steps to ensure that the discussion and rationale leading to the decision to use an award fee contract is documented in the acquisition planning or other documentation in accordance with NFS 1816.405-270.

3. Closeout and Unliquidated Obligations (ULOs)

The survey team’s review of closeouts and unliquidated obligations consisted of trend analysis, closeout schedule time frame by contract type, use of quick closeout procedures, open actions, and overage ULOs for the closeout of completed contracts, purchase orders, grants, cooperative agreements, inter-agency agreements and the status of unliquidated obligations.

The Agency closeout contractor, BRACE Management Inc., provides support to the JSC Procurement Office. BRACE is responsible for preparing all closeout documentation for the Contracting Officer’s review and approval with the exception of purchase orders, interagency agreements, and task orders. The procurement office monitors actions outside of the BRACE responsibilities on a monthly basis and briefs the Procurement Officer quarterly regarding the latest status. BRACE has a database that tracks and reports open contract population, action plan, overage not closed, and a monthly report reflecting actions closed. The overage report reflects why the actions are over due. For example, pending DCAA audits, property issues, additional funding requested, final invoices, and disallowed cost pending. As of March 6, 2006, JSC has approximately 124 physically completed actions which are not administratively closed where obligated monies are not yet liquidated. The unliquidated amount for these actions is $4.1M.

JSC over the last eighteen months based on data reviewed in the system by the team, has improved its timeline for closing out actions.

STRENGTH:

The JSC Procurement Officer is commended for focusing on closing out contract actions and improving its closure rate over the last eighteen months.

4. Undefinitized Contract Actions (UCA’s)

The team randomly reviewed the twelve (12) JSC UCA’s identified in the most recent monthly report (January 2006) submitted to the Headquarters Office of Procurement and actions issued over the last 24 months. The file review focused on age, dollar value, appropriate approval level, justification and file documentation in accordance with NFS 1843.70. JSC is to be commended for prudent justification and file documentation prior to the issuance of undefinitized contract actions. Since the last report to Headquarters, JSC has reduced the number of UCAs by 40%. The Shuttle Office has experienced a number of credit UCAs since the Program Office requirements have been adjusted for numerous reasons. Of the nine (9) UCA’s remaining, 5 exceed 180 days. The absolute value for the open actions is $55.7M. JSC has initiated a plan that anticipates definitizing all outstanding actions no later than April 2006.

Over the last 24 months JSC has continuously reduced the number of undefinitized contract actions and dollars in support of its programs.

STRENGTH:

The JSC Procurement Officer is commended for its prudent analysis and sound file documentation prior to issuance of UCA’s and aggressively reducing the number of UCA’s at the Center.

CONSIDERATION:

The JSC Procurement Officer should establish a methodology with the program offices that will eliminate the need to carry credit proposals on the books for an inordinate amount of time.

5. Contractor Performance Evaluation (NF 1680)

NFS 1842.1502 requires that within 60 days of every anniversary of the award of a contract having a term exceeding one year contracting officers must conduct interim evaluations of performance on contracts subject to FAR subpart 42.15 and the corresponding NFS subpart. On such contracts, both an interim evaluation covering the current period of performance and a final evaluation summarizing all performance must be conducted. This evaluation is performed through completion of NF 1680, "Evaluation of Performance". The survey team reviewed twelve contract files meeting the criteria of FAR 42.15 to determine compliance with the above requirements. These twelve contract files encompassed 16 required evaluation periods. Fourteen NF 1680s were completed. All contract files reviewed included at least one past performance evaluation.

The survey team evaluated the way Johnson procurement managers oversee the completion of past performance evaluations. Johnson’s procurement management conducts informal training through the staff meetings of the various procurement offices to explain the process for completing the past performance evaluations and to emphasize the importance of these evaluations. Team leads issue a quarterly report to identify evaluations that are due and follow up with each office. Past performance metrics are reported to the Procurement Officer each quarter.

In addition to the NF 1680 being included in the contract file, Procurement Information Circular (PIC) 01-12, "Contractor Performance Information and Past Performance Data Base (PPDB)" requires that the information on the NF 1680 be entered into NASA's PPDB. All of the 12 contract files reviewed had at least one evaluation period entered into the PPDB; however the most recent evaluation period had not been entered into the PPDB for two contracts.

NFS 1842.1501 emphasizes the importance of communications with the contractor. The narratives of the NF 1680 greatly enhance the usefulness of the contractor performance evaluations for anyone conducting a check of a contractor's past performance evaluations and history. These narratives also provide contractors with feedback about their performance. Therefore, it is important that the narratives be detailed and support the ratings received. Most of the NF 1680s reviewed included, good narrative support to the ratings given and that narrative had been entered into the PPDB. Several 1680’s had little or no narrative to support the ratings given. These instances were on award fee contracts and referred to the “Award Fee Evaluation in contract file”; however in two of the three instances the last evaluation period had at least some narrative, leading the reviewer to conclude that JSC is emphasizing the need to include narrative input in the PPDB to support ratings for all contracts evaluated, including Award Fee contracts since the information in the PPDB will be used in past performance evaluations as part of the source selection process.

STRENGTH:

JSC Office of Procurement is commended for the management attention and emphasis given the timely completion of NF 1680s and input to the PPDB resulting in significant improvement to the quality of the information in the PPDB, including specific information on contractor quality and safety performance.

CONSIDERATION:

To ensure the PPDB contains the most useful qualitative contractor performance information for all contracts evaluated, JSC should continue to emphasize that Contracts personnel input at least summary performance narrative information in the PPDB for Award Fee contracts.

6. Contracting Officer Technical Representative (COTR) Delegations and Training

a. Delegations

The NASA FAR Supplement (NFS) authorizes contracting officers to appoint qualified Government employees to act as their representative in managing the technical aspects of particular contracts. The NASA Form 1634, “Contracting Officer Technical Representative (COTR) Delegation”, is used to appoint COTRs. Each COTR must acknowledge receipt and accept the delegation by signing the delegation form which is maintained in the contract file.

Several contract files were reviewed for appropriate file documentation in accordance with NFS 1842.270(e). In most cases the appropriate delegations were properly signed and maintained in the contract files. Two files reviewed did not contain the necessary delegations. The cognizant Contract Specialist confirmed that in one case, the delegation was not executed. A review of the COTR training database revealed that all but one of the appointed COTRs for the contracts reviewed received either basic or refresher training within the last year. One COTR for a recently awarded contract had not received training since 2002 according to the “List of Trained COTRs”.

b. Surveillance Plans

The submission of a contract surveillance plan is one of the key components of COTR delegation. When required, the COTR must establish and provide a contract surveillance plan to the contracting officer. In accordance with JPI 1846.401, contracting officers are responsible for ensuring that contract surveillance plans are included in the contract file for all contracts greater than $1 million.

The contract files reviewed contained the necessary surveillance plans as appropriate in all but two cases. In one case, the contract was recently awarded and the COTR was still developing the plan for review and approval by the contracting officer. In the second case, the surveillance plan was not required at contract award since it was below the required dollar threshold. However, a note to the file indicated that the COTR delegation would be amended and a plan established upon exercise of the first option. The option was exercised without amending the COTR delegation or establishing a contract surveillance plan. The cognizant contract specialist admitted the oversight.

c. Training

The NFS requires all COTRs and alternates to attend mandatory training prior to appointment. Further COTRs must attend a refresher training at least every five years. The Johnson Procurement Instruction (JPI) requires current COTRs to attend refresher training on an annual basis.

The JSC “List of Trained COTRs” maintained on the JSC Procurement Website contains over 1100 names in the database and includes information for all individuals receiving COTR training during the course of their career at JSC. The vast majority of individuals listed in this database have not received refresher training within the last year as required by the JPI and many have not received refresher training within the last five years as required by the NFS. A sample of 30 individuals was reviewed for training currency. Twenty-four of the thirty did not have current training; the data reflected that some received training as long ago as 1992. Five of the individuals in the sample were no longer in the JSC telephone database. Of the six COTRs with current training, three received the refresher training more than five years after receiving the basic training.

The Data Systems Team Lead explained that the “List of Trained COTRs” contains the universe of COTRs both active and inactive and serves as a guide to determine COTR appointment eligibility. The Policy Office maintains a separate list of active COTRs in spreadsheet format. The “active COTR” spreadsheet includes the following categories of information: contract number, contract completion date, contractor name, procurement organizational code, appointing contracting officer, technical organization, primary COTR, appointment date, refresh date, alternate COTR, appointment date, and refresh date. The data on this list is much more current than the previously referenced listing and contains useful information for tracking and verifying COTR eligibility for appointments.

There are approximately 225 active COTRs on the list. A sample of approximately 30 COTRs was reviewed and with the exception of two individuals (one primary and one alternate), all COTRs had received at least a refresher training within the one year JPI requirement. The review did reveal that the vast majority of these individuals received the basic COTR training several years prior to the refresher. In some instances, it appears that appointments were made prior to receiving the required refresher and in some cases the basic training.

A review of the list also revealed that not all fields were properly populated; contract completion date and COTR appointment date were left empty in many cases. It is also noted that the list does not include a field for the basic COTR training.

d. COTR Training Course Content

The NFS requires the mandatory COTR training to cover the following core topics: 1) contracting authority and contract modifications; 2) inspection and surveillance; 3) changes and performance-based contracting; 4) contract financial and property management and 4) disputes.

The JSC Procurement website contains several documents related to COTR refresher training. The training material covers most of the required topics. The information includes a general overview of COTR responsibilities in a presentation entitled “Professionalism in Procurement”, separate presentations discussing ‘post award’, ‘surveillance plans’, ‘lessons learned’, ‘justifications for other than full and open competition’, ‘COTR dos and don’ts’ as well as copies of past refresher training presentations. Neither the general overview nor the focused presentations appear to address contract finance and property management or disputes in detail. However, refresher presentation charts from 2002 and 2003 did address all of the required areas.

Within the last year, the JSC Procurement Policy Office began conducting ‘in house’ COTR refresher training. This training focused on selected topics (typically three to four) that address timely issues or areas identified for improvements to the procurement process. These sessions are conducted on a quarterly basis. Previously, JSC utilized the services of a support contractor to provide COTR training.

The basic COTR training course is conducted by contractors. It contains comprehensive coverage of the topics required by the NFS as well as other pertinent topics relevant to procurement and COTR responsibilities. The basic course is offered semi-annually.

STRENGTH:

JSC Policy Office should be commended for the wealth of training information made available for COTRs on the JSC Procurement Website and through the training classes.

WEAKNESS:

JPI requires COTRs to receive annual refresher training. The JSC Procurement Office should ensure that all active COTRs receive training in accordance with Center requirements (annually) and document completion appropriately. (Repeat Finding)

CONSIDERATIONS:

1. JSC Procurement Office should ensure that COTR delegations are included in all files as appropriate and that appointed COTRs have a current training status prior to appointment.

2. JSC Procurement Office should ensure that surveillance plans are included in the contract files as appropriate.

3. JSC Policy Office should ensure that all required core topics are addressed to some extent in both the COTR training and refresher cores.

4. The COTR Training database currently posted on the procurement website contains the universe of COTRs trained at JSC. The vast majority of the individuals listed in the database have not received COTR training in several years. This database should be removed from the website and replaced with the List of Active COTRs maintained within the JSC Policy Office. Further the JSC Policy Office should ensure that all fields in the Active COTR database are properly populated and amend the list to include the two most recent training sessions in order to track compliance with the JPI.

7. Contract Safety Requirements

Nine contracts were reviewed to verify compliance with NASA FAR Supplement requirements relating to Safety and Health (S&H) and other safety related policies and procedures. Specifically, contracts were reviewed for inclusion of the Safety and Health and Major Breach of Safety and Security clauses, Safety and Mission Assurance (S&MA) office review of contract requirements, submission, review and approval of Safety and Health plans, and the proper evaluation of Safety and Health in the source evaluation and performance evaluation processes.

All files reviewed required inclusion of the Safety and Health and the Major Breach of Safety and Security clauses, and therefore also required submission and review of a Safety and Health Plan. All contracts reviewed were found to include the aforementioned clauses and all had evidence that the S&MA personnel were involved in review of the procurement package for inclusion of appropriate Safety and Health related clauses and requirements. All files included Safety and Health plans, and evidence that the plans had been reviewed and concurred on by the appropriate personnel. Information in the files also indicated that S&H plans were tailored to the specific requirements of the contract and were updated when requirements or conditions changed.

In all competitive procurements reviewed it was clear that S&H personnel were closely involved in the source evaluation process and S&H plans were appropriately reviewed; and the S&H personnel appeared to be similarly involved in the performance evaluation process for Award Fee contracts. Further, in both the source evaluation process and the performance evaluation processes historical safety indicators (e.g. Experience Modifier Rates), OSHA recordable metrics (e.g. days away from work, etc.) and leading safety indicators are considered

STRENGTH:

JSC is commended for its management attention to safety and health issues including close involvement of the safety office in procurement and contract management activities and the use of safety indicators and metrics in the contractor past performance evaluation, award fee and source evaluation processes.

8. Exercise of Options

The review of options focused on the presence of the required option clauses as well as determination documents in exercising the options. Determinations to exercise were very thorough and included all required information in accordance with FAR 17.207 Exercise of Options and NFS 1817.207 Exercise of Options.

One file in particular did include documentation to support that exercising the option was in the best interest of the government and the most advantageous way to acquire the government’s requirement. The contracts specialist understood that the documentation should have been included in the file but was unable to locate it.

It should be noted that the file documentation in support of the option exercised on Contract NNJ04JA83C was excellent. This file documentation is an excellent example for others to use when exercising options.

Return to Contents

SECTION V

PRICING — FINACIAL — AUDITS

1. Cost and Price Analysis

Twenty-six files were reviewed for a determination of adequacy of the cost and price analysis. The scope of the review included competitive, noncompetitive, fixed price and cost reimbursable contracts ranging in price from $300K to $325M.

Cost and price analysis preformed by Pricing Team price analysts for the contract specialists were extremely well documented, comprehensive, and consistently addressed all elements of a cost proposal. Major subcontractor costs were consistently addressed in the evaluations. The integration of tailored requested field work and technical evaluations was of a consistently high level providing expanded depth to the evaluation. Substantial justification was provided supporting the logic provided for questioned cost positions. Cost and price analyses performed by the contract specialist were not always as comprehensive. There were instances where reports prepared by contract specialists accepted the technical evaluation recommendations for, travel, material and other direct cost as the basis for accepting the proposed cost. All reports provided traceability to source documents with many documents included as attachments to the report or located under the same contract file tab.

In all required situations, the requirement for cost and pricing data was appropriately identified in the solicitation document. Also, the files contained properly executed and timely submitted Certificates of Current Cost or Pricing Data.

STRENGTH:

There is a discernible improvement in the quality of the cost and price analysis performed by the Contracting Officer. The JSC Office of Procurement should be commended for its continuing effort to improve the quality of cost and pricing analysis.

2. Pre-Negotiation and Post-Negotiation Memorandums

The survey team selected fifty one contract files for review; from these a review of twenty two files was preformed to ascertain the adequacy of pre-negotiation position memorandums (PPM) and post negotiation memorandums (PNM). Utilizing the standard VPO memo format, pre-negotiation position and post negotiation memorandums consistently addressed all eleven areas specified in FAR 15.406-3 and NFS 1815.406-170. A majority of the memorandums reviewed provided traceability to source documents and information. Minor deficiencies noted were, the occasional failure to cite the reliance on cost or pricing information provided, failure to provide a statement that the price was fair and reasonable, the current status of any contractor systems and the extent they affected or were considered in the negotiation, and adequate rationale for the Government's profit/fee objectives. The memorandums reviewed consistently addressed all the required subject areas.

STRENGTH:

The JSC Office of Procurement should be commended for a continuing demonstrated improvement in the quality of pre-negotiation position memorandums (PPM) and post negotiation memorandums (PNM).

3. Technical Evaluations of Cost Proposals:

A review of over twenty contracts consisting of competitive and non-competitive procurements was preformed to ascertain the acceptability of technical evaluations and their compliance with the requirements established in FAR 15.404. The files were reviewed considering the adequacy of the technical evaluation of the cost proposal and their uniformity with the technical proposal. The majority of the requests for technical evaluations were tailored to address only specific elements of a proposal requiring an evaluation. Evaluators are providing detailed information as to the reasons for acceptance of the labor skill mix and hours. Also, the analysis of Other Direct Cost has improved over past reviews. When technical evaluators failed to adequately address elements of a proposal, Contracting Officers communicated the need for further evolution and documentation. This resulted in evaluators providing adequate discussions in support of their position, which allowed the Contracting Officers to negotiate from a position of strength. File documentation was acceptable to above average and ranged from formally written evaluations prepared on a template to a series of e-mails.

STRENGTH:

The JSC Office of Procurement should be commended for a continuing demonstrated improvement in the quality of technical evaluations. Improved communications between the Contracting Officers and technical evaluators have resulted in a notable enhancement in the quality of the evaluations and their increased worth to the ensuing procurement.

4. Use of Structured Approach in Profit/Fee Determinations

Guidance for determining profit or fee objectives and the required use of NF 634, "Structured Approach Profit/Fee Objective" is promulgated in NFS 1815.404-470 and 1815.404-471. Over twenty contract files were reviewed for the appropriate use of and compliance with the requirements for NF 634. All Pre-negotiation Position Memorandums (PPM) showed the contractor's proposed profit and NASA's position, along with rationale as to how the profit/fee objective was obtained. The preponderance of the files contained written justification for the selection of other than the normal Assigned Weighting, Assigned Value, Weight Range, and Weight Designated percentages. Performance risk was addressed in the majority of the reviewed Pre-negotiation Position Memorandums (PPM). Fee calculations were adjusted for contracts proposing facilities capital cost of money. No statutory fee limitations were exceeded and the limitations were consistently addressed in the PPM. The NF 634 was used to document an acceptable fee percentage in all of the reviewed files.

Several files had misfiled or incorrect references in the PPM for the NF 634. One file documented the fee rationale in the PPM yet the NF 634 was in direct conflict with the written justification. This file should have cited an exception to the use of NF 634.

STRENGTH:

The preponderance of the NF 634 reviewed were well documented in accordance with NFS 1815.404-470 and 1815.404-471. The uses of other than the normal Assigned Weighting, Assigned Value, Weight Range, and Weight Designated percentages were well documented and supported in the PPM.

5. Audit Follow-Up

NASA Procurement Informational Circular 00-06 provides guidance on contract administration and audit support services provided by the Defense Contract Management Agency (DCMA) and the Defense Contract Audit Agency (DCAA). NFS Part 1842 implements OMB Circulars A-50 and A-133 setting forth the requirement for NASA contracting officers to resolve reportable audit issues in a timely manner.

The Survey Team evaluated how Johnson tracks, resolves, and closes audit recommendations. The Procurement Office has two representatives that oversee audit activity affecting JSC. The Johnson Audit Liaison Representative (ALR) oversees audit activity from the Office of the Inspector General (OIG) and the General Accounting Office (GAO). A representative from the Policy and Analysis team oversees audit activity from the DCMA and DCAA.

JSC uses the NASA-wide Corrective Action Tracking System (CATS) to report and track reportable audits. Additionally, both representatives maintain their own tracking system to track all audit activity. The liaisons indicated that they conduct validation activities to ensure corrective actions have been implemented. As part of their duties, the liaisons track the closure of recommendations and write closure letters to the DCAA and OIG as appropriate. Both representatives attend and participate in training conferences and teleconferences.

The senior management of Johnson strongly supports audit activities. The ALR reports the status of all reportable OIG audits to the Center Director’s Office each quarter. The Center Director or his representative reviews and signs all correspondence regarding OIG audit activities. Johnson resolves GAO audit recommendations affecting its Center; however, GAO audits are tracked, reported, and closed at the Headquarters level. The status of DCMA/DCAA audits is reported to the Director of the Office of Procurement each quarter. Both representatives felt that senior management supports their activities and are willing to assist in resolving recommendations, if necessary.

JSC currently has no open DCAA reportable audits. Johnson currently has no open OIG audits or open OIG or GAO recommendations; however there are 10 OIG and 4 GAO audits in progress relating to JSC. JSC has had no open reportable audits for approximately 18 months.

6. Contractor Insurance/Pension Reviews

The team reviewed contract files to ensure that a thorough analysis of the contractor’s insurance/ pension plans, related policies, procedures and practices, and costs to determine whether they are in compliance with the rules and regulations. The team held discussions with the procurement personnel to ensure that the files were documented with reports, correspondence with DCMA/DCAA personnel and how issues were being addressed and resolved. The procurement staff readily uses the DCMA/DCAA local offices to review and monitor contractor insurance and pension plan activities for the International Space Station and Space Flight Operations Contract programs. The majority of the files contained all the necessary information or it was provided to the team as further supporting documentation. None of the files identified any outstanding issues.

STRENGTH:

The JSC Procurement Officer is commended for its overall understanding of the process, file documentation and interfacing with DCMA/DCAA regarding scheduling of reviews and monitoring of any issues and/or concerns in a timely manner.

CONSIDERATION:

The JSC Procurement Officer should ensure that individuals transferred to more complex contracts with contractor insurance/pension review requirements be tutored if they don’t have prior experience.

7. Financial Management Reporting (NF-533)

Seven contract files were reviewed for compliance with NFS 1842.72, NASA Contractor Financial Management Reporting. All of the contracts met the contract type and dollar thresholds requiring financial management reporting. All of the contracts included NFS clause 1852.242-73 requiring NASA Financial Management Reporting. The clause requires that the detailed reporting categories to be used, be set forth in the contract, as well as the number of copies and time and manner of submission. All of the contracts contained data requirements descriptions (DRD) that covered the required information for areas/levels of reporting, e.g., Systems Engineering to WBS Level 3, but not specifically for individual cost element reporting categories, e.g., direct labor hours, direct labor dollars, equipment, etc.

NFS 1842.7201(a)(1) requires COs to monitor contractor cost reports on a regular basis to ensure cost data reported is accurate and timely, and to pursue adverse trends and discrepancies discovered in cost reports through discussions with financial and project team members. The COs for all seven contracts were reviewing and analyzing the NF533 reports as required, and most were documenting their review in the files. It was noted that the JSC Procurement Homepage contained a good NF533 analysis guide for the COs.

STRENGTH:

The COs are performing the required review and analysis and following up with contractors to resolve NF 533 issues.

CONSIDERATION:

The JSC Office of Procurement should consider revising the Financial Management Reports DRD to require specific cost element reporting categories to be designated for the individual contracts, and remind COs to document their files with the results of the NF533 review and analysis.

Return to Contents

GRANTS, COOPERATIVE AGREEMENTS, SIMPLIFIED ACQUISITIONS, AND OTHER ISSUES

1. Grants & Cooperative Agreements

NASA established goals for grants/cooperative agreements lead-time are: 60 days from purchase request receipt to award, and 29 days from complete package to award. All of the files reviewed were awarded in a shorter time frame than the established goals.

On July 27, 2001, a standard NASA policy for informing Members of Congress about the selection of NASA research proposals was issued. Under this policy, all grants and cooperative agreement selections, regardless of value, must be reported. All files reviewed included evidence of the Congressional notification. All of the award documents contained the appropriate Handbook provisions.

The Handbook requires Grant Officers to ensure that all necessary certifications, disclosures, and assurances have been obtained prior to awarding a grant or cooperative agreement. All awards were in compliance with this provision. In the last survey, it was noted that certifications and assurances were consistently obtained, and Johnson is again commended for addressing this issue.

The Handbook sets forth the policy on incremental funding of grants and cooperative agreements. The intent of this policy is to reduce the administrative burden in funding awards. All awards complied with this policy. Johnson procedures require a legal review of award instruments valued at or over $500,000. A review of the grants and cooperative agreements indicated that all awards that required legal review included the legal review in the file.

All files included a detailed price negotiation memorandum. It appears that areas including labor, material, indirect rates, travel and other direct costs were reviewed by the technical monitor but the Contracting Officer provides less than adequate rationale for concurrence with the technical monitor’s findings. All files reviewed were in compliance with the Handbook provisions on delegation of administration.

The Handbook states that “the Grants Officer is responsible for submitting the Individual Procurement Action Report (NF 507) for all grants and cooperative agreement actions.” All files reviewed included a NF 507 for all basic awards and supplements (modifications to basic awards). The Handbook requires Grant Officers to complete a NASA Form 1356, “Committee on Academic Science and Engineering (C.A.S.E.) Report on College and University Projects” for awards to nonprofit institutions of higher education and nonprofit institutions that are operationally affiliated or integrated with an educational institution. All awards that met these criteria included a completed NF 1356 in the file. . In the 2004 JSC survey, the team reviewed competition metrics for all NASA grants and cooperative agreement awards and concluded that JSC needed to take action to improve competition for research and training awards. The 2006 survey team commends JSC for taking actions to remedy the weakness by soliciting specifically to minority institutions and not-for-profits which reach out to students in the areas of math, science, engineering, and technology. JSC awarded a three year, multi-award grants to two minority institutions and one not-for-profit. JSC is continuing effort to actively solicit to minority institutions and not-for-profits to award more grants.

STRENGTHS:

(1) The Johnson Office of Procurement is commended for the following: meeting its goals for grant and cooperative agreement lead-time metrics; complying with the requirements of the Grant and Cooperative Agreement Handbook with regard to reports, delegation of administration, certifications and assurances, and Congressional reporting.

(2) Office of Procurement is commended for its efforts to specifically solicit to minority institutions and not-for-profits, which is an improvement from the previous Survey.

WEAKNESS:

In the last survey, it was recommended that Grant Officers work closely with their technical Officers to conduct an analysis of the proposed costs. JSC Office of Procurement needs to emphasize the importance of performing analysis of the proposed cost in conjunction with the Technical Officer. (REPEAT FINDING)

2. SBIR Program

Small Business Innovation Research (SBIR) programs fund research and development efforts of a high risk nature that may have excellent commercial potential. Under the Small Business Innovation Development Act, each agency with an extramural R&D budget in excess of $100 million must establish an SBIR program.

All SBIR contracts were awarded on or before the date specified for the SBIR Program. Because of the short time frame for award and the number of awards, JSC is commended for making these awards in a timely manner. To promote timeliness, the Institutional Program Office has bi-monthly meetings with the Contract Specialist assigned to these awards to review their progress and identify any problems.

SBIR technical data rights clause at FAR 52.227-20 apply to all SBIR awards, including subcontracts to such awards, that fall within the statutory guidelines of Phase I, II, or III of the SBIR program. The Survey Team recommends that Contract Specialist insert the provisions of (b) (1), (2), and (3) of the most recent authorization act into all SBIR Phase I, II, and III awards. All SBIR awards reviewed referenced FAR 52.227-20. A 507- “Individual Procurement Action Report” was completed for all awards reviewed.

The Handbook sets dollar and period of performance limitations for SBIR awards. SBIR Phase I awards are for periods of up to six months in amounts up to $70,000. SBIR Phase II awards are for periods of up to two years in the amounts up to $750,000. All awards reviewed were in compliance with these requirements. The Handbook also sets limitations on subcontracting. For SBIR Phase I awards, the proposing small business must perform a minimum of two-thirds of the research/analytical work and for Phase II, a minimum of one-half of the work must be performed by the proposing firm. All SBIR awards reviewed complied with these requirements.

The Survey Team evaluated each price negotiation memorandum. The NFS waives the requirement for the submission of cost or pricing data when contracting for SBIR Phase II contracts. However, Contracting Officers shall ensure that the appropriate level of information other than cost or pricing data is submitted to determine price reasonableness and cost realism. All files included a detailed price negotiation memorandum. Each technical evaluation stated that the proposed budget and individual elements were appropriate and most of the price negotiation memorandum placed heavy reliance on the technical officer. However, it was recommended on the last survey that the Contracting Officer work closely with the Technical Officer to conduct a more in-depth analysis of travel, material, and other direct cost. In accordance with NFS 1815.404-4, the NASA structured approach for determining profit or fee objectives, described in 1815.404-471 shall be used to determine profit or fee objectives in the negotiation of contracts greater than or equal to $100,000 that use cost analysis and are awarded under the SBIR program. All the files reviewed including a signed NF 634, “Structured Approach Profit/Fee Objective.” All SBIR awards reviewed contained a detailed technical evaluation and the appropriate Representations and Certifications.  

STRENGTH:

The JSC Office of Procurement is commended for conducting training sessions and awarding contracts on or before the date specified in by the SBIR Program.

WEAKNESS:

In the last survey, it was recommended that Contracting Officers work closely with their technical Officers to conduct an analysis of travel, material, and other direct cost. JSC Office of Procurement needs to emphasize the importance of performing analysis of the proposed cost in conjunction with the Technical Officer. (REPEAT FINDING)

CONSIDERATION:

The survey team recommends, that the NASA/HQ Office of Procurement, Contracts Management Division work with the SBIR Program Office and Patent Counsel to develop appropriate provisions to include in all SBIR awards that address the requirement of Section (8) (b) (4) of the latest SBIR Policy Directive.

3 . SIMPLIFIED ACQUISITIONS

Numerous simplified and commercial acquisition files were reviewed . All of the simplified files reviewed were sole source. The quality of the documents for sole source acquisitions was often poor. In accordance with FAR Part 13.106 (b) (1) acquisitions not exceeding the simplified acquisitions threshold should only be based on “one source being reasonably available”, with sufficient technical information to allow that determination to stand.

At least half of the files were tabbed and contained a file content checklist making it easy to locate required documentation. In some cases a price analysis was not performed and in other cases it was insufficient to justify that a fair and reasonable price was being obtained for the government. The price analysis when done, would often reference a price list or historical data but that data would not be placed in the file. Many of the Simplified Acquisition files were awarded sole source, based upon this occurrence, price analysis is paramount to ensure that the government is receiving items at a "fair and reasonable" price.

Three BPA’s reviewed were for secretarial support. Authority was properly sited, the files included set-aside considerations signed by the Contracting Officer and the Small Business Specialist and overall, the files reviewed were well organized and properly documented. While there is no policy directing the Contract Specialist to do so, in the future a Memorandum to the File should be developed to document why the award of a BPA is in the best interest of the government based on the reasons given at FAR 13.303-2.

FAR Part 5.301 requires contracting officers to synopsize awards exceeding $25,000 unless certain conditions apply. The files that were reviewed that required a pre-award synopsis did include it in the file. However, the files reviewed that did require a post award synopsis did not contain the documentation in the contract file.

STRENGTH:

In the 2004 survey, many of the files were not well organized with miscellaneous correspondence filed on top of the file content checklist. This survey has revealed significant improvement. Use of the file checklist for simplified acquisitions has helped improve the organization and quality of the file documentation. The files reviewed were well organized and file contents were easy to locate.

WEAKNESS:

(1) None of the files reviewed over $25,000 included a post award synopsis. Except for contract actions described in paragraph(b) of FAR 5.3 and as provided in 5.003, contracting officers must synopsize through the FedBizOpps awards exceeding $25,000.

(2) The Procurement Officer needs to take efforts to improve simplified acquisitions specifically in the areas of sole source justifications and price analysis.

CONSIDERATION:

The survey team recommends consistent use of FAR 13.303-3 for preparation of BPA. A Policy should be developed requiring a memorandum to the file to detail why the award of a BPA is in the best interest of the government based on FAR 13.303-2.

4. JSC Bank Card Program

As of February 1, 1999, each JSC organization is required to use the government purchase card for all allowable purchases of $2500 and below. JSC currently has 253 active cardholders and at least 253 approving officials. Potential cardholders and approving officials are receiving training prior to receiving their Procurement Officer Delegation Letters. Also, the single and monthly purchase limits are included in the Procurement Officer’s delegation letter as detailed in PIC 03-20 “Purchase Card Account Management”. The survey team reviewed all of the current policies, procedures, and records regarding the JSC bankcard program and found it to be extremely well managed, well documented, and well reported. JSC provides an outstanding website providing extensive information such as the user’s guide, online refresher presentation, and instructions on specific bankcard issues, responsibilities and other areas.

The JSC bankcard manager conducts training, auditing and is readily available to users to discuss problems and issues. The bankcard manager maintains a record of each cardholder’s designation, credit and purchase limitations, purchase card training, and any changes to the cardholder’s limitations. Two purchase cards were confiscated in 2005. Furthermore, a spreadsheet of individuals deleted from the bankcard system with the reason for deletion is maintained.

Refresher training is to be taken annually. Training is provided every other year in-house and the odd years on-line. The bankcard manager is also responsible for initial training and refresher training of users and approving officials and updating their records appropriately. The Survey Team interviewed several bankcard holders, one individual from the office of procurement and the others from various technical directorates. All respondents indicated a general level of satisfaction. All respondents believe the training and audit process are adequate. During the random interview one individual still had their purchase card but had not used it in over a year because they are detailed to an organization where the card is not needed.

STRENGTH:

JSC is commended for the excellent effort associated with the Bankcard Program. The bankcard manager does an excellent job in overseeing the program, training, auditing, and many other areas of the program. They also maintain an excellent website for the bankcard program.

CONSIDERATION:

Cards from individuals on detailed assignments to another organization where the need doesn’t exist for a purchase card should be retrieved and reissued once they return to their permanent organization.

5. Construction Contracts

The survey team reviewed six active construction contracts including two indefinite delivery-indefinite quantity contracts. The review focused on the pre-award and post-award phases and included several contract modifications.

The contract specialists ensured the safety officer reviewed all construction project packages during the acquisition phases and recommended the FAR and NFS safety and health clauses to be inserted into the solicitations and contracts. The contracting officers forwarded all contractor submitted safety and health plans to the safety officer for review and coordinated all required changes with the contractor. Notices to proceed were not provided to the contractor until the safety and health plans were approved, however, the approved plans were not formally included as attachments to the contract in accordance with NFS 1852.223-73 Safety and Health Plan.

Most of the construction progress payment invoices reviewed did not include supporting documentation as required in FAR 52.232-5 and PIC 00-10. This clause lists specific information that must be submitted as part of a contractor’s request for progress payments, including supporting data listed in paragraph (b)(1), specifically the contractor certification statement and notification/certification of subcontractor payments. In addition, a contractor’s request for payment must also include the use of Standard Form (SF) 1443 ‘Contractor’s Request for Progress Payment” as prescribed in FAR 32.503-1. Contracting Officers did substantiate that the work was accomplished prior to approving invoices for payment.

Modifications are well documented and include adequate justification for change, including justifications for increased price and extending contract performance.

It should be noted that the entire file for contract NNJ04JI55C was outstanding. Although a moderately sized construction project ($7M), both the placement and contract administration files showed pride and excellence. This file could be used as a training tool for other construction contracts.

WEAKNESS:

Contracting officers should ensure that construction invoices contain the required documentation and certifications, specifically in relation to invoice certification and subcontractor data, and SF 1443 usage in accordance with FAR 52.232-5 and PIC 00-10.

CONSIDERATION:

(1) The JSC Procurement Officer should ensure that safety and health plans are formally incorporated into the contract by modification upon approval in accordance with NFS 1852.223-73(d).

(2) NASA/HQ Office of Procurement, Contract Management Division should modify the language at NFS 1852.223-73, Safety and Health Plan to clarify the need to formally incorporate the Safety and Health Plan for construction contracts by modification.

6. A-E Contracts

The Procurement Management Survey Team reviewed three architect and engineering service IDIQ contracts and twelve delivery orders. All the files reviewed contained detailed government estimates, technical evaluations, and price negotiation memorandums.

The Johnson Space Center has an established A&E evaluation board to receive and maintain data on firms wanting to be considered for A/E contracts. Board membership consists of professional engineers and procurement officials. The Boards' rankings of offerors, evaluations, and recommendations of offerors to hold discussions with were well documented and all files contained written selection reports.

All of the A&E contracts contained the appropriate FAR Part 36 clauses. Many of the delivery order files lacked consistency in file set-up. All of the delivery orders contained SOW’s; however two of the SOW’s did not include a total construction estimate which is necessary in order to remain below the 6% design fee limit as required by the Brooks Act. Four of the delivery orders reviewed exceeded the 6% design fee limitation. This issue was limited to the one contract for pre-construction design. Starting with the 7th delivery order, the specialist identified the design limitation in the price negotiation memorandum.

WEAKNESS:

Contracting Officers should ensure the negotiated price for producing and delivering the designs, drawings, and specifications are within six percent of the estimated cost of construction per FAR 15.404-4(c)(4)(i)(B).

Twelve deviation request files submitted over the last two years were reviewed for compliance with FAR Part 1.4 and NFS 1801.4. With three exceptions, the requests met the FAR and NFS requirements, were well-documented and timely, and did not require additional data or clarification. With two exceptions, all were concurred with by the HQ technical directorates, the Office of the General Counsel, and the Office of Procurement Contract Management Division, and have been approved by the Assistant Administrator for Procurement. The exceptions required additional information and revision or could not be approved due to statutory requirements.

Seven waiver requests were reviewed for compliance with NFS 1832.702-70 regarding NASA policy for waiving conditions necessary to incrementally fund fixed price contracts. The requests contained the appropriate justifications and approvals, and the approval records were being maintained by the Procurement Officer during the fiscal year as required by NFS 1832.702-70(f).

8. Metrics

JSC collects 9 metrics. Of those 9 metrics, 6 (Award Fee Average Score, Award Fee Timeliness of payment, Office of IG (open recommendations), Prime Socioeconomic Goal Achievement, and Subcontracted Socioeconomic Goal Achievement) are reported to the JSC Metrics webpage, which is used to keep senior center management abreast of issues throughout the Center. These metrics are reported using red, yellow and green to define how the organization is performing.

Furthermore, the JSC OP collects metrics for its internal use on customer satisfaction. An email is sent from the OP Team Leads to the Technical Directorate(s) they support requesting feedback on how well the Division is serving the customer. This data is reported to the Procurement Officer using red, yellow and green to define how the organization is performing. It also collects Corrective Action reports using the Past Performance Database and Contract Award Fee Process in order to request corrective actions from Contractors when they are not performing well.

Finally, the JSC OP collects UCA data that is reported to Headquarters in order to gauge how effectively the Center manages the definitzation of its open UCAs.

STRENGTH:

JSC Office of Procurement is commended for collecting an array of metrics that are useful in determining the health and management of the JSC Office of Procurement.

9. Internal Policies and Procedures

The Procurement Policy and Systems Office is responsible for establishing and maintaining internal policies and procedures at JSC. The Policy and Analysis Team is primarily responsible for policies and procedures while the Data Systems Team is responsible for maintaining the JSC Procurement website. Internal policies and procedures include JSC Procurement Instructions (JPI); JSC Procurement Advisory Notices (PANs); JSC Procurement Information Circulars (PICs) and other relevant guidance and information.

The JPI establishes the uniform operating instruction for the JSC Procurement organization. PANs are used to establish procurement policy and procedures not suitable for publication in the JPI. PICs provide a forum for disseminating short term information of interest to the procurement organization.

The Procurement Policy and Systems Office maintains the JSC Office of Procurement Internal Home Page. The website provides numerous links which make a wide variety of procurement related information available to the procurement and technical communities. The information provided on the website is divided into several different categories:

• What’s New – provides notice of newly posted documents and procedures
• Procurement Master Documents – contains procurement regulations, policies and guidelines as well as JSC Work Instructions
• Internal Acquisition Procedures – provides JSC information, policies and guidelines
• Procurement Business Page – provides access to JSC business opportunities, NAIS, etc.
• Commercial Purchase Card Program – contains policy, guidance and information on the purchase card program at JSC
• Procurement Reports – provides access to databases of various procurement reports
• Procurement Forums – contains the schedule for the monthly procurement forums and presentation charts from previous forums
• Procurement Calendar – contains the calendar of scheduled meetings, training opportunities and other events relevant to the procurement organization
• Other Useful Links – provides access to other NASA and Government websites relevant to procurement issues

It is noted that most of the pages on the website are kept current however the “What’s New” page has not been updated since December 2004. According to the JPI, the purpose of this particular page is to provide a summary or notice of newly posted documents and policies, if this site is to be maintained it should be updated. A list of current NASA Procurement Notices (PNs) is maintained within the Procurement Master Documents page, however it would be helpful to procurement personnel to be advised via the “What’s New” of recently issued PNs (i.e., the recent language regarding Publicizing Contract Actions).

The JSC Procurement Coordination and Approval Matrix provides detailed description of the coordination and approval levels for a wide variety of procurement actions. The matrix distinguishes between dollar value, contract type and complexity and provides clear guidance for contract specialists, contracting officers, and managers.

The Office of Procurement Unique Work Instructions covers ISO processes such as procedures for control of quality records, quality systems documents, corrective action procedures, and non-conformances for simplified acquisitions. They are maintained on the JSC Procurement Website’s Organizations Master list. When accessed, the documents appear with the ‘track changes’ mechanism engaged. This feature makes it difficult for the reader to easily follow the current or final version of these quality control documents.

The JSC Source Selection Manual last updated in Jan. 2006 contains useful information both generic and JSC specific regarding the source selection process for actions greater than $10M.

STRENGTHS:

(1) The JSC Procurement Website contains a wealth of information that is extremely useful to not only the procurement community but the technical community as well. It provides policy and guidance, templates, samples, best practices information and general reminders regarding various procurement related procedures. For the technical community, the website contains extensive information regarding COTR responsibilities and purchase card policies, procedures and guidance. The JSC Procurement Office is to be commended for making this excellent resource available.

(2) The JSC Coordination and Approval Matrix provides clear and detailed guidance to the procurement organization for a wide variety of procurement actions. The matrix covers almost every conceivable procurement action and is a useful tool for procurement personnel at all levels from the entry level professionals to procurement managers.

CONSIDERATIONS:

(1) The JSC Policy Office should post the quality control documents (Unique Work Instructions) in final version rather than in ‘track changes’ mode. If there is a need for individuals to view the document edits, this should be made available as supplemental information.

(2) The JSC Source Selection Manual is a useful tool however the Procurement Office should consider using the standard Agency Guide (currently available in draft form) as its primary resource for source selection information and supplementing it with JSC specific information as necessary. A considerable amount of time and effort was invested in developing the Agency-wide source selection tool to provide consistency in processes across the agency.

(3) The JSC Policy Office should ensure that the various pages on the Procurement Website are kept current.

10. Adequacy of Documentation for Awards Resulting from Broad Agency Announcements (BAA)

Three contract files were selected for review. All three were awarded as a result of an Exploration Systems Mission Directorate (ESMD) BAA released from Headquarters.

The resultant contracts were well-documented and included all necessary supporting documentation in the files, specifically, all of the Headquarters documentation leading to selection and all subsequent JSC documentation required from selection through contract award. The files were reviewed and approved at the appropriate level at JSC.

11. Exploration Systems Procurement Office

The survey team reviewed the organization, structure and contracts of the Exploration Systems Procurement Office. The office administers the Crew Exploration Vehicle (CEV) contract. The CEV is predicated on the Nation’s Vision for Space Exploration that sets a goal of developing a new Crew Exploration Vehicle by 2014 that is capable of carrying astronauts beyond low Earth orbit and a goal of landing astronauts on the Moon no later than 2020. The CEV acquisition is a phased down select awarding two phase 1 contracts in June 2005, with an anticipated phase 2 down select to a single contractor in 2006. The Commercial Orbital Transportation Services (COTS) Demonstration procurement is also being solicited for Earth to orbit space flight demonstrations of cargo and crew transportation capabilities.

The survey team concentrated on reviewing the following:

• Staffing and skill levels of 1102 personnel
• Modification of the Small Business Subcontracting Plan
• Safety and Health Requirements
• Justification for Other than Full and Open Competition (See specific section of the report)
• Exercise of Options
• Contractor Insurance/Pension Reviews

The current level of procurement staffing and skill mix appears to be adequate for the workload and complexity level of the procurement actions. The office manager and the deputy are senior COs, who worked the initial CEV procurement conducted at NASA HQ by the Exploration Mission Directorate. The Phase 1 contracts are administered by two experienced COs assisted by a contract specialist. The phase 2 source board is staffed by a senior CO assisted by senior and junior contract specialists. The COTS procurement is staffed by a senior CO (headquarters detailee) and a contract specialist. The staff is further complemented by analysts from the Pricing Team staff on an as needed basis.

Pursuant to FAR 19.702(a)(1) submission of a revised subcontracting plan is required for modifications exceeding $500K. Adherence to this requirement for each CEV change exceeding $500K would necessitate nearly continuous updates to the plan, given the large volume of high dollar change traffic on that contract. The CO requested an alternate procedure from FAR 19.702 to allow plan revisions on a semi annual basis to address changes that have occurred during the previous six months. The response from the NASA Office of Procurement allowed the contract management team to utilize a revised update procedure for semi-annual updates to the subcontracting plan.

A Safety and Health Plan was not required for the Phase 1 effort. A safety plan is required for the Phase 2 effort.

All procurement actions were full and open competition thereby negating the requirement for a Justification for Other than Full and Open Competition. There are no options in the current contracts.

Contractor Pension/Insurance Reviews (CIPR) for all contractors were current with copies available in the Exploration Procurement Office. There were no CAS non compliant issues directly affecting the current contracts.

STRENGTH:

The JSC procurement office should be commended for the staffing and management of the exploration systems CEV and COTS contract efforts.

12. ADMINISTRATIVE SUPPORT SERVICES CONTRACTS

a. NNJ05JD77B- Administrative/Clerical Support Services

This contract, entitled “Administrative/Clerical Support Services,” was awarded to REDE/Critique on February 4, 2005, with a period of performance through

November 30, 2009, should all three option periods be exercised. This ID/IQ contract established amounts of $240,000 and $2,600,000 as the minimum and maximum amounts respectively. The contract contained a one-month phase in period with full up responsibility commencing on March 1, 2005.

REDE/Critique recently completed their first year of contract performance on February 28, 2006. Consultation with both the cognizant Contract Specialist and Contracting Officer’s Technical Representative (COTR) indicated that, to date, performance has been highly satisfactory. As of the date of this survey, no NASA Form 1680, “Evaluation of Performance” has yet been prepared.

Review of the contract award files indicated that, in general, the documentation supporting the award was appropriate to the acquisition and filed under the appropriate tabs. Review of this documentation also indicated that it was both thorough and in accordance with applicable guidance. It was noted however, that though subject to the Service Contract Act of 1965, no SF 98 signed by the Contracting Officer could be located in the award files. In addition, no Department of Labor response to the notice, forwarded by the Center’s Industrial Labor Relations Specialist, could be located.

Utilizing the appropriate NASA Form 1634, Contracting Officer Technical Representative/ Alternate COTR Delegation, both a COTR and Alternate COTR have been appointed and have accepted the delegations. It is noted that Block 3 (M) of each delegation referenced inclusion of the New Technology clause which was not included in the contract and which would not be appropriate based on the 8(a) status of the awardee.

The contract utilizes Delivery Orders to provide the required effort, each DO type being tied to the appropriate non-exempt labor category and associated exempt supervisory labor. For instance, DO-1 provides Secretary II labor and supervisory labor appropriate to the effort. Subsequent funding actions for each are supplemental to the original DO. For instance, DO 1-1 provided additional funding for the Secretary II and supervisory labor. Both the Contract Specialist and COTR indicated that this system appears to work efficiently.

The contract is Firm-Fixed Price ID/IQ as is therefore considered to be 100% performance based. The contract utilizes a surveillance plan executed by the Contracting Officer, COTR, and Chief of the Quality and Flight Division of the Safety and Mission Assurance Directorate. The surveillance plan utilizes an oversight rather than insight approach and delineates performance standards tied to each WBS section of the Performance Work Statement. Scrutiny of this plan indicated however, that the standards are actually a restatement of the PWS requirements of Attachment J-1. Therefore, no “performance standards” with corresponding Maximum Allowable Defect Rates, weights, and deductions actually exist. Consultation with the COTR indicated that any defects in performance (only one to date), are discussed with the contractor and, if appropriate, could be reflected on the NF 1680 evaluation. However, emphasis is placed on corrective action.

The 58-month contract just reached the 12-month performance milestone. Based on comments received, no major problems have been realized and the contract appears to promote a level of performance desired by the requiring organization.

b. NAS9-01120- Financial Administrative Systems Support

This contract, entitled “Financial Administrative Systems Support,” was awarded to Science Applications International Corporation on August 24, 2001, with a period of performance through August 23, 2006, should all three 1-year option periods be exercised. This CPFF ID/IQ contract established amounts of $200,000 and $9,500,000 as the minimum and maximum amounts respectively.

Consultation with the cognizant Contract Specialist indicated that, to date, performance has been highly satisfactory. NASA Forms 1680, “Evaluation of Performance” are prepared annually, discussed with SAIC, and entered into the Past Performance Database.

Review of the contract award files, which utilized Midrange acquisition procedures, indicated that, in general, the documentation supporting the award was appropriate to the acquisition and filed under the appropriate tabs. It was noted that though the contract utilized predetermined labor rates, it was not subject to the Service Contract Act of 1965, as all labor was exempt from the requirements of the act.

Utilizing the appropriate NASA Form 1634, Contracting Officer Technical Representative/ Alternate COTR Delegation, both a COTR and Alternate COTR have been appointed and have accepted the delegations. The originals of each appointment are maintained in the award file.

The contract utilizes CPFF Delivery Orders to provide the required effort. Contract effort is currently being provided by Delivery Order No. 6. It is noted that Modification No. 908 converted contract funding from Delivery Order funding to funding of the contract as a whole in order to reduce administrative burden. Feedback regarding contractor performance is in the form of day to day contact with SAIC, meetings as required, and utilization of the annual NASA Form 1680, which allows the contractor to provide responses to the evaluation of their performance provided on the form.

The contract is nearing completion and is in the process of being recompeted. Based on comments received, no major problems have been realized and the contract appears to promote a level of performance desired by the requiring organization.

13. Space Shuttle Procurement Office- Space Flight Operations Contract (SFOC) NAS9-20000

a. Contract Management

Due to the Columbia accident in February, 2003, division of the work in the Space Shuttle Procurement Office has changed since the last survey report, which was completed in May 2004. In addition, the activities on the SFOC contract have also significantly increased. At the time of the last survey, one team was entitled the Contract Procurement Team. This team primarily managed contract modifications to the prime SFOC contract. The other team, called the Contract Management Team, primarily managed the smaller support contracts and handled other administrative issues. Since that time, the sheer volume of activities necessitated the establishment of two teams within the Shuttle procurement office that perform essentially similar functions.

The SFOC contract with United Space Alliance (USA) was set to expire on September 30, 2004, shortly after the last survey review was completed. This contract contained provisions to exercise either two 3-month options or one 2-year option. If used, the options were required to be exercised by July 31, 2004. It was decided prior to this date to exercise the 2-year option period. Therefore, the contract is currently extended through September 30, 2006. A follow-on contract is currently in process.

The SFOC contract uses the Space Program Integrated Contract Environment (SPICE) for contract management capabilities. SPICE is an internet based database application that provides contract management capabilities including conformed contracts, modification logs, electronic copies of modification documents, and reports.

STRENGTH:

Continued use of the SPICE database system provides a valuable tool for contract management and administration. This is a tremendous benefit considering contract complexity and the number of contract personnel that support SFOC contract administration.

b. Contract Changes

Since the last survey of the SFOC contract completed in May 2004, 236 modifications to the NAS9-20000 contract have been executed by the Shuttle procurement office. This includes 124 supplemental agreements under the Changes clause, 39 administrative modifications, 26 undefinitized contract actions, and 47 funding actions.

Unchanged since the last survey is the process utilized for changes under the SFOC contract. Such changes use an Activity Based Costing (ABC) methodology or “partnering” to streamline this process. Under the partnering process, contract changes are initiated by generating a Change Partnering Agreement (CPA) which is prepared jointly by representatives of NASA, the contractor, and major subcontractors, and which defines the technical requirements, cost, fee, and contractual aspects of the changed SFOC requirements. The contractor also utilizes the CPA to build its cost proposal and apply wrap rates and fee. The Government technical evaluation also validates that the ABC inputs were incorporated into the cost proposal.

The SFOC contract contains a core effort “swing clause” allowing changes up to certain dollar thresholds, depending on the specific work to be issued and completed without an equitable adjustment to the contract. In addition, the contract contains a Program Provisioning Clause that establishes an annual list to cover efforts for which it is possible to anticipate, estimate, and negotiate work required in the near term. This list contains an annual pool established for the performance of provisioning changes not specifically identifiable at the beginning of the fiscal year that are individually estimated to cost less than $1M. The current amount of this pool for 2006 is $9M. Each year, NASA and the contractor partner a Program Provisioning Task List (PPTL) that when definitized adds in scope of work and associated contract value (pool). These efforts are capped at specific dollar amounts and are authorized using a Program Provisioning Technical Direction /Decision Package (PPTD/DP). In the event that effort authorized using a PPDT/DP later becomes the basis for a contract change (e.g. hardware change), the dollars are rolled in the changed effort definitized amount and are not tracked against the definitized pool.

After initiation of the CPA, changes to the contract are completed using several different methods. For changes that exceed the thresholds established in the swing clause and the program provisioning clause, the Contracting Officer issues a change order, or if time permits, a supplemental agreement. For changes under the thresholds, Contracting Officer authorization is required. The last survey review indicated that the previous practice of Technical Management Representative (TMR) authorization of effort under the program provisioning efforts was inappropriate without CO concurrence. Since that time, Contracting Officer concurrence on all actions of this type has been obtained.

Several supplemental agreements for the definitization of contract changes were reviewed in order to determine if the appropriate policies and procedures were being followed. Each file contained the appropriate Change Partnering Agreement which was executed by the required parties. Pre-negotiation Position and Price Negotiation Memorandums were thorough and in accordance with applicable guidance. In addition, comments provided as the result of policy and legal reviews were adequately dispositioned. Two instances were noted where the dates reflected in the Certificate of Current Cost and Pricing Data did not match the dates of negotiation reflected in the Price Negotiation Memorandums.

The SFOC contract contains a “Launch Schedule Variations” clause that requires support for all flights within a given year. Due to the drastic reduction in flights attributable to the Columbia accident and the STS-114 foam loss and subsequent grounding, a credit to the contract would have been forthcoming. However, as a result of the drastic increase in contract activities in support of return-to-flight, this credit was actually offset.

For the contract changes definitized to date since the last survey, virtually all utilized the exception for use of the NASA Form 634, Structured Approach Profit/Fee Objective, located at FAR 15.404-4(c)(6), which states that “If a change or modification calls for essentially the same type and mix of work as the basic contract and is of a relatively small dollar value compared to the total contract value, the contracting officer may use the basic contract’s profit or fee rate as the prenegotiation objective for that change or modification.” No files were identified for review that negotiated a fee different from this rate.

STRENGTH:

The Space Shuttle Procurement Office is commended for handling a high volume of changes in an appropriate manner since the previous survey in May 2004.

c. Fee Payments

Fee payments under the SFOC contract were discussed with the Shuttle procurement office as a follow-up to the May 2004 survey. At the time of the last survey, incentive fee tied to cost incentives had been removed. As a result, SFOC fees were awarded through either award fee (evaluated every six months), a flight performance fee for the achievement of major Shuttle flight milestones subsequent to an actual flight, and a “Program Plus” fee for accomplishment of Program Manager objectives tied to discretionary items of performance. Examples of Program Plus fee awards have been tied to early delivery of “cold plates”, NSTS 07700 Volume 10 mapping, Return-to-Flight, and Columbia Accident Investigation Board (CAIB) recommendation implementations. Consultation with the Shuttle procurement office indicated that this was still the current arrangement on the SFOC contract.

It is noted that since the last review, there have been three award fee periods, one flight performance fee award, and several Program Plus fees awarded. Award fee scores since that last survey review have all exceeded 90%, or Excellent.

d. COTR Delegations and Technical Direction

Due to the size and complexity of the SFOC contract, as well as multiple locations of work performance, the contract allows the COTR to appoint multiple Technical Management Representatives (TMRs) in order to monitor specific work areas. TMR duties and responsibilities are therefore delegated for specific PWS sections. Generally, TMRs are responsible for monitoring contract performance, reporting contract compliance problems, establishing the surveillance plan for their area(s) of responsibility, and documenting surveillance activities. Currently, there are 12 TMRs that have been formally appointed by the CO.

Review of the appropriate files revealed that the original TMR delegations with both the Contracting Officer and TMR signatures were on file. In addition, the COTR and Alternate COTR appointments with original signatures were also in the file. This was not the case during the previous survey in May 2004. Review of the delegation files also indicated that the TMRs were aware of the requirement to take COTR training and had either taken the training or had been scheduled to take the training. Review of the JSC Office of Procurement Home Page, which maintains records on individuals who have taken the training, indicated that the training of these individuals had been appropriately documented.

14. International Space Station Procurement Office

The ISS Procurement Office is divided into two procurement teams. One team is mainly responsible for the prime contract while the other team is responsible for the non-prime contracts. The prime contract team consists of a team lead, 3 contracting officers, 3 contract specialists and a support contractor. The non-prime contract team consists of a team lead, 2 contracting officers, 4 contract specialists and a co-op student. There are currently six active non-prime contracts. Four of the six have contract values above the Master Buy threshold. This survey review focused on both prime and non-prime contract activities.

a. Non-Prime Contract Review

General

A review was conducted on three contracts with varying dollar values and contract types. Focus was placed primarily on contact activities conducted since the last procurement survey. One contract reviewed was recently awarded; consequently both pre-award and post-award documentation was reviewed. Generally file documentation was thorough, the necessary reviews and approvals were obtained in accordance with the JSC Coordination and Approval matrix. Review comments or concerns were properly addressed and documented when present.

Financial Management Reports

The financial management report (533 report) files were reviewed on the cost reimbursement contracts. The ISS resources office provides an earned value assessment of the monthly 533 submissions. The JSC Procurement Website contains a 533 Analysis Guide that states the contracting officer’s responsibility for integrating any analysis performed outside of procurement into the contract file. The guide further states that, “the contracting officer must perform analysis and periodic reviews of the 533 reports to ensure contract compliance and discuss issues with the contractor in a timely manner.”

In one case, the earned value assessment and contracting officer’s or contract specialist’s variance analysis was contained in the contract file prior to September 2005. Since that time only one monthly earned value assessment was present and no evidence of a 533 analysis or review was present. The two most recent 533 reports were not yet filed and no earned value assessments were evident.

The review revealed that early warning analyses on the reports had been conducted but could only account for them through July 2004. The early warning analysis contains a narrative description of the earned value assessment summary, 533 report and variance trends, program management review summaries, etc. It appears that the practice of conducting the early warning analysis ceased and was replaced with a variance analysis of the 533 data via spreadsheet. Neither has been conducted since September 2005.

In another case, the review revealed that 533 analyses was conducted and documented in the file from contract inception through May 2005. From June 2005 through December 2005, it appears that the 533 reports were not properly filed, earned value assessments were not reviewed and no analysis or reviews of the reports were conducted. The two most recent 533 reports were properly filed and a separate folder is maintained with the early warning analysis that includes the earned value assessment; a variance analysis conducted by the contract specialist and a thorough description of both the 533 report and the earned value assessment.

It is noted that personnel on this team could benefit from focused training on conducting analysis and review of 533 reports. There does not appear to be a consistent approach to 533 analysis and review among team members.

WEAKNESS:

The ISS Procurement Office should ensure that contract specialists are fully trained and familiar with the appropriate method of conducting analysis and review of financial management reports. Further, contracting officers should verify that the analysis is conducted and files are maintained properly.

CONSIDERATION:

The non-prime team lead may want to consider establishing consistent practices for all contracts regarding the analysis and review of financial management reports and file documentation as appropriate.

Contract Modifications

Contract modification files were well maintained and documented. Modification files contained detailed logs of contract activity, contract value and funding actions. Again it is noted that the actual method of reflecting the contract activity varied among contract specialists. For one contract, the modification log/history is displayed prominently in the modification folder. In the other contract, the modification log/history is incorporated as an official part of each contract modification. Modifications involving negotiations with the contractor contained the necessary documentation and appropriate reviews and approvals (pre-negotiation memoranda, technical evaluations, cost/price analysis, negotiation memoranda, etc.).

Voucher Files

Voucher files reviewed were in order and well maintained. Detailed voucher logs tracing contract funding, cost and fee vouchers were maintained and current. Vouchers were date stamped upon receipt and processed in a timely manner.

Award Fee Files

A review of the two most recent award fee periods was conducted. Files were found to be in order with thorough documentation. The appropriate clearances and reviews were received and the evaluation was performed in accordance with the timelines and requirements of the NASA Award Fee Contracting Guide.

b. Prime Contract Review

Contract Modifications

Contract modification files were well maintained and contained the necessary documentation and appropriate reviews and approvals (pre-negotiation memoranda, technical evaluations, cost/price analysis, negotiation memoranda, etc.). Evidence of technical office concurrence on pre-negotiation memoranda over $2.5M is contained in the files as appropriate.

Change Orders

Modification files for contract change orders contained the appropriate documentation including the Change Directive signed by the Space Station Program Change Board (SSPCB); authority to proceed from the Head of the Contracting Activity (HCA); memos to the files documenting the need for the change and the change order modification.

It is noted that the Change Directive clearly states that “the contractor is not authorized to proceed with the change effort until authorized by the Contracting Officer.” However, there is no evidence of procurement participation in the SSPCB activities that require the issuance of change orders. In many cases there appears to be a significant lag in activities between the issuance of the Change Directive and actions by the ISS Procurement office. The team lead indicated that he regularly attends SSPCB meetings and that he and the contracting officer are well aware of change actions prior to the issuance of the Change Directive.

In some files reviewed, the Change Directive was issued 2 – 3 months prior to receiving HCA approval and the issuance of the change order modification. In all cases, the file documentation stipulated that the changes were necessary and urgent. The existence of significant time lapses seems to negate the need for urgency in these cases. When questioned, the team lead stated that frequently the time lapse was caused by the lack of available funds. The HCA will not authorize the change without available funds. Typically, the change is issued promptly after receipt of funds and HCA authorization. The JSC Headquarters Program Operations Procurement Analyst confirmed that the ISS Procurement Office does not move forward on actions without receiving the necessary HCA approval and funding.

Definitization Modifications

Modifications that incorporated the definitization of open change orders or involved other negotiated changes with the contractor were also reviewed. These files were well documented and contained the appropriate review and approval documentation (pre-negotiation memoranda, technical evaluations, cost/price analysis, negotiation memoranda, etc.). It is noted that in some cases, documents referenced items in the file under tabs that were not present in the consolidated contract file under review (i.e., Tab 65 – Technical evaluation or Tab 72 – cost analysis). Both the contracting officer and the team lead indicated that the files are often so voluminous that the background data and documentation is filed separately and maintained in secure access vault.

Award Fee Files

The prime contract award fee plan indicates that the contractor is evaluated on ‘on-ground’ performance and ‘on-orbit’ performance separately. The ‘on-ground’ performance is evaluated by a Performance Evaluation Board (PEB) with a Fee Determining Official making the final determination. The ‘on-orbit’ performance is evaluated by a ‘super’ board that operates separately from the PEB with a Performance Incentive Determining Official making the final decision. The ‘super’ board is designed to make an independent assessment of the contractor’s on-orbit performance and may include non-governmental members. ‘On-ground’ performance is evaluated on a 6 month basis while ‘on-orbit’ performance is evaluated annually.

Award fee documentation for both the on-ground and on-orbit evaluations during the last two years were reviewed. Files were thoroughly documented and maintained. The appropriate clearances and reviews were received and the evaluation was performed in accordance with the timelines and requirements of the NASA Award Fee Contracting Guide. In the case of the on-orbit evaluation, the necessary disclosures, clearances and approvals were received for the non-government board members.

Past Performance Documentation

Contractor performance is recorded in the Past Performance Database appropriately. It includes adjective ratings and an overall summary of the contractor’s performance and cost performance for the award fee (on-ground) period as appropriate. The summary also mentions the award fee score and notes the discussion with the contractor. The information is recorded into the database in a timely manner.

STRENGTH:

The ISS Procurement Office is to be commended on the thoroughness and consistency of the file documentation on the prime contract. Information is easily accessed and necessary reviews and approvals are present.

CONSIDERATIONS:

(1) The ISS Procurement Office should consider the possibility of having a procurement representative concur on Change Directives that result in the issuance of undefinitized contract actions. Further, contract change order files should be annotated to reflect procurement’s involvement in the early stages of the change process.

(2) The ISS Procurement Office should ensure that contract files are documented to explain significant lapses in time between the issuance of the Change Directive and subsequent HCA approval and change order execution.

(3) The ISS Procurement Office should ensure that contract files are annotated to reflect the location of file documentation stored separately. This will eliminate the appearance to the outside reviewer that certain documents do not exist.

15. Katrina Review

The Survey Team reviewed all contracts that were identified as being used in support of Katrina recovery at the Michoud Assembly Facility and Stennis Space Center. The files were documented indicating that the work was in scope and the price was fair and reasonable for the requirement. In a time of emergency, JSC is to be commended for providing invaluable support to the affected Gulf Region.

STRENGTH:

The JSC Procurement Officer is commended for its agency support and delineating the files as Katrina related activities. This identification allowed the Agency to track and report to Congress in an expeditious manner.

16. Self-Assessment

The team reviewed JSC’s self-assessment program to ensure the Center was in compliance with the Agency Self-Assessment Guide, and focusing on areas in need of improvement. The JSC Procurement Office has been conducting assessments in accordance with the procedures outlined in the NASA Self-Assessment Guide. JSC has been performing its self-assessments on a quarterly basis. An in-depth analysis was performed on all the reports issued in FY 2005. Review topics for self-assessment mirrored some topics that had been identified in the last JSC procurement survey report conducted in CY 2004. The report was well organized. It included the purpose, findings, overall assessment and how to handle corrective actions. However, it is suggested that future self-assessments highlight all weaknesses identified in survey reports. Since the self-assessment is being conducted on a quarterly basis, each self-assessment may incrementally cover a portion of the weaknesses along with the procurement office other areas of interest for that quarter.

Since the last survey, JSC has developed a consistent approach for performing its self-assessment program and distributing the findings in a timely manner so corrective action(s) can be implemented by procurement managers and training if appropriate.

STRENGTH:

The JSC Procurement Officer is commended for developing a more consistent self-assessment approach and how to disposition corrective actions in a timely manner.

CONSIDERATION:

It is recommended that all procurement survey weaknesses be highlighted and focused on to avoid repeat findings in those areas. The weaknesses may be reviewed incrementally since self-assessments are conducted on a quarterly basis.

17. 1102 Career Development Training

The JSC Acquisition Training Coordinator (ATC) continues to provide outstanding support to the Career Acquisition Training Program in support of JSC’s procurement workforce. Interviews with the ATC, procurement personnel and an examination of training files and the training and certification database were included in this review.

JSC maintains a comprehensive database that includes individuals warrant status, certification, classes taken/classes required, degree held and whether the individual has met the 24 business hour requirement. A random check of individual training files verified the accuracy of the database. The files also included all class certificates, certification documents, IDP’s and requests for waiver.

JSC has 120 people at the 1102/GS-12 level or above which are targeted for certification and all 120 are certified at some level. JSC has been successful in bringing in several entry-level contracting personnel. These personnel have established training profiles in place in an effort to provide realistic goals to promote career progression. An example is an employee at the GS-7/9 level, being certified level 1 and completed training at level II within two years. The JSC Procurement Officer and Training Coordinator continue to encourage employees to take advantage of available training.

STRENGTHS:

(1) The JSC Procurement staff is commended on achieving 100% certification of 1102/GS-12 and above.

(2) JSC is commended for placing a level of importance on acquisition training that provides even the newest in the procurement workforce with the tools required for career success.

18. Interagency Agreements

Ten interagency agreement files were reviewed for compliance with FAR 17.5 regarding Interagency Acquisitions under the Economy Act; NFS 1817.70 regarding Acquisitions with Military Departments; and NFS 1817.72 regarding Interagency Transactions.

All of the files contained the required Determinations and Findings (D&F). Some used the template from the VPO, and some did not. Those that did not use the VPO template appeared to contain more justification regarding the determination than those that used the VPO template, most of which simply checked the boxes. Some D&Fs (non-VPO type) cited both the Economy Act and the Space Act as the authority. The appropriate justification is the Space Act. The JSC Coordination and Approval Matrix requires D&F concurrence by the Procurement Office Manager and the technical office. Neither concurrence was evident in any of the files.

All of the files used the NDPR Form 523 in accordance with NFS 1817.7002. A couple of the agreements had full statements of work or a detailed list of task descriptions, but most only contained very general (one phrase) work statements such as “provide support to the ABC experiment”, with no further elaboration. Most of the agreements cited the Space Act as the authority, however, some did not cite any authority, one cited the Economy Act, and one cited both the Space Act and the Economy Act. All of the agreements contained a payment provision requiring submittal of a final invoice within six months of completion of the order, and all contained a property clause even though it did not appear to be required in some agreements. None of the agreements contained a provision for procedures for resolution of disagreements between the parties in accordance with FAR 17.504(c).

WEAKNESS:

The JSC Office of Procurement should ensure compliance with the FAR and NFS requirements for interagency transactions regarding information to be included in the D&F and the NDPR, and ensure review and concurrence in accordance with JSC policy.

CONSIDERATION:

The JSC Office of Procurement should consider encouraging the addition of more narrative justification when using the VPO D&F template and inclusion of a more detailed description in the Dnepr’s of the work to be performed. The Office of Procurement should also consider developing a guide for preparing interagency acquisitions similar to GSFC’s internal Procurement Circular 99-3. Many of the provisions included in the GSFC Procurement Circular are available in the Contract Management Module for use when developing an interagency agreement.

Return to Contents