NMO 2005 Procurement Survey

The NASA Headquarters Office of Procurement conducted this Procurement Management Survey at the NASA Management Office (NMO) at the Jet Propulsion Laboratory (JPL) under the authority of NASA Policy Directive 1000.3, The NASA Organization. The survey was conducted from July 18 through July 22, 2005.

An exit briefing was held at the NMO on July 22, 2005 to discuss the survey findings. Dr. Robert Parker, Director and Mr. Jeff Lupis, ACO/Procurement Officer represented the NMO. Mr. Tom Luedtke, Assistant Administrator, and Ms. Monica Manning, NMO Survey Manager represented Headquarters.

This report serves as a basis, in part, for fulfilling internal control requirements in accordance with the Federal Manager's Financial Integrity Act of 1982 (P.L. 97-255).

Monica Manning
NMO Survey Program Manager
Analysis and Assessment Team
Office of Procurement, NASA Headquarters

OVERVIEW

The Procurement Organization at the NASA Management Office is providing meaningful support to their program customers. The Procurement Management Survey Team interviewed randomly selected technical and program representatives to ascertain any issues or concerns with the current processes. A significant degree of customer satisfaction is apparent from the interviews with the technical community. Additionally, interviews with numerous acquisition professionals, at all levels of the NMO organization, regarding the effectiveness of the procurement office were conducted.

The interviews of technical and acquisition personnel are given roughly equal survey emphasis as the review of contracting actions which focused on compliance with procurement statutes, regulations, and procedures. The thrust of the compliance portion is directed towards systemic procurement processes, as opposed to focusing on individual file anomalies. Attention was also directed to current procurement innovations, both Agency-wide and Center specific.

The results of the compliance reviews and the interviews are detailed as strengths, weaknesses, and areas of consideration. Also, to promote the exchange of successful lessons learned and innovative procurement methodologies between Centers, the team sought to identify NMO processes or initiatives that might benefit other Centers and, likewise, looked to other Centers for suggested approaches that might be exported to the NMO.

The exit conference at the conclusion of the survey consisted of a direct exchange of observations and ideas between the participants. To emphasize Center ownership of the resolution of any identified weaknesses or considerations, the survey follow-up process will focus on the corrective actions or initiatives undertaken by the Center. At an appropriate interval (approximately six months after this report is issued) the NMO Procurement Officer will brief the Assistant Administrator for Procurement and the survey team leader on Center achievements in these areas.

The survey could not have been accomplished successfully without the support from Johnson Space Center, Marshall Space Flight Center, Stennis Space Center and the following individuals:

ORGANIZATION - MANAGEMENT

1. Organization -- Management

The basic organizational structure of the NMO's Contract Management Section remains largely unchanged since the 2002 Procurement Management Survey. The staff is organized into two functional teams, Contract & Task Placement and Contract Administration. These teams are lead by GS-14 contracting officers who report to the Procurement Officer. As a change to the organization that was implemented in 2003, two non-1102 Administrative Specialists (GS-301 positions) were added to the organizational structure, in order to focus additional resources on support of office business systems. One senior staff (GS-13) has overall responsibility for business systems operations, and a second individual (GS-11) provides additional business systems operations support with emphasis on inter-agency funds transfers.

The primary responsibility of the Contract & Task Placement Team is to award and administer task orders, both NASA direct funded and orders funded by other Government agencies, under the JPL prime contract. The team is also responsible for the small number of other procurement actions awarded and managed by the NMO, most notably the Australian and Spanish Deep Space Network contracts. The Contract Administration team oversees a wide variety of contract administration functions including subcontracting, small business, award fee administration, contract closeout and overall procurement policy. These functions also include on-sight monitoring and approval of JPL business systems including accounting, compensation, purchasing, and establishment of final cost rates under the contract. Such functions are generally delegated to DCMA by NASA Centers but are retained by the NMO consistent with their role as providing all contract administration necessary for oversight of the JPL prime contract.

The NMO has twelve contracting officers, including the Procurement Officer and the two Team Leads, two administrative specialists, and one procurement assistant. Contracting Officer Warrant Authority is established in an office Standard Operating Procedure ("SOP") and it is noted that the NMO instituted policies in 2004 that standardized warrant authority and provided limited warrant authority for more junior level personnel. Overall, staffing is at a substantially higher level than during the 2002 Procurement Survey, 15 FTE vs. the previous 12 FTE. However, it is noted that two of these individually are almost exclusively dedicated to providing support of business systems largely necessitated by the agency's move towards the Integrated Financial Management (IFM) Program. The current Procurement Officer has been on board for approximately two years of what is designed to be a four- to five-year rotation in that position.

Despite a somewhat limited staffing situation, the NMO has initiated a wide range of office procurement initiatives over the past two years. Many of these initiatives have focused on moving the NMO towards a more proactive role in providing value added contract administration support. Most notably, NMO's cost control (estimated value at completion) report, task plan/cost estimate template, major subcontract reviews, and program executive training have all increased the rigor imposed by the office in ensuring that cost and performance issues are addressed contractually in a timely manner. The office has coupled these improvements with internal office policies that have allowed staff to work more efficiently and emphasize important administrative processes, which have included a purchase request tracking system, standard operating procedures for key office functions, improved framework for managing the award fee process, and a more rigorous system for tracking contract reporting requirements.

Due to the size of the NMO Procurement Office, interviews were conducted with the entire 1102 workforce to gain insight into their perspectives on the effectiveness of the organization. Interviewing each procurement professional allowed the Survey Team Program Manager to make an assessment utilizing full representation of each grade level and branch within the NMO Office of Procurement.

The preponderance of the NMO Office of Procurement workforce stated that morale is good, co-workers and the procurement officer are hard workers and even in the mist of disagreements within the organization, work assignments are accomplished in a professional manner. In general, employees expressed positive opinions regarding management and all feel that the organization will continue to experience positive impacts from the Procurement Officer's initiatives.

Lots of positive adjectives were used to describe management (i.e. the Procurement Officer) such as: caring, dedicated, knowledgeable and pro-active. All emphasized improvements in the procurement office (from process improvements to employee promotions) since the appointment of the current Procurement Officer. However, the staff did recommend that it would like to see more delegation and autonomy which would engender trust and respect in the workforce. It was noted, and communicated in the interviews that to assist management in fostering trust (at all levels) and developing techniques to improve communication, office cohesiveness and positive appreciation of team dynamics; "Team Building Developmental Retreats" are being held on a regular basis. A core team, "Icebreakers" is responsible for developing retreats that address issues that are a hindrance to the organization. In addition, this team is responsible for measuring the appropriateness and effectiveness of the retreats; which is found necessary since many individuals expressed discomfort and concern as it relates to the value of additional retreats in its current format.

While the vast majority of feedback was positive, constructive feedback was provided in the areas of on-site training and knowledge sharing/retention. Most, if not all employees feel they are afforded the opportunity to receive formal training; however, in-house development opportunities such as special projects and focused training sessions are desirable. In addition, there is a concern that knowledge sharing and transfer (i.e. information retention) and the ability to fluently transition workload in a meaningful way as new employees are hired or people leave the organization is not being managed well. Although this has been an ongoing concern, it is noted that standard operating procedures are being developed to facilitate such transitions.

The Survey Team interviewed numerous customers to obtain their input regarding satisfaction with the service provided by the Procurement Office. Individuals chosen for the interviews were selected from the current customer database. Several Program Executives located at NASA HQ were interviewed.

Based on the interviews conducted, the technical community at NASA HQ has limited direct contact and communication with the Procurement Staff at the NMO. The vast majority of customers interviewed stated primary interface is with the Jet Propulsion Laboratory (JPL) Contracts Management Office (CMO) and Task Managers. Since Task Plans and Program Operating Plans are issued and updated by JPL most Program Executives work directly with the Lab and the NMO involvement is seamless to most HQ customers. Most customers are interested in developing a more meaningful relationship with the NMO Procurement Office if there is value added. For example, it was stated multiple times that the signature review process is unclear and only when something is kicked-back does the customer find out a process step is missed; in addition, it was noted that this process seams to be inconsistent at best and identified at the end of a process. Improved relationships would minimize these types of situations.

To assist the NMO in removing communication barriers customers recommended a valuable relationship would exist if the specialist or contracting officer: (1) participated in the monthly and quarterly reviews, (2) developed a working knowledge of the projects and (3) provided outreach to the customers.

The Procurement Officer has expressed commitment to adding value to the procurement process for NMO customers. The NMO has started to address these customer concerns within the past year by implementing several different initiatives such as:

As part of the NMO Office of Procurement long term vision planning, they want to be viewed as a reliable, professional resource to their customers.

At the request of the Procurement Officer, interviews were conducted with the JPL Business Operations Directorate regarding the effectiveness of NMO Contracts Management Section Management and Staff.

The JPL Business Operations Directorate appears to be very satisfied with the Procurement Officers effectiveness in managing the NMO Contracts Management Section and the relationship that has been established with the Directorates. Comments were uniformly positive; however constructive feedback was provided as requested by the Procurement Officer to assist with increasing the office's ability to provide value added contract administration.
Most of the interviewees communicate directly with the Procurement Officer, those relationships are evolving and a healthy government/contractor relationship exists. Although each party has different perspectives there is open dialogue and a willingness to provide solutions to areas of concerns. It's JPL's perspective that the current Procurement Officer is holding JPL, the NMO, and the Procurement Officer position to a higher level of accountability and higher level of expectations.

JPL interviewees noted the initiatives and process improvements occurring at the NMO will be beneficial and await such results. There is a concern that the process improvements while needed be implemented in a more manageable approach. The aggressive approach is at times overwhelming due to the magnitude of impact points to include infrastructure, culture and policy changes they may have to occur.

It was also noted that although relationships with individual procurement professionals continue to improve; continued emphasis could be placed on employee skill mix, response time, standardization and communication.

The Legal Office and the Contracts Management Section appear to have a true partnership in identifying effective ways to reviewing and awarding requirements. A Standard Operating Procedure that outlines "Legal Participation in the NMO Acquisition Process" was issued in March 2005.

An interview was conducted with NMO's Chief Counsel regarding interaction between the Legal Office and the Procurement Office indicated that overall there is a good functional relationship between the two organizations with no substantial barriers. The Chief Counsel expressed the NMO Contracts Management Section is a great group of individuals to work with and would like to see the organization continue to emphasize, encourage, and foster growth and development of procurement professionals.

PRE - AWARD

The survey for pre-award activities review focused on the prime contract, NAS7-03001, between the National Aeronautics and Space Administration (NASA) and the California Institute of Technology, which establishes the relationship for the operation of the Federally Funded Research and Development Center (FFRDC) known as the Jet Propulsion Laboratory (JPL).

The Survey Team reviewed JPL's NMO Master Buy Plan (MBP) submission procedures and processes. The NMO currently submits its MBP electronically through the MBP Database. The two Department Managers are responsible for canvassing the office for submittal information. The information is then provided to a single source for input into the system. This information is provided to the Procurement Officer, who is responsible for final approval and submission to Headquarters. Contract NAS7-03001 was reviewed to ensure the Master Buy Plan submission was documented. The file contained the required documentation.

In accordance with FAR Part 10, agencies are responsible for conducting and documenting market research in a manner appropriate to the size and complexity of the acquisition to determine if commercial capabilities can satisfy NASA's requirements. Contract NAS7-03001 was reviewed for market research. The file contained a very detailed Market Research Analysis that was filed under "Market Research" on NASA Form 1098. The Market Research was also addressed in the JOFOC.

An Acquisition Strategy Meeting (ASM) was conducted at NASA HQ for NAS7-03001 and appeared to follow the FAR, NFS, and the NASA Guide to Conducting ASM Guide. The file contained a Memorandum For Record with resolutions of ASM minutes that was coordinated with Code H.

The survey team reviewed the JOFOC for compliance with FAR and NASA FAR Supplement requirements. The JOFOC cited statutory authority exception 3, "Industrial mobilization; engineering, developmental, or research capability; or expert services." The JOFOC reviewed contained the appropriate reviews, concurrences, and approvals. This JOFOC was well written and contained the appropriate rationale to support the exception.

There were 10 deviations requested for NAS7-03001. All that required NASA Headquarters approvals had received the appropriate approval, with legal concurrence, as required by the NASA FAR Supplement. The core period of performance for this contract is October 1, 2003 through September 30, 2008. The contract includes a provision that will allow the Government to extend or shorten the contract term based on the contractor's performance. Assuming the contractor achieves the highest performance rating, the Government may extend the period of performance for a maximum of five additional years, through September 30, 2013. A blanket waiver was issued by NASA's Code H granting authority to deviate from the requirements of FAR 17.204(e)(i) and NFS 1817.204(e)(i), which limits the period of performance of contracts to five years.

POST - AWARD/CONTRACT ADMINISTRATION

Contract NAS7-03001 was awarded on 26 Nov 2002 to California Institute of Technology for operation of the Jet Propulsion Lab in Pasadena, CA. The contract start date was October 1, 2003. It is a Cost Plus Award Fee contract, with work to be designated by the issuance of task orders. The contract also has an award term provision. The core performance period is five years, which can be adjusted upward or downward by increments of three or nine months, depending upon the rating earned. The contract may not be extended by more than a total of five additional years.

For task orders, the contract does not specify minimum or maximum dollar amounts, maximum performance periods, or ordering periods. Thus, this review does not evaluate task orders for compliance with the aforementioned ordering parameters.

Task orders are awarded based on "task plans" submitted by JPL which contain work statements, cost estimates, and other information, and are approved by various NASA personnel before submission to NMO.

Upon request, NMO provided the files for the 10 most recent task orders issued under the contract. This review is based upon those 10 files.

1. None of the files reviewed contain an NF 1098 Checklist for Contract Award File content as required by NFS 1804.803-70, and were not tabbed in any manner. This makes it difficult to locate various documents or trace the history of a task order.

2. JPL has adopted a task plan template for task plans, which is available on its website. The intent is to improve quality and consistency in the format of information provided, including both technical and cost estimate information. It also forces task plan developers to answer questions pertaining to the existence of potential conflicts of interest, performance incentives, new construction requirements, NEPA compliance, export control, earned value management, schedule, performance period, and deliverables. Two of the ten task orders didn't appear to use the template. Moreover, the Cost Estimate template requires that the "Procurements" line item contain a list of details on separate sheet, each item and cost. Of the ten files reviewed, six contained procurements and three did not appear to contain the supplementary information.

3. The last JPL survey in 2002 contained a Consideration to improve the sufficiency of cost estimates provided with task plans. This was a repeat finding. The finding was based on the fact that there was no price or cost analysis performed on the cost estimates.

FAR 15.403-1 requires the submission of cost or pricing data when the threshold is exceeded, and no exceptions or waivers apply. A waiver was signed for the basic contract on 22 Nov 02, but the waiver doesn't state whether it applies to task orders. The rationale for the waiver (uncertainty of the work to be performed) doesn't seem to apply to the task orders, since task plans define the work to be done. There was no competition held for either the basic contract or any of the task orders. Of the 10 task orders reviewed, 6 exceeded the threshold ($550,000), including one order that exceeds $184,000,000. None of the task orders contained cost or pricing data, or discussed the absence of cost or pricing data. The aforementioned waiver states that subcontractors will be required to submit cost or pricing data, but again no files contained any evidence of this, including a task order which was not within the scope of the contract (a JOFOC was processed). Information other than cost or pricing data was found to be minimal in all files.

FAR 15.404-1 (a) requires that the Contracting Officer evaluate the price reasonableness of offered prices and to perform price or cost analysis when cost or pricing data are required. FAR 15.406-1 requires establishment of prenegotiation objectives before the negotiation of any pricing action. FAR 15.406-3 requires the documentation of any negotiation (Price Negotiation Memorandum). None of the task order files reviewed contains any evidence of price or cost analysis or determination of price reasonableness. One file did contain a memo that questioned various aspects of a cost estimate, but did not demonstrate how that estimate was corrected before the task order was awarded.

NMO recently conducted an office self-assessment which acknowledges some of the issues above. NMO is just starting to take various steps to address the issues.

Prior to the most recent self-assessment conducted within the Contracts Management Section (CMS), the Procurement Officer requested via e-mail dated May 31, 2005 that all staff perform all technical and cost evaluations in accordance with the task order Standard Operating Procedure (SOP). In addition, the Procurement Officer requested that all staff review their evaluations with both their team leads and the Procurement Officer prior to approval.

Subsequent to the issuance of the most recent self-assessment, there were findings indicating that cost estimates were not being submitted by JPL in compliance with the mandated format. CMS has worked with JPL to improve the quality of the cost estimates, as evidenced through a series of upper management meetings, e-mail and formal letter correspondence. Since these cost estimating requirements are a new process for JPL, there has been a learning curve associated with implementing these changes.

The NMO is continually working with JPL to improve the quality of their submissions, with regular feedback and correspondence. In addition, the CMS has been working with the Program Executives at HQ to establish a greater level of dialog regarding the need for Independent Government Cost Estimates, and thorough technical evaluations.

4. The task order numbering system used under this contract appears to conflict with NFS 1804.7103, which prescribes an instrument numbering system, including task orders. The system is a 10-digit number which is required for e-gov, FDPS, and SAP purposes. The award number should identify the center, fiscal year, and the type of award. NMO's task order numbers do not identify this information. NMO states that because the current JPL contract was awarded before the current instrument numbering requirement became effective, the JPL task orders were exempt from the requirement. No other NASA center is known to have such an exemption. Normally, new task orders placed against older IDIQ-type contracts should contain a task order number that is consistent with 1804.7103.

5. One task order contained a Justification for Other than Full and Open Competition because it contained work which could have been performed by the commercial market. The JOFOC stated the pricing was fair and reasonable, but there was no documentation in the file to support this. Also, the authority cited was "Urgent and Compelling," which is questionable since decision to award the work to JPL appears to be cost-based in nature, not urgency-based. Also, the award was not synopsized as required by FAR Part 5. Another question is whether the work should have been awarded as a separate contract to Cal Tech instead of a task order under the JPL contract.

6. One of the Contracting Officers mentioned that the expected turnaround of task orders, once a task plan is received is 5 days. This is not in writing, but is an understanding between NMO and JPL. In order to comply with FAR pricing requirements, this expectation is unrealistic.

7. Changes to task orders are referred to as "Amendments" instead of "Modifications." FAR has long used the former term to apply to Solicitations, and the latter term to apply to contracts (which includes task orders). This could lead to confusion to for contracting personnel.

For this survey, the award fee administration on prime contract NAS 7-03001 was reviewed. While there is no award fee plan as part of the basic contract, there is a very detailed Office Work Instruction (OWI) that is used for the evaluation of the California Institute of Technology.

This contract is unique in that findings are generated here at JPL then ultimately presented to a PEB and subsequently the Fee Determination Official located at Headquarters. Although this could present logistical and schedule difficulties, the NMO is to be commended for the timeliness of Fee Determination and payment to the contractor:

The OWI indicates that the award fee criteria letter should be issued to the contractor no later than the start of the performance period. It was noted that the current periods letter was not sent out until 83 days after the period had started.

When reviewing the Performance Evaluation Board (PEB) charts, there was no indication of cost performance expected compared to cost performance realized (at the total contract or task order level). Further, contract cost performance did not appear to be a consideration in the recommendation of an award fee score to the FDO.

The NASA FAR Supplement (1816.405-274) contemplates that cost control shall be an evaluation factor in all award fee contracts and that its weighting shall be no less than 25% of the total weighted evaluation factors. Currently, cost control is evaluated under "Institutional Management Performance" along with four other items with an overall evaluation weighting of 25%. Although there is not a specific breakout of the weightings of the sub-factors, it is clear that cost control is something less than 25%.

The JPL purchasing system received system approval on July 12, 2002 (a subsequent review and approval was obtained on June 3, 2005). As part of the approval, the NMO and JPL agreed to a number of changes to the subcontracting clause of the prime contract. Other than a few minor exceptions (plant equipment, special tooling, etc.), a threshold of $25 million was established for subcontracting actions that require consent and a threshold of $1,000,000 for the submittal of Advance Notifications. The parties also agreed that certain subcontracts may be identified for special surveillance and would then be subject to the consent requirements of the contract.

The modifications to the subcontracting clause of the prime contract reflect the assessment by the NMO of the level of surveillance required to ensure that JPL is effectively managing its purchasing program and therefore meets the requirements at FAR 44.304. The thresholds noted above appear to provide appropriate insight to the subcontracting activity given the number and complexity of JPL's subcontracting actions. It should be noted that a random sampling of subcontracts under the below noted thresholds is conducted monthly along with a monthly status meeting where JPL outlines acquisition strategies for upcoming procurements giving the NMO additional insight into the quality and appropriateness of subcontracting activities.

More specifically, in reviewing numerous subcontract files and associated consents, it was apparent that there was a very specific process that is used by the NMO for review and disposition. A log has been kept of all submissions, a checklist is utilized for each review and the subsequent approval letters are appropriately documented.

NFS 1842.1502 requires that within 60 days of every anniversary of the award date of a contract having a term exceeding one year, contracting officers must conduct interim evaluations of performance on contracts subject to FAR 42.15. However, NFS 1842.1503 states that evaluations used in determining award fee payments satisfy the requirements related to evaluating contractor performance and do not require completion of NASA Form 1680. Clarification of this policy is provided in PIC 01-12, which requires that award fee evaluations must be summarized on NF 1680 and entered into the NASA Past Performance Data Base (PPDB). However, since the language at NFS 1842.1503 does not refer to the PIC for added clarification, there is a potential for confusion regarding the need to submit inputs on NF 1680s and in the PPDB under award fee contracts.

The NMO has been inconsistent in its application of the policies regarding the submission of NF 1680 data since the last procurement survey. For the current JPL prime contract, NAS 7 03001, no NF 1680 data has been input into the PPDB, nor is there a hard copy of the NF 1680 in the contract file. NMO personnel indicated that there has been some confusion regarding the requirement for such inputs under award fee contracts (see discussion above). However, it is unclear why there are some inputs in the PPDB for the predecessor award fee contract, NAS 7 1407.

Five other contracts/purchase orders administered by the NMO since the last procurement survey have also had mixed results regarding the implementation of the PPDB and maintenance of the NF 1680 as part of the contract documentation. Of the 13 inputs to the PPDB for NMO managed contracts, only six hard copies of the NF 1680s were found in their respective contract files. Further, many of the 13 evaluations that were input into the PPDB were not initiated or completed within the 60-day period. It is difficult to assess the timeliness of evaluations, since most of the inputs are not supported by contract file documentation. The NMO is having difficulty meeting the requirements of the contractor performance evaluation process, including meeting the 60-day lead-time for completion of the evaluation and entering the data into the PPDB; providing substantive narrative to justify the ratings; and including hard copies of the NF 1680s in the contract files.

We reviewed the contract files (NAS 7 03001) for compliance with NFS 1842.72, NASA Contractor Financial Management Reporting. The new contract contains NFS clause 1852.242-73 requiring NASA Financial Management Reporting (reference Clause G-2 of contract). The clause requires that the detailed reporting categories to be used shall be set forth in the contract, as well as the number of copies and time and manner of submission. All of the contracts contained data requirements descriptions that covered the required information.

NFS 1842.7201(a)(1) requires contracting officers to monitor contractor cost reports on a regular basis to ensure cost data reported is accurate and timely, and to pursue adverse trends and discrepancies discovered in cost reports through discussions with financial and project team members. Until recently, there was little opportunity to identify adverse trends or discrepancies from the 533 data provided under the JPL contract. The content and volume of the data and the format used to present the information was such that it was not meaningful. The 533 was used primarily for costing data, but could not be used to identify potential cost over/underruns, because JPL did not identify variances between the Estimate to Complete and the Total Value of each Task Order, as required by 533 reporting clause. This lack of compliance to 533 reporting has been a consistent weakness for the JPL contract that dates back to the previous contract, NAS 7 1407.

Since last year, the NMO Procurement Officer has been working with JPL to develop more useful reporting data by task order to identify potential cost overruns and underruns. Without this information, the Contracting Officers and technical overseers for the tasks under the JPL contract did not have the information necessary to properly manage JPL tasks. In April of this year, a new report, known as the JPL Task Order Cost - Estimated Value at Completion (EVAC) Report was delivered to the NMO by JPL to address this shortfall of prior 533 reporting. The new report, which is still evolving, has been introduced to the technical and financial oversight communities who fund the tasks under the contract and it has provided the catalyst necessary to begin properly managing costs under JPL task orders.

Deliverables under the contract come in two forms: contract-level requirements that apply to the entire contract and task-specific deliverables that are called out in each task order. Contract-level reports include such things as financial reports like the 533, environmental reports like the NASA Pollution Prevention - Annual Progress Report, and Human Resource Reports like the Workforce Analysis (EEO) Report. In all, there are 133 such deliverables. Task-specific reports include final reports, interim reports, and other reports that are specifically requested by the task initiators for a given task.

The NMO, in coordination with JPL, has established a comprehensive tracking system for deliverables that are required at the contract-level. This process is documented in the newly issued NMO Standard Operating Procedure #10, Contract Data Requirements List. The NMO relies on JPL to track and maintain many of the contract deliverables at the contract-level, however the NMO provides oversight through periodic spot checks of the JPL tracking/maintenance records. The NMO has also recently devised a questionnaire that is sent to report recipients biannually to determine that the reports are responsive and timely. As recommended in the last Procurement Survey, JPL now transmits reports using electronic submissions and receipts for many of the contract deliverables.

The NMO does not track or monitor the task-specific deliverables on a regular basis. Instead, the Contracting Officers responsible for the tasks rely on their technical counterparts to perform this function. No follow-up is performed to ensure that such deliverables have been received until the tasks are closed out. In most cases, close out activities for tasks occur long after the tasks are complete, which compromises the ability to confirm whether the deliverables were satisfactory or timely.

A review of the subcontracting plan administration activities on contract NAS 7-03001 was performed by the survey team to ensure that they comply with the requirements of FAR 19.7. The contract contains the necessary FAR and NFS clauses one would expect in a contract of this magnitude. In accordance with the requirements of 52.219-9, the contractor is submitting Standard Form 294 and Standard Form 295 to document its progress in meeting the subcontracting goals established in the subcontracting plan. The contract also contains a special clause G.10 entitled Small Business Subcontracting Plan, which states that "small business procurement goals shall be established annually by mutual agreement between the contractor and NASA." The subcontracting plan goes on to state that, "final goals will be established within 45 days after finalization of the NASA appropriation by Congress." This process creates a situation in which subcontracting goals for the fiscal year are not established until the fiscal year is well under way. This is evidenced by the fact that the FY05 subcontracting goals were not incorporated into the contract until January 25, 2005.

In accordance with NFS 1816.405-274(g)(1), the contractor's performance against the subcontracting plan incorporated in the contract shall be evaluated during the award fee evaluation process. The NFS goes on to state that the evaluation weight given to the contractor's performance against subcontracting goals should be significant (up to 15 percent of available award fee). A review of the award fee documentation for the last award fee evaluation did not demonstrate that the contractors' performance in meeting its small business subcontracting goals was considered by the Performance Evaluation Board in recommending an award fee score to the Fee Determination Official. This issue was discussed with the Procurement Officer who conceded that the evaluation was not being performed.

A Contractor Pension/Insurance Review (CIPR) is an in-depth evaluation of a contractors insurance program; pension plans; and related policies, procedures, practices, and costs to determine whether they are in compliance with the FAR and pertinent contract clauses. DCMA guidelines state that a CIPR is normally conducted when a contractor has over 40 million dollars of qualifying sales to the Government during the contractors preceding fiscal year. The review of a contractors compensation structure, including pension plans, and its insurance plans are two of the contract administration functions found in FAR 42.302 normally delegated by NASA to the contract administration office (CAO) which is usually the Defense Contract Management Agency's Administrative Contracting Officer (ACO). CIPR's are usually self-initiated by the ACO; however, if NASA or any other Government agency believes that a review should be conducted, a recommendation to that effect should be provided to the ACO. If the ACO concurs, the review may be conducted as a special CIPR or as part of an already scheduled CIPR. On this contract, a CIPR report has not been received. However, the Contracting Officer intends to work with the appropriate parties to request and have a review conducted.

The NMO has one Contract Specialist who is primarily responsible for DCAA, IG, and GAO audit tracking. This task takes up approximately 50% of the Specialist's time. The Specialist is assigned other tasks as well, including contract close-out and JPL task order administration.

Follow up on JPL reportable audits is being effectively managed. The NMO tracks DCAA audits in CATS II, the online Contract Audit Tracking System (CATS). Each DCAA audit is entered into CATS with a target resolution and target disposition date, and is updated whenever there are changes. There is no center-level written directive for audit tracking; NMO merely follows NFS 1842.73.

In addition to CATS II, NMO Maintains two additional spreadsheets for audit tracking. One spreadsheet, used mainly for briefing purposes, tracks all ongoing DCAA audits. This spreadsheet was provided to the Survey Team ahead of time. A second spreadsheet tracks all DCAA, IG, and GAO audits and contains more information than the DCAA briefing spreadsheet. The Contract Specialist maintains this second spreadsheet because it is easier for him to use than CATS II.

As of 7/19/05, NMO had 9 open DCAA audits (although this changes frequently), 8 IG audits, and 6 GAO audits. All audits pertain to the JPL/FFRDC contract. There are currently no ongoing DCMA audits, but a DCMA audit of JPL's Earned Value management is planned.

The NMO Contract Specialist also noted that JPL (Cal Tech) itself has an audit liaison representative and an effective audit tracking system. NMO and Cal Tech meet at least once per month to ensure consistency in audit tracking.

The vast majority of the NMO's closeout workload is associated with task orders issued under the series of contracts with California Institute of Technology for operation of the JPL. The various contracts associated with JPL, their dates, and the number of task orders remaining to be closed out are as follows:

The previous survey (July 2002) identified a weakness that task orders were not being closed out in a timely fashion. Over the next two years, it appears that no NMO staff were assigned to closeouts and the situation grew worse before it improved. In October 2004, for example, JPL reported that the number of Task Orders w/ open issues under Contract NAS7-1260 was 420, but the number of those orders in the closure process was 0. In August 2004 NMO hired a Contract Specialist to focus on closeouts, and the situation has since improved accordingly. By the end of July 2005, NMO expects 150 of the task orders will be closed out. With a goal of 30 closeouts per month, NMO expects to complete closeouts of the 1260 contract in April 2006. NMO expects to begin closeouts of the 1407 task orders in January 2006, and hopes to establish a special group or team to accelerate the rate of closeouts.

NMO does not use the agency-wide Brace Management contract as do most other NASA Centers. However, NMO is considering using on-hand contract support staff to assist in the close-out process. NMO currently follows its Office Work Instruction ("OWI") entitled "JPL Task Order Closeout" which dates from 2002. This OWI is listed on the NMO G-drive (shared) for the NMO office. NMO has drafted a revision to the document but the revision has not yet been finalized.

The NASA Goal for ULO's is to have no more than 25% of ULO dollars be overage. The last meaningful reported metric was from April 2004, which showed NMO having only 15% of ULO dollars as being overage. NASA HQ is no longer tracking the metric, since the Contract Management Module will be able to perform this function. Since then, NMO has not tracked this information either. It is noted that there is no apparent written requirement to track this metric (in NFS, PIC, PN, or otherwise), other than "Office of Procurement Requirement" according to the "NASA Headquarters Code H Reporting Requirements" web page.

The Procurement Officer stated that NMO is able to compute the total dollar amount of all ULO's, but would not currently be able to track the age of ULO's. The Procurement Officer also stated that NMO is fairly aggressive about de-obligating funds from task orders even before they are due for close-out. Under the 1260 contract, NMO reports $510,138 in ULO's. Under the 1407 contract, NMO reports $12,994,136 in ULO's.

Contract #	Dates	Number of task order to be closed (app.)
NAS7-918/920	10/01/83- 9/19/93	0 (Basic contracts have yet to be closed)
NAS7-1260	9/20/93 - 9/30/98	291
NAS7-1407	10/01/98 - 9/30/03	1,184 (no task orders closed out)
NAS7-03001	10/01/03 - 09/30/08	None yet

OTHER ISSUES

NASA has had non-competitive international agreements for the operation of the Deep Space Network for 40 years. Currently, NMO oversees agreements with Spain and Australia. The current agreements began in 1986 and 1980, respectively, and have been extended several times since. The agreements are "contracts" mostly in a cosmetic sense. NASA HQ has since indicated a desire to implement contracts that are FAR-based. Accordingly, in 1999, NMO began the process of executing a new contract with Australia. The process of executing numerous waivers and deviations, plus a 3-year process of executing a separate Memorandum of Understanding between the two Governments, translates into a 6-year (and counting) pre-award process. As of this writing, most issues have been resolved. The Governments are seeking to award a new contract on or about August 14, 2005, which is the date when the Governments will observe the 40th anniversary of the U.S. - Australia Deep Space working relationship. The new contract carries an estimated value of $120,000,000. This was the value originally established in 1999 to last over a 10-year period. The contract is now scheduled to last from date of award (August 2005) until 2010, when the current international agreement expires. This is roughly half of the original contract period, but the estimated value of the contract has not been adjusted.

In Spain, a new 10-year international agreement was signed in 2003. The contract which has been in place since 1986 has been extended on a year-to-year basis from every year since 1994. A follow-on FAR-based contract is almost fully negotiated, and NMO hopes to award the contract before the end of fiscal 2005. The new contract carries an estimated value of $150,000,000.

For extensions of the current contracts, the contract files contain a "blanket" Memorandum For Record (MFR) pursuant to NFS 1806.302-4, which covers international agreements. No JOFOC is required in these circumstances (10 U.S.C. 2304 [f] [2] [E]), only an MFR signed by the Procurement Officer. In the situation of the Deep Space Network contracts, MFR's were signed in 2001 for Spain and 2000 for Australia to cover the following year plus "any subsequent extension which may become necessary in the event there are further delays." The MFR's do not contain award dollar amounts; it is not clear if this information is required in the MFR.

It does not appear that any announcements or synopses of the contract extensions have been made; however, the contracts are not likely to result in any subcontract awards (FAR 5.301 [a] [2]), so it is likely that such announcements are not required.

The Survey team member performed a cursory review of the administration of the currently existing contracts over the last few years. Other than incremental funding modifications and contract extensions, the NMO Contract Specialist advised there have not been any major administrative actions, such as performance issues, safety violations, work interruptions, payment issues or the like. The Modification files themselves were tabbed well and contained logs of Mods issued. Performance Evaluations are reviewed elsewhere in this Survey report.

For the new awards, both pre-award files contain Prenegotiation Position Memoranda (PPM). Both are signed by the Contracting Officer only; there is no known local requirement at NMO for a higher-level review of PPM's at any dollar level even though NFS 1815.406-171 requires contracting activities to establish such procedures. Neither PPM is dated, although it is likely that the Australia PPM was signed in 1999 and the Spain PPM in 2005. Both PPM's document very clearly all of the deviations and waivers of FAR and NFS clauses that have been obtained for each contract.

Each contract file currently contains a draft contract document. While the contracts contain a Section J List of Attachments (including the International Agreement with each country), the reviewer did not find the Attachments themselves with the contract or in the file.

The contract type is cost-reimbursement without fee. Payments under the contracts are made in U.S. dollars and are made in advance using a deviation of the Advance Payments clause.

Pricing under the international contracts is accomplished by annual management and budget meetings with the governments of Australia and Spain, which is a procedure established over the last 40 years. These meetings establish budget forecasts for upcoming fiscal years. This process appears to be independent of contract performance periods or the contract price. Price or cost analysis for the upcoming contracts (and for extensions of the existing contract) is apparently not accomplished when the awards are made. PPM's do not contain prenegotiation objectives as required by FAR 15.406-1. Contract files do not contain a Tab 72 (Price/Cost Analysis). It appears that pricing/budgeting practices under these contracts are based on historical methodology only. Although some such authority may exist, the reviewer did not find any actual authority exempting the contracts from the pricing requirements in FAR 15.4 (other than a waiver of cost or pricing data). The reviewer did not find any documentation in the files of how the estimated contract values were developed. NMO advised that contract values are based on cost history.

The NASA Management Office has performed semi-annual self-assessments under the authority of, and using the procedures outlined in, the current NASA Self-Assessment Guide. During the review, it was noted that the philosophy regarding self-assessments has changed during the last year. The focus of the self-assessment program has been expanded to focus on a broader range of functions currently being performed within the organization.

Review topics for the self-assessments are generated by the finding in the previous Procurement Management Survey, a review of the self-assessment guide, and other issues raised within the organization and by NASA Headquarters. This list is provided to the Procurement Officer for review and concurrence. Participants in the self-assessment include the self-assessment coordinator and other volunteers within the organization.

The self-assessment reports as a whole were thorough and well organized. The reports included sections on organization and management; the prime contract; and other issues, which identified strengths, weaknesses, and considerations. Once the self-assessment is published by the self-assessment coordinator, the Procurement Officer writes a memo which provides for the disposition of the action items identified in the self-assessment. This memo documents what action needs to be taken, who is responsible for the action, and the timeframe for completion. The self-assessment coordinator is responsible for ensuring that the corrective actions are implemented by the responsible individuals identified in the disposition memo.

The NMO administers two foreign contracts to provide support associated with the Deep Space Network. These contracts, one with Spain and one with Australia, are each more than 20 years old, and neither is FAR based. The NMO has been working to negotiate new FAR-based follow-on contracts for these services, but the process has taken longer than expected. It is currently estimated that these contracts will be awarded by September 30, 2005.

Despite the fact that these contracts have been extended numerous times, there have been no deviations recognizing that the period of performance for these contracts exceeded the 5-year limitation imposed by FAR 17.204 (e)(i) and NFS 1817.204(e)(i). The NMO specialist responsible for these contracts cited FAR 1.405 (c) to support the decision not to submit a deviation to the FAR to exceed the 5-year limitation. This provision states that, "any deviation from the FAR required to comply with an executive agreement is authorized unless the deviation would be inconsistent with FAR coverage based on law." Technically, since FAR 17.204 is based on law, specifically the Service Contract Act, the extensions to the contract beyond the 5-year term, should have been approved through the deviation process.

SAP went live at the NMO in March of 2003. The NMO has one super-user with a back up who has not had the in-depth training needed for this position. The NMO does have access to the On Line Quick Reference (OLQR), but no NMO specific guidance has been developed to date. Ten (10) task orders, two (2) cooperative agreements, one (1) purchase order and three (3) contracts were reviewed for consistency between the hard copy and the data in the system and the completeness of the data. There were systemic problems found in the recorded data in SAP. Under the NASA Data Tab;

Also, in one instance the hard copy of the cooperative agreement stated it was funded in the amount of $531,000. However, in SAP $631,000 was obligated.

The NASA Managment Office Training Coordinator is responsible for overseeing the Career Acquisition Training Program in support of NMO's acquisition workforce. An examination of the training files as well as interviews with procurement personnel, and the training coordinator were included in this review.

The training coordinator maintains a database which tracks grade level, warrant status, CON courses completed, and certification levels for all of the 1102s at the NMO. This database indicates that all of their employees have received the appropriate amount of training and appropriate certification levels for their grade and length of service at the NMO. A review was performed of the backup documentation to support the database. The training coordinator informed the team that the only backup documentation available was a scanned copy of the latest CON level certification earned by the employee in question. This practice means that the training coordinator does not have copies of all of the CON level certificates or the individual class certificates to substantiate that the database is correct.

The NMO utilizes a number of activities to ensure that their employees in the acquisition workforce meet the minimum 40 hours of continuing education training every two years. The training coordinator maintains a database which tracks the classes taken by individuals within the procurement organization to verify that this requirement has been satisfied. Some of the training opportunities include classes and seminars sponsored by a relevant professional organization, procurement related training sponsored by the NMO or other NASA centers utilizing outside training professionals, Defense Acquisition University on-line courses, and internal workshops utilized to address areas of concern within the procurement organization. NMO acquisition employees are required to complete Individual Development Plans (IDP's) which document near term and long term training requirements. These IDP's are discussed with their supervisors during the performance evaluations process. During employee interviews, the employees were very satisfied with the availability and applicability of training opportunities offered by management.

NFS 1852.270 stipulated that the cognizant contracting officer may appoint a qualified Government employee to act as their representative in managing technical aspects of a particular contract. If an appointment is made, NASA Form 1634, Contracting Officer Technical Representative (COTR) Delegation, shall be used to appoint COTRs. These COTR's are required to obtain comprehensive training and subsequent refresher training once every five years in accordance with the NASA Far Supplement "1842.270 Contracting officer technical representative (COTR) delegations." Subparagraph (e) also states that each COTR shall acknowledge receipt and accept the delegation by signing the original delegation letter. The original of the COTR delegation letter shall be filed in the applicable contract file. Copies of the signed COTR delegation letter shall be distributed to the COTR, the contractor, and each cognizant contract administration office. Acknowledgment and distribution for terminations of COTR delegations and COTR delegations which revise authority, duties and responsibilities shall follow the same rules. A review of the contract file documentation indicates that COTR delegations are being made on the required NASA Form 1634 and that these individuals have received the necessary training, except under contract NAS7-03001. The COTR delegation was granted under subparagraph (g), which allows for delegations to be made if an urgent need arises for the appointment of a COTR and no trained and otherwise qualified individual is available, then the procurement officer may make a temporary COTR appointment not to exceed six months. Temporary appointments must be so identified and clearly reflect the appointment expiration date. The delegation in the file clearly stated this was a temporary delegation not to exceed six (6) month and no technical direction could be given. However, the file did not contain a signed copy of the delegation.

It should be noted that the NMO maintains a database which list the COTR's for each contract.

SMALL AND DISADVANTAGED BUSINESS (SDB) UTILIZATION

The NASA Management Office carries out on a part-time basis, the role and functional duties of a Small Business Specialist. The most significant activity of the Small Business Specialist is leading, in collaboration with the OSDBU Program Manager for Science, the annual negotiation of socio-economic business goals in the prime contract with JPL representatives.

The Small Business Specialist at NMO is the primary point of contact for all small business matters related to the oversight of small business requirements pertaining to the Caltech prime contract. The Small Business Specialist remains actively engaged at the Agency-level and participates in Monthly Center telecoms led by the Assistant Administrator for Small and Disadvantaged Business Utilization; maintains regular dialogue with the OSDBU program manager for Science; attends national small business specialists meetings twice annually; serves as on-site NASA representative to JPL's Business Opportunities Office; and participates in other federal, state, and local small business activities as assigned. It was observed that at the time of this survey, the Small Business Specialist is retiring, thus a new one is expected to replace the outgoing Specialist. It is further noted that the Assistant Administrator for Small and Disadvantaged Business Utilization be notified in advance of such appointment of a replacement in accordance with applicable procedures.

NMO Management confers with Assistant Administrator for OSDBU on appointment or hiring of new Small Business Specialist.

Since the last reporting period the Small Business Specialist no longer provides Subcontract Consent of JPL subcontracts. This procedure formerly served as an effective mechanism to ensure that purchases up to a pre-determined threshold were reviewed in advance. It appears that this function is carried out by the Manager, Business Opportunities Office at JPL, and no longer by the Small Business Specialist.

The OSDBU Program Manager for Science maintains a very good relationship with NMO senior Management with "tag-ups" occurring at least twice a year at NMO and Headquarters. The OSDBU Program Manager for Science and the NMO Small Business Specialist participate in the Performance Award Evaluation exercises every six months.

During the review of the NMO PAEB briefing charts to JPL, it was apparent that little to no mentioning of comments provided by OSDBU and the NMO Small Business Specialist was officially conveyed to JPL leadership. While the JPL performance of its goals since the last reporting period have been stellar, the small business program is of a very significant and high profile nature. The OSDBU was unaware of this oversight and strongly recommends that at least one briefing chart be dedicated solely to the performance of socio-economic business goals and overall program execution.

Overall, the functions required to carry out the duties and oversight of the JPL small business program meet OSDBU program requirements and expectations.

Return to Contents

MONICA MANNING (HQ)	NMO Survey Team Manager (Organizational Management, Interviews- Procurement Professionals, Customers and Chief Counsel)
JOHN TRAHAN (JSC- HQ Detail)	Co-Lead, Subcontracting Plan Administration, Self-Assessments, 1102's Career Development Training
MELINDA DODSON (MSFC)	Pre-Award Activities on Prime Contract, IFMP/SAP Implementation, COTR Delegations and Training
DIANE FRAZIER (HQ)	Contractor Performance Evaluation (NF-1680), Financial (HQ) Management Reporting (NF 533), Tracking Data Requirements List (DRL) Items, Deviations and Waivers
ROBERT LABRIER (JSC)	Award Fee Administration, Subcontract Consents, Contractor Pension/Insurance Reviews
RICHARD MANN (SSC)	Task Order Administration, Audit Follow-up, Closeouts and Unliquidated Obligations, International Agreements

DONNA SPRINKLE	Headquarters Procurement Data Support
REBEKAH BREWER SUSIE MARUCCI	Headquarters Administrative Support
REBECCA WILKINSON	NASA Management Office at Jet Propulsion Laboratory Point of Contact

Procurement
Management
Survey Report

NASA MANAGEMENT OFFICE

at the

JET PROPULSION LABORATORY

OFFICE OF PROCUREMENT
HEADQUARTERS
WASHINGTON, D.C.

OVERVIEW

ORGANIZATION - MANAGEMENT

PRE - AWARD

POST - AWARD/CONTRACT ADMINISTRATION

OTHER ISSUES

SMALL AND DISADVANTAGED BUSINESS (SDB) UTILIZATION

Procurement Management Survey Report