The NASA Headquarters Office of Procurement conducted this procurement management survey at the Langley Research Center (LaRC) under the authority of NASA Handbook 1101.3, The NASA Organization. The survey was conducted from January 27 through February 6, 2004. The report contains the survey strengths, weaknesses, and considerations.

An exit briefing was held at LaRC on February 9, 2004 to discuss the survey findings.

This report serves as a basis, in part, for fulfilling internal control requirements in accordance with the Federal Manager's Financial Integrity Act of 1982 (P.L. 97-255).

Don Abrams
Survey Program Manager
Lead, Analysis and Assessment Team
Office of Procurement , NASA Headquarters

OVERVIEW

The Procurement Organization at the Langley Research Center is providing meaningful support to their program customers. The Procurement Management Survey Team interviewed randomly selected technical and program representatives to ascertain any issues or concerns with the current processes. A significant degree of customer satisfaction is apparent from the interviews with the technical community. Additionally, interviews with numerous acquisition professionals at all levels of the LaRC organization regarding the effectiveness of the procurement office were also conducted. The consensus from the procurement staff was also extremely positive.
The interviews of technical and acquisition personnel are given roughly equal survey emphasis with a review of contracting actions focused on compliance with procurement statutes, regulations, and procedures. The thrust of the compliance portion is directed towards systemic procurement processes, as opposed to focusing on individual file anomalies. Attention was also directed to current procurement innovations, both Agency-wide and Center specific.

The results of the compliance reviews and the interviews are detailed as strengths, weaknesses, and areas of consideration. Also, to promote the exchange of successful lessons learned and innovative procurement methodologies between Centers, the team sought to identify LaRC processes or initiatives that might benefit other Centers and, likewise, looked to other Centers for suggested approaches that might be exported to LaRC.
The exit conference at the conclusion of the survey consisted of a direct exchange of observations and ideas between the participants. To emphasize Center ownership of the resolution of any identified weaknesses or considerations, the survey follow-up process will focus on the corrective actions or initiatives undertaken by the Center. At an appropriate interval (approximately six months after this report is issued) the LaRC Procurement Officer will brief the Associate Administrator for Procurement and the survey team leader on Center achievements in these areas.

The survey team was led by Don Abrams, Lead of the Analysis and Assessment Team in the Analysis Division (HC) of NASA Headquarters. Below is a list of team members and the areas reviewed by each:

The survey could not have been accomplished successfully without the support of the following individuals:

ORGANIZATION - MANAGEMENT

1. Organization -- Management

The Langley Research Center Office of Procurement has made significant progress in becoming a first rate acquisition operation. They have proactively addressed recurring needs and future requirements, positioning themselves to respond to changing opportunities and challenges. Theirs is an organization designed to focus on meaningful customer service while nurturing an internal staff structure that encourages professional growth.

To achieve this, the Procurement Officer has combined activities to streamline organizational management, hired experienced personnel who have quickly contributed to an already strong procurement staff, and increased internal communications amongst procurement personnel at all levels. Additionally, the LaRC Office of Procurement has strongly supported the President's Management Agenda by offering up a substantial Competitive Sourcing activity, while also dedicating three full time procurement employees to facilitate the deployment of the Integrated Financial Management Program (IFMP).

The Survey Team program manager interviewed a broad cross section of employees throughout the various grade levels and branches in the procurement organization to gain insight into their perspectives on the effectiveness of the procurement organization. While interview candidates were selected randomly, an effort was made to assure that both long time employees and newer hires were in the sample, so as to get a more representative picture of the organizational health.

Overall, the preponderance of feedback is that the procurement staff feels things are working well for them. Employees speak highly of the procurement management and appreciate the vision of the organization. The Procurement Officer, her staff of
senior managers, and the Branch Heads were repeatedly mentioned as inspiring respect through their knowledge and dedication. The working climate is viewed as conducive to achievement and the staff feels that their individual assignments are meaningful. People are satisfied, feel they are treated fairly, and have the necessary tools, training and opportunities to succeed in progressively challenging assignments.

The restructured branch configuration with team leads appears to be working well. There were no comments regarding additional duties without compensation, or any frustrations with the selection process for choosing team leads. There seems to be some issue that although there is grade equality, the equality does not carry over to workload distribution.

Since the last survey, in addition to selecting a new Deputy Director and two new Branch Heads, the Procurement Office has hired 15 new contract specialists, 12 of whom are experienced procurement professionals. To a person, all the new hires interviewed are extremely happy at LaRC, while expressing that they were not dissatisfied at their previous jobs. Although everyone has been most welcoming of the experienced new hires coming from other Government agencies, a perception issue has arisen. Some of the longtime LaRC employees feel that although they have been given experiential criteria that should be met before they can contemplate advancing to the next grade level, the same criteria is not always applied to the recently hired experienced careerists. The perception is that since several of the newly hired personnel, who transfer in at the same grade they were at their previous job, are promoted in under a year, a question persists whether these new hires are being held to the same experiential criteria as the long time LaRC employees.

The interviews with the procurement staff did not disclose any real concern regarding the effect the recently announced buy-out provisions would have on the LaRC Office of Procurement. Moreover, there was no apprehension about the effect on procurement of budget uncertainties resulting from an Agency re-prioritization of space exploration. The workforce feels they have meaningful assignments and that their management is proactively positioning the organization to be capable and flexible in addressing future responsibilities.

The survey team interviewed a number of employees from various project and program offices regarding the effectiveness of acquisition support and the level of customer satisfaction.

The technical community appears very satisfied with the level of acquisition support being provided by the procurement personnel, with comments being uniformly positive. They find the procurement staff to be responsive, especially when critical problems arise. The technical customers appreciate that all their contracts are currently aligned with one Specialist and/or Contracting Officer. The customer interviews disclosed a concern that some specialists/COs may have too many projects, and also an apprehension about further workload distribution brought on by the buy-out and the possibility that procurement might be losing billets.

An almost universal complaint volunteered by the technical interviewees centered on the IFMP process. Clearly, the customers don't label their significant issues with the SAP product as being caused by, or associated with, Procurement. Nonetheless, the introduction of the Core Finance Module to LaRC is viewed as a significant step backwards for the center, from the previous electronic tracking system that had performed so ably for LaRC. Moreover, many technical customers continue to feel they have a major concern with visibility into the SAP process. While the technical representatives appreciate that Procurement is working under additional burdens brought on by the current IFMP rollout, the customers still feel a need exists for more communication with Procurement due to the uncertainties brought on by the SAP process.

The LaRC Acquisition Training Coordinator (ATC) continues to provide outstanding support to the Career Acquisition Training Program in support of LaRC's Acquisition Workforce. An examination of the training files as well as interviews with procurement personnel, the ATC, and the Deputy Procurement Officer were included in this review.

LaRC has recently been able to recruit and hire entry level and experienced contracting personnel. LaRC has a total of 70 contracting professional of which 62 are in the 1102 series. Additionally, approximately 95 percent of their employees received the appropriate amount of training and the vast majority of employees have achieved the appropriate certification levels for their grade and length of service at the Center.

The official training records were well documented and files reviewed contained extensive information including copies of previous course certificates, notes to the Headquarters Training Coordinator requesting review of specific training requirements, etc. Those who lack the necessary courses are identified and scheduled to attend the appropriate courses. Individual Development Plans have been prepared, updated and reviewed semi-annually (during mid-term and annual performance) for all employees in the acquisition workforce.

LaRC has implemented a number of initiatives to assure that their employees in the acquisition workforce meet the minimum 40 hours of continuing education training every two years. The ATC has established a tracking database that contains real-time training profiles, to include education and certification requirements, on each acquisition professional. In addition LaRC offers on-site training to assist their employees in meeting their training needs.

PRE - AWARD

The Survey team reviewed acquisitions in process where Acquisition Strategy Meetings (ASM) were held. Each of the solicitations addressed "risk management", are in draft form, and will or have posted to NAIS. The identification of sources and communication with the "market" was thorough. Documentation in the files indicated good team communication, oversight and attention to detail. These acquisitions are either totally or partly IDIQ. One acquisition had a maximum value of $39M and a minimum value of $100K. Multiple awards are planned. Another IDIQ is planned as a single award and has a maximum value of $20M and a planned minimum value of $1K. The $1K value is apparently to satisfy a customer driven desire to be "as low as possible". A prior survey comment about supporting maximum value has been resolved as reflected in the basis of maximum value for these acquisitions. The contract specialists assigned to these acquisitions were well informed about the acquisitions. It was evident they are members of a "team".

2. Justification for Other Than Full and Open Competition (JOFOC):

LaRC has few JOFOC actions. However, two were identified and reviewed. They were associated with technical services to support the Columbia Accident Investigation Board (CAIB) and were approved by the LaRC Competition Advocate. The Justifications are properly supported and have the required content.

NFS 1804.7003(b) requires that solicitations, draft or final, not be released until a purchase request (PR) has been received that contains a certification from the requiring organization that indicates compliance with or non-applicability of NPG 7120.5, "NASA Program and Management Processes and Requirements." COs must ensure prior to releasing a solicitation that the NPG 7120.5 certification has been properly completed and placed in the file. Since SAP implementation, this certification must be separately requested and obtained from PR initiators. This is burdensome, particularly on simplified acquisitions, considering the volume, customer base, and other SAP impact on customers. Previous to SAP, the certification was linked to the LaRC PR system. This problem is being worked and is not unique to LaRC. It will be resolved by finding a way to link the recently issued NF 1707, which includes the 7120.5 certification, to SAP.

Several fixed price contract files were reviewed that utilized "Milestone Payments". These payments are often tied to monthly reports, reviews, etc. Since there is no language in the contracts that makes these payments conditional, they are in fact delivery payments but it is implied that they are conditional financing payments.

The survey team reviewed several files involving the furnishing or acquisition of government property. The proper property clauses were included and had been updated by contract modification to reflect the latest 1018 reporting requirements. This is double checked by a Supply and Equipment Management Office (SEMO) review. For all off site efforts, NASA LaRC property administration is either delegated to Defense Contract Management Agency (DCMA), or the Office of Navy Research (ONR). DCMA and ONR are responsible through their delegations to ensure accuracy and verification of the NF 1018. These organizations perform periodic property system analyses and provide system status to LaRC. The Industrial Property Administrator advises that some delegations have recently been late but these are identified and followed up with reminders. The Industrial Property Administrator did not indicate any significant problems. There are approximately 140 contracts with 1018 reporting, of which 40 percent report "zero" property.

Under the procurement survey review two clauses came to the attention of the survey team that may need to be revised and or deleted.

This clause is not consistent with NFS clause 1852.235-73, "Final Scientific and Technical Reports (Feb 2003) as may be used with Alternate I or with Alternate II. (The NFS clause is primarily for the purpose of export control and is prescribed at NFS 1835.070.) Depending on whether the basic NFS clause or an alternate is selected, there are different review requirements prior to the release of "data produced". However, the advance review required by the LaRC clause is a long established LaRC practice and is for reasons that are perhaps additional to the stated purposes of the NFS clause.

Two solicitations for work to be performed onsite contained the Observance of Legal Holidays clause (1852.242-72). One solicitation contained the basic clause. The second had the clause with both Alternates I and II. PIC 00-07 "Use of Observance of Legal Holidays Clause" provides the following guidance:

"It is very important that contracts with on-site performance requirements either include or not include Alternate I, depending on whether or not the on-site contractor employees can work without the presence of the Government work force, as on an established Government holiday or other "special" Government holiday. Alternate II should be included in most contracts, to inform the contractor about dismissing its employees in the event of emergencies such as extreme weather conditions and power outages. For those contracts where work can be performed without a Government presence, Alternate I should not be in the contract, and the contractor's holiday schedule should be determined by its own holiday policies. Alternate I, which directs the contractor not to work on Government holidays, should be used only for on-site contracts where work cannot proceed effectively without the presence of the Government work force."

POST - AWARD

The survey team performed an analysis of several contract files meeting the criteria of FAR 42.15 to determine compliance. The files reviewed covered 21 evaluation periods. LaRC issues a report each month to Branch Heads that detail which NF 1680s are due in 30 days, due immediately, and which are overdue. Branch Heads follow up with the Contracting Officers to ensure the NF 1680 are completed. Most of the NF 1680s evaluated were entered into the Past Performance Data Base (PPDB) in accordance with PIC 01-02.

The team noticed that the COs signed and dated most of the NF 1680s according to NFS 1842.1503(b). The Contracting Officers did not sign two of the 19 NF 1680s evaluated; however, all of the NF 1680s that were evaluated included a response from the contractor. Also, NFS 1842.1502 requires contracting officers to conduct interim evaluations of performance on contracts within 60 days of every anniversary of the award of a contract having a term exceeding one year. Twelve of the nineteen NF 1680s were completed within this 60-day time frame. To improve the timeliness of NF 1680s, Langley Procurement Managers have started the NF 1680 process 60 days before the anniversary date of the award instead of at the anniversary of award

Additionally, as a management tool, Langley Procurement Managers include past performance evaluations (NF 1680) among their Office of Procurement metrics. The number of NF1680s received versus number required is measured each quarter. For the first quarter of FY 2004, Langley Procurement received a "Green=Good" rating for past performance evaluations.

The vast majority of the NF 1680s evaluated included a narrative that was detailed enough to support the ratings received in accordance with NFS 1842.1501. However, some evaluations included rationales that were not detailed enough to be useful for evaluating past performance.

The survey team noticed significant improvement in the processing of NF 1680s over the past three years. Most NF 1680s that did not meet the criteria above were completed in 2000 and 2001. A review of the ten NF 1680s that were completed in 2003 indicated that seven NF 1680s met all of the criteria listed above. The survey team concluded that Langley procurement managers emphasize the importance of completing past performance evaluations.

The Survey Team reviewed several contract files to ensure that award fee evaluations were (1) compliant with applicable clauses and regulations, (2) followed the applicable award fee plans and (3) complied with the provisional award fee limitations of the NFS.

Award fee payments for all the periods reviewed were made well within the 60-day time frame required by the NFS with the exception of one invoice because the contractor's submission was not timely. Several of the files reviewed contained no evidence of paid invoices in the file; however the file did have emails notes indicating that the invoice had been paid. In some instances copies of the AMS and SAP printouts were included in the files as documentation of paid invoices and summary of the provisional award fee payments made.

All files reviewed included the Fee Determination Official (FDO) determination, Contractor self-evaluations, and appropriate technical, business and Performance Evaluation Board (PEB) reports. The contract files also include an analysis of the 533M for the evaluation period, and debriefing charts to the contractor, including the adjective rating score, summary of the provisional payments and available fee for the period.

All dates that document the fee evaluation process were included in the files, however in some instances there was no evidence to indicate PEB members received the PEB report prior to the meeting of the PEB. Discussion with the CO disclosed that PEB reports are hand-carried to the PEB members 3 days prior to the PEB meeting.

In the majority of the evaluation files reviewed, the PEB report supported the final fee determination given to the contractor. However in one instance, it was revealed that the contractor, during this evaluation period, violated NASA policies regarding security and safeguarding of classified data which resulted in the issuance of a stop work order on two tasks orders. All documentation to the file including memos from NASA security concluded that this infraction was severe. The PEB minutes to the file indicated that the contractor should be sent "a clear message" that continued disregard for NASA policies and procedures would not be tolerated. The debriefing charts to the contractor stated the following "Stop work order issued as a result of Security incidences. Contractor took immediate corrective action". The final award fee adjective rating as well as the adjective rating in the management and safety areas were excellent. After discussions with the CO, it was mentioned that the management and safety score had come down from the prior period. The score for management and safety from the prior evaluation period was 94 and the current period decreased to 93.6. The contractor maintained an overall score of 95.9. The Contracting Officer also indicated that NASA shared in the blame for the security infraction and that in this facility NASA personnel and contractor personnel work very closely together. There was no discussion in the contract files nor in the minutes to the PEB meeting of NASA sharing fault with the contractor.

CONSIDERATIONS:

Several contract files were reviewed. All included the appropriate signed COTR delegations and updated and signed alternate COTR delegation form. Additional documentation in the file included the letters to the contractor indicating the COTR delegations and the changes to alternate COTRs. Internal memos were also included appointing the alternate COTR and resending delegation to the outgoing alternate COTR.

All delegations were to people not positions and included an extensive attachment with regard to additional details of the COTR delegations. There was no evidence that any CO duties were being relinquished to the COTR.

Several contract files were reviewed for compliance with NFS 1842.72. All the contracts reviewed required the submission of the 533M and 533Q. In one instance, there was no evidence to support that the 533Q was being submitted by the contractor. After discussion with the CO, it was determined that the file contained no evidence of a waiver to the 533Q.

Most of the files reviewed did include the 533M and were initialed and dated by the CO when received. Several files reviewed contained letters from the contractor indicating delivery of the 533M and variance analysis if applicable. There were several months were there was no letter in the file from the contractor, however the memo from the CO was included. In some instances, the file contained a memo to the file from the CO indicating that an analysis of the 533 was made part of the file and resided with the CO. After discussions with the CO, it was discovered that the analysis was the electronic version of the 533 that was submitted by the Contractor. There was no evidence that a written analysis has been performed by the CO on the monthly 533. However after discussion with the CO, it was discovered that discussion do take place between the COTR, CO and FMD but only when issues arise. After additional discussions with the COs it was concluded that more detailed analysis of the 533 are completed during the award fee evaluations. The level of reporting is sufficient for reporting with the COTR.

Discussions were held with several Contracting Officers and with the contractor responsible for these metrics, Brace Management. The reviewed resulted in the following:

Nine contracts were included on this list and each one was reported to be closed out. The contractor, Brace Management, verified that each item was not reported in SAP and therefore had completed their close out prior to conversion to SAP. The current AMS also showed no activity for these contracts.

Only two contracts appeared on this report. One of the contracts listed was reviewed with the Contracting Officer. This is an IDIQ contract and the contract ceiling is not being reported on AMS but rather the value of some existing task orders. After review of the current AMS by Headquarters, there continues to be discrepancy where cumulative obligations continues to exceed the contract value. After discussions with the CO, the CO admitted to his lack of knowledge in the AMS system but provided internal spreadsheets which track the issued task order with their cumulative values. The other contract has been closed out.

Seven contracts were listed on this report with all being reported closed out with the exception of one. The contracting officer was contacted about the one open contract and it was reported that in July 2002 funds were late reaching the center and therefore the contract could not be incrementally funded. The CO indicated that once funds were received the contract was funded immediately and the issue was resolved. The contract is currently being transferred to closeout.

Based on discussions with the contractor, Brace Management, they meet often with the Procurement Officer to ensure that monitoring of ULOs are resolved in a timely manner.

The files reviewed did not delegate SR&QA to DCMA. The basic contract did contain the NFS clause and the safety plan was included in the contract. Based on discussions with the Contracting Officer, coordination with the Office of Safety and Mission Assurance Monitor and the COTR occur only when infractions occur.

The survey team completed a review to determine if options were exercised properly. The review indicated that the exercise of options was carried out properly. A memo to the file (MTF) was found that documented the advantage to the Government along with a technical justification submitted by the COTR. The basic contract included the priced option and the modification was exercised unilaterally. The revised SF98 was included as part of the modification.

PRICING - FINANCIAL - AUDITS

The review indicated that cost price/analysis performed by the Pricing, Closeout, and Analysis branch for the contract specialists were quite comprehensive, properly addressed proposed cost elements, and overall were well documented. However, cost/price analyses performed by the contract specialist were not as comprehensive. There were instances where cost analysis reports prepared by contract specialist appeared to accept technical evaluation recommendations for other direct cost as the basis for the recommended cost. Also, there was an instance where no analysis of other direct cost was performed.

Overall there has been an improvement in the quality of technical evaluations since the previous survey. COTRs are providing detailed information as to the reasons for acceptance of the labor skill mix and hours. However, COTRs should be reminded to provide more rigor in the analysis of Other Direct Cost. For example historical data or price lists could be used to determine the reasonableness of material costs.

In the review of Pre-Negotiation Memorandums the Contract Specialist are generally following the guidelines at NFS 1815.406-170. There were some instances where Pre-Negotiation Memorandums failed to address the status of the contractor's systems. Cost analyses summaries were broken down by major cost elements, which is an improvement from the previous survey.

The files reviewed revealed very detailed explanations of the negotiation settlements. Major elements of cost were addressed as well as an explanation to the development of the settlement cost. Generally, the LaRC PNM checklist were used and provided an excellent guide for the contract specialist to include all of the necessary information. In files that meet the dollar threshold a certificate of current cost and pricing data was included in the file. However, there was no statement included in the PNM as to reliance on cost and pricing data.

WEAKNESS:

Pre-Negotiation Memorandums that were reviewed where the contractor bid profit/fee, all included the required NF-634. All selected weights were within the appropriate ranges. In several instances no justification was given in the file as to the reasons for choosing the weights.

The survey team reviewed several major cost-reimbursable contract files to determine whether the contracting officers were familiar with the requirements of PIC 00-23 "Contractor Insurance/Pension Reviews", which provides guidance on Contractor Insurance and Pension Reviews (CIPRs). The dollar values of the contracts reviewed ranged from $122,086,899 to $72,825,000.

None of the contract files reviewed demonstrated evidence that LaRC contracting officers were familiar with the requirements of PIC 00-23. This was discussed with LaRC management who verified that familiarity with CIPR requirements was not being emphasized.

The survey team reviewed several major cost-reimbursable contract files to determine whether the contracting officers were familiar with the requirements of Earned Value Management and the pertinent clause was included in the contract. The dollar values of the contracts reviewed ranged from $122,086,899 to $72,825,000.

All four of the contract files reviewed included evidence that the contracting officer was familiar with EVM requirements and the required clause was included in the contract. In addition, LaRC has an appointed "EVM focal point" that tracks EVM contracts.

The survey team conducted a review of audit follow-up reviewing of several major contracts with dollar values ranging from $122,086,899 to $72,825,000. The review was limited to assessing that cost-questioned/cost sustained information as achieved during contract negotiation was being provided to DCAA. Generally this consists of the Contracting Officer supplying DCAA with a copy of the negotiations memorandum or a final contract copy to facilitate DCAA determinations on questions cost sustained during the negotiations. In each of the contract files reviewed, information documenting that the negotiations memorandum was provided to DCAA was not present in the file. The lack of such documentation was discussed with Center personnel. Current policy and practices are to routinely supply DCAA with the required information and there was universal certainty that this was being performed. But such efforts may be by email or regular mail and are not being documented as completed.

GRANTS, COOPERATIVE AGREEMENTS, SIMPLIFIED ACQUISITIONS, AND OTHER ISSUES

The Langley Office of Procurement - Grants and Research & Development (R&D) Studies Branch awards approximately 150 grants and cooperative agreements per year. These awards range in dollar value from $3,000 to $69 million, and include awards to universities, non-profits, and commercial firms. The Survey Team reviewed several files to determine compliance with the provisions of the NASA Grant and Cooperative Agreement Handbook.

The Langley Office of Procurement tracks grant lead-time among its customer satisfaction procurement metrics. The established goals are: 60 days from purchase request receipt to award; and 29 days from complete package (technical evaluation, Unsolicited Proposals Evaluation Summary, etc.) to award. All files reviewed were awarded in a shorter time frame than the established goals. The Grants and R&D Studies Branch has implemented a number of changes to assist in meeting their established goals; e.g., developing standard checklists to assist Grant Specialists, and transmitting certain documents electronically.

The files reviewed showed compliance with Langley Policy Directive 2030.1, "Requirements for Legal Review of Acquisition Matters". All files that required legal review included a legal review and resolution of any comments.

On July 27, 2001, the NASA Chief of Staff issued a standard NASA policy for informing Members of Congress about the selection of NASA research proposals. All of the awards issued after the implementation of the aforementioned policy indicated evidence of Congressional notification. To streamline their grant and cooperative agreement award process, the Grant and R&D Studies Branch issues this notification electronically to the Office of Legislative Affairs.

Two of the grants/cooperative agreements were awarded as a result of a NASA Research Announcement (NRA). Both of the grant/cooperative agreement files included the NRA document in accordance with NFS 1835.016.71(d)(8)(i). These files also included a solid technical evaluation that contained a discussion of the strengths and weaknesses of the proposal, unique innovative methods, approaches, or concepts demonstrated, overall scientific/technical merits, and reasonableness of costs. These two technical evaluations included a very detailed analysis of costs.

Of special note when determining the appropriate Award Instrument in accordance with Provision §1260.12 Langley Technical Officers provide a recommendation on the type of award instrument in their technical evaluation. Additionally, the Grants and R&D Studies Branch has a process in place whereby the Grant Officers review each proposal and determine the appropriate award instrument based upon the guidance in Provision §1260.12 "Choice of Award Instrument". Grant Officers sign each proposal to indicate the award instrument before the proposal is given to the Grant Specialists for processing. Not all proposals that come to the Grant Officer are selected for grants or cooperative agreements; approximately ten percent are selected for contracts. The Survey Team reviewed each award and concluded that the type of award instrument selected for each effort was appropriate.

Also, the Grant and R&D Studies Branch created a form for their use entitled "Memorandum of Negotiation". This form documents the negotiation including the negotiation of salaries, travel, equipment, supplies, and materials. The use of this form is a good practice that can be shared with other grant offices. In many instances, this memorandum accepts the recommendation of the Technical Officer, which do not always provide a through evaluation of costs.

Several of the files reviewed were the result of unsolicited proposals/earmarks. In accordance with §1260.17(c), all unsolicited proposals recommended for acceptance as grants shall be supported by a Justification for Acceptance of an Unsolicited Proposal (JAUP) prepared by the cognizant technical office. The JAUP serves as the technical evaluation for the unsolicited proposal. To streamline the technical review process, the Grant and R&D Studies Branch combines the JAUP with NASA Form 884, "Unsolicited Proposal Evaluation Summary". Each technical evaluation conducted a review of the uniqueness and merit of the proposal and all were signed by the appropriate offices. Some of the awards reviewed did not thoroughly evaluate the proposer's cost proposal. For example, the technical evaluations determined the costs to be reasonable "as compared with similar awards", or simply included a statement that the costs were reasonable. Most of the technical evaluations accepted the proposer's cost proposal as submitted. Technical and Grant Officers are not required to prepare a Price Negotiation Memorandum, similar to contract awards; however, these technical evaluations could conduct a more thorough evaluation of costs. The Grant Officer interviewed during the survey mentioned that the Grant and R&D Studies Branch has placed a strong emphasis on the quality of technical evaluations in the past year and have been working with Technical Officers to ensure that technical evaluations include a thorough cost evaluation.

Annually the Langley Grants and Research and Development (R&D) Contracting Branch awards approximately 35 SBIR Phase I, 52 SBIR Phase II, five SBIR Phase III awards, three STTR Phase I and four STTR Phase II awards. The Survey Team reviewed several files that were taken from each of the aforementioned categories. The Master Files for the SBIR/STTR Phase I and Phase II awards were also reviewed. The files were reviewed to determine their compliance with FAR/NFS, the NASA SBIR/STTR Handbook, and the Small Business Administration's "Small Business Innovative Research Program Final Policy Directive. The Survey Team concluded that overall, the Langley SBIR/STTR awards were well managed.

All SBIR/STTR purchase orders/contracts were awarded on or before the date specified in the electronic SBIR/STTR Handbook. The SBIR/STTR Phase I and II Master Files contained the referenced documents. All SBIR/STTR Phase I and II files reviewed referenced the Master Files for further documentation and the file documentation was complete. SBIR Phase III files documented why the requirement was appropriately handled as a Phase III award. All files reviewed included a NASA Langley Form P115, "Determination of Responsibility" signed by the contracting officer. All files reviewed were in compliance with these requirements contract length and value. All SBIR/STTR awards reviewed complied with the limitation on subcontracting requirements. All SBIR/STTR awards reviewed contained a detailed technical evaluation and the appropriate Representations and Certifications. All SBIR/STTR awards meeting the criteria of NFS 1815.404-4 included a signed NF 634, "Structured Approach Profit/Fee Objective". All SBIR/STTR awards included the clause at FAR 52.227-20 "Rights in Data - SBIR Program. A 507 - "Individual Procurement Action Report" was completed for all awards reviewed. The Handbook requires a "Phase III Notification Form" to be submitted to the local SBIR Program Office at the time of initiation of a purchase request for a Phase III effort. Both SBIR Phase III files reviewed included this notification.

The Survey Team evaluated each memorandum of negotiation for the award of SBIRs Phases I , II, and III and STTRs Phases I and II. Nine files included a detailed negotiation memorandum documenting all aspects of the negotiations. Separate pricing reports for rates and factors were evident in all files. Two of the negotiation memorandums referenced the technical evaluation in the areas of Travel and Other Direct Costs; however, the technical evaluations themselves simply stated that the proposed costs were acceptable without any further analysis. The schedule for SBIR/STTR awards may have been a contributing factor to these technical evaluations. SBIR/STTR awards are required to be completed by January 16th of each year. January awards are impractical because of the amount of coordination that is required to award a purchase order/contract, and at a time when Federal employees are using significant amounts of vacation leave. The schedule, combined with the number of SBIR/STTR awards processed, may affect the amount of time which can be devoted to cost analysis.

The survey team completed an analysis of the simplified acquisition files. Overall organization of the files was good and generally contained the required documentation and a contents checklist (LaRC form P323). However, on sole source actions, backup or explanatory information for price reasonableness was sometimes lacking. In one case, there was no price reasonableness determination, as noted in LaRC's self assessment review of the particular file.

Langley uses the simplified acquisition process for NRA and SBIR awards within the simplified acquisition threshold. When this is done, appropriate terms and conditions are used in lieu of the FAR Part 12 terms and conditions.

The NPG 7120.5 certification is not presently being obtained for simplified acquisitions due to the transition to SAP and the difficulty attaching the certification to electronic P.R's. When the present level of customer difficulty with SAP is resolved, the NPG 7120.5 certifications will be resumed.

The survey team performed an analysis of LaRC Acquisitions utilizing Mid-Range procedures. The acquisitions reviewed utilized Best Value Selection (BVS) procedures using value characteristics. The value characteristics used were similar to the usual Mission Suitability sub-factors and did not appear to be characteristics above a specified baseline of acceptability. For example, some used the following language: "Understanding the Requirement and Management Approach". This, coupled with the use of cost type contracts and providing a Government assessment of the cost of the work in the solicitation would seem to make a selection based on a tradeoff between cost and any offered value characteristics difficult and vulnerable to protest.

There are 574 active cardholders in the LaRC Bank Card Program and 167 approving officials, which equates to a 3.4 ratio of Cardholders to Approving Officials. Potential Cardholders and Approving Officials are receiving training prior to receiving the Procurement Officer Authorization Letters. Since the issuance of PIC 03-20 the PO Delegation letter has been updated to include the single purchase limit and the monthly purchase limit. Furthermore, approving officials and cardholders are receiving refresher training based on PIC 03-20, issued October 31, 2003. A certificate is maintained in the cardholder's file when a refresher-training course has been completed.

LaRC continues to monitor purchases on a monthly basis. Additionally the LaRC Center Purchase Card Program Manager (CAPC) performs desk audits. A list of potentially improper purchases is kept and the LaRC CAPC interfaces with the cardholder to determine the nature of the purchase. If the purchase is found to be improper, the cardholder is required to payback LaRC the full amount of the purchase. LaRC policy for purchase cards LMS-CP-4540 has been updated in order to be in compliance with PIC 03-20.

The number of purchases is monitored and the LaRC CAPC purges the system quarterly of persons who do not use their bankcards. In the period of October through December 2003, 109 Cardholders were deactivated in the system and 18 Cardholders were activated in the system. A list of persons who have been purged from the system is maintained along with the reason for their termination. Furthermore, cards from individuals who retire, move to another job, etc. are taken from those individuals immediately. Also, when an individual moves to another organization on-center the CAPC along with that individuals new management, makes a determination as to the need for them to possess a bankcard in that new organization. If a card is not needed in that new organization then it is retrieved and the individual is removed from the system.

The LaRC CAPC has developed a checklist to ensure that all necessary processes are completed before issuance of a Bank Card (Training, Delegation Letter, etc.) Furthermore, the checklist has the after training items that need to be completed to ensure the cardholder has access to the system. There is a separate checklist for Authorization Officials to include the list of cardholders they approve. Additionally there is a checklist for canceling an approver or cardholder out of the system. Files are kept on individuals that are terminated in order to ensure a record of who has been terminated.

The agency wide bankcard program manager (AWBCM) located at LaRC in the Office of Procurement provides synergy for all of the Center CAPCs. In order to increase communication between the CAPC's, the AWBCM has created a monthly telecon tag-up with the CAPCs in order to discuss relevant issues. Participation has been at about 95%. At this tag-up current issues with SAP and policies are discussed. The AWBCM regularly emails all of the CAPCs information he receives from GSA on fraudulent purchases that have occurred at other Federal Government agencies. The AWBCM has stressed the importance of auditing throughout the program. LaRC has been especially diligent with its auditing since the AWBCM is present at the Center. The AWBCM meets quarterly with GSA who managers of the federal wide program and Bank of America, the bankcard provider, to discuss issues relevant to the program.

The P-Card system implemented by SAP has produced significantly more work for the CAPCs. The AWBCM has done an excellent job of providing leadership to the CAPCs in dealing with the SAP system. The AWBCM is proactive in working with accounts payable at LaRC to develop work-arounds in the SAP system. Through the tag up sessions the AWBCM provides the information obtained to the CAPCs.

The NASA Office of Inspector General (OIG) recently completed an audit of NASA's purchase card program. This audit found that the purchase card program was managed effectively and that management had established reasonable control over the programs.

Use of convenience checks for purchases is authorized only when in a sole source scenario and the vendor does not accept a bankcard. LaRC pays Bank of America 1.9% of the price of the purchases for maintenance of the convenience check account. The convenience check owners must maintain a convenience check log which should contain the convenience check number, date issued, payee, description of purchase, amount and account fee (1.9%) The checks are used for requirements under the $2500 limit. However, a Contracting Officer may use the checks to pay for requirements up to $25,000. The head of simplified acquisition is the only person in procurement who possesses convenience checks; however, from interviewing this person there is a possibility that 2 other procurement employees may receive convenience checks. Also, a technical customer in the aircraft function at LaRC has convenience checks to purchase items from Boeing for aircraft parts. The customer received bankcard training and a special delegation letter, which detailed his responsibility with respect to use of convenience checks. The customer has only used the convenience checks once because Boeing does not send invoices for individual purchases. The customer's possession of these checks and lack of use is a vulnerability in the procurement system.

LaRC Procurement Office has found it difficult to de-obligate funds in the SAP system. Because of the difficulty, the LaRC Procurement Office in conjunction with the IFM Project created a policy to process De-Obligations. The policy has not been approved; however, LaRC personnel were informed, via LaRC's "@larc "website that the policy must be followed in its current form. The policy was put into place in order to put the requirement on the end user (e.g. COTR, TO, Program Analyst) to determine Accounting Line Items (ALI) and Procurement Line Items (PLI) that de-obligations need to take place against in SAP. In order to perform this analysis, the end-user must coordinate with the Accounts Payable and Cost Management Financial Management Offices. This requirement helps the procurement office to perform de-obligations more efficiently in the SAP system. After the aforementioned coordination has taken place, a Memorandum is written by the end user to procurement to justify the need for a de-obligation. A paper modification is done outside the SAP system. After approval of the modification the procurement operations performs the de-obligation in the SAP system along with several other functions.

In order to perform de-obligations where cost has been accrued but payment has not been made, coordination in SAP is required between three Full-Time Equivalents (FTEs): one employee to reverse cost, one Contract Specialist to effect de-obligation, and one Contracting Officer to approve transaction. When cost has been accrued and payment has been dispersed it takes five FTEs to perform a de-obligation in the SAP system. The representatives come from the following offices: two FTEs from accounts payable, one FTE from cost management or logistics management, and two FTEs from procurement.

This process is much more time consuming compared to the process before the implementation of IFMP/SAP. Under the old process a memorandum was written by the end-user to justify a de-obligation. Next the procurement operations would use the memorandum to produce the modification to de-obligate the funds. Then procurement operations would send the modification to accounting/finance and it would de-obligate the funds in the LaRC legacy financial system.

Under the LaRC legacy system for processing simplified acquisition most tasks associated with processing a Purchase Order were completed electronically. However, under the new SAP system LaRC Procurement Operations is completing many simplified acquisition functions manually. Once the purchase order is completed and awarded outside of SAP, the data associated with the purchase order is entered into SAP in order to obligate funds against the purchase order. Furthermore, since the implementation of SAP, technical customers are using their secretaries are requisitioners. Since the technical customers' secretaries are not familiar with the technical issues associated with the purchase order, specification information and sole source justifications are missing from the purchase request package. The missing information slows down the process because the CO has to go back to the technical customer's office to obtain the needed specifications or sole source justification. Since the implementation of SAP, the LaRC Procurement Operations in conjunction with the IFM Project has developed coffee break sessions with the end-users secretaries in order to develop better information flow and to give the secretaries a better understanding of the information needed to complete a purchase order. These coffee break sessions have improved the efficiency of the Simplified Acquisition process.

Since the 2000 Procurement Management Survey LaRC Office of Procurement (OP) has reduced their documented procedures from forty-five to twenty-six procurement related activities excluding the deletion of the Regulation Implementation Guidelines (RIG). To date, LaRC OP has 10 Center Procedures and 16 Organizational Procedures. Currently under review is one center procedure regarding processing deobligations in the AMS/SAP IFMP system and six of the current operating procedures are being revised.

In addition LaRC OP has deleted and or modified forms that are in the Forms System. For example, in the previous survey it was noted that the 1098 Guidance Document (LMS-TD-4547) was similar enough in nature to that of the Virtual Procurement Office Concept. LaRC reviewed the document and determined the procedures were either (1) duplicative of those existing in Federal regulations, other Office of Procurement procedures, or other readily available reference material; and/or (2) contained obsolete information. The1098 Guidance Document was subsequently deleted and replaced with filing guidance attached to the NF 1098.

Also since the previous survey, all of the procurement forms (PForm) were converted to Informed Filler. Each PForm was reviewed, persons who used the forms contacted, and decisions made to either delete or convert the form to Informed Filler. LaRC currently has 41 Pforms; before the forms conversion there were 58.

As a means to further simplify the internal process, all of the LaRC unique clauses were reviewed by an attorney, a policy analyst and an independent reviewer to determine if they were current, in need of revision, or should be deleted. Since the 2000 survey 48 LaRC unique clauses have been deleted, 37 remain and 2 additional ones are pending.

There are no records of deviation request for this review period. The Commercial Acquisitions, SF1449, Solicitation/Contract/Order For Commercial Items was returned to compliance, by deleting the additional block 27(c), thereby correcting the weakness under Commercial Acquisitions noted in the 2000 survey, since there is no record of having obtained a deviation to amend this Standard Form.

As an additional commitment to standardization LaRC the policy office continues to emphasize the use of the VPO. Contracting officers and specialist that were interviewed identified it has a helpful tool.

SECTION VII

SMALL AND DISADVANTAGED BUSINESS (SDB) UTILIZATION

LaRC's last procurement survey, which included a review of the small business program, was conducted in October 2000. This survey will review the findings of that survey and evaluate the progress made since that time together with an update of new activities. The overriding purpose of this review is to ascertain how policies, programs and initiatives, established at NASA Headquarters, are being implemented at LARC. The following is a recapitulation/update of selected considerations and recommendations from the October 2000 survey, and findings from this review.

1. Small Business Technical Advisor:

2000 Survey Weakness: (1) The Procurement Officer should assure that form P-327 (Determination of Small Business Goals for Procurements Over $500K) is revised to add signature blocks for the technical representative and the center technical advisor. (2) The Procurement Officer should assure that form P-327 is completed by procurement personnel and inserted in the contract file under the appropriate tab.

Current Team Finding: The P-327 was revised to include the center small business technical advisor (SBTA), but did not include the technical representative. The current SBTA was on travel and was not interviewed on his duties.

2. Program Management:

2000 Survey Weakness: It is recommended that the Procurement Officer take steps to assure that LaRC Small Business Council is established and staffed.

Current Team Finding: This weakness has been superceded by the Langley Contractors Steering Council, which consists of small and large companies from LARC's on-site and near-site contractor organizations as well as NASA members. The mission of the council is to provide timely information and advice on Center-wide issues of mutual interest to its members. There is also an Executive Committee, which provides assistance and guidance to the co-chairs, one each from small and large companies, respectively.

STRENGTH:

3. Automation of the SF 294 Subcontract Reporting System

NASA has developed the subject system in order to replace the current labor- and paper-intensive system for contractor reporting of awards made to small businesses on a semi-annual basis by large prime contractors. The SF 294, called the Subcontracting Report for individual contracts, itemizes the cumulative dollars awarded to small, large, SDB, WOSB, HUBZone, HBCU/OMEI. Veteran-Owned, and Service-Disabled Businesses. The old paper system of reporting took 4-6 months before Headquarters got all the Center reports and consolidated them for the Agency submission to the Federal Procurement Data Center. As a result of the delays by late submittals on the part of prime contractors and summarization by the Centers, the data was not available early enough to be used for estimating future-year requirements. The NASA NAIS working group, whose leader is at LARC took on the task of developing this initiative.

STRENGTH:

4. File Review:

2000 Survey Weakness: The Procurement Officer should assure that all contracting officers are aware of the guidance provided under NPD 5000.2 (Uniform Methodology for Determination of Small Disadvantaged Business Subcontracting Goals) and comply fully with that guidance.

Current Team Finding: The procurement officer has taken steps to ensure that this issue was addressed and will continue to be addressed at Socio-Economic Training Stand-down sessions.

5. First Contact Counseling Checklist:

2000 Survey Weakness: The Procurement Officer should ensure that the NASA Checklist (OSDBU Form 134) is completed during counseling sessions by the small business specialist (SBS) and centrally filed for future functional reviews.

By letter dated August 7, 2000, the Office of Small & Disadvantaged Business Utilization (OSDBU) established a requirement for Center small business specialists to utilize OSDBU Form 134, First Contact Counseling Checklist. The goal of this new checklist was to further the OSDBU's implementation of ISO 9001 certification by assuring that each small business seeking information about how to do business with NASA receives, at a minimum, the same information about NASA's small business programs and initiatives. .

Current Team Finding: The team review revealed that OSDBU Form 134 was being utilized. Discussions with the center SBS revealed that he has converted to this ISO process, and is formally gathering and using this information to improve his counseling process.

STRENGTH:

6. Customer Service Assessment:

Interviews were conducted to assess the "quality" of customer service that small businesses had received though their interaction with LaRC's Small Business Office (SBO). Three small business representatives that currently have contracts were interviewed and appeared exceptionally satisfied with the advice and support furnished from the Industry Assistance Office.

STRENGTH:

7. Small Business Subcontracting Goals:

2000 Survey Weakness: Some members of the LaRC procurement staff apparently do not understand the distinction between small business solicitation goals based upon "total contract value" (as specified in an RFP for proposal/evaluation purposes) and subcontract plan goals based upon "percentage of planned subcontracts" (as specified in the subcontracting plan, SF294, Subcontracting Report for Individual Contracts). The Procurement Officer should assure that this distinction is fully understood by providing appropriate training and guidance.

Current Team Findings: Random interviews with contracting officers and specialists confirm that the distinctions between NASA's methods for evaluating goals is now well understood by the procurement staff.

8. Return to Flight (RTF) Program and Establishment of the NASA Engineering and Safety Center (NESC)

As a result of the Columbia Accident Investigation Report (CAIB), The NESC has been established at LARC. In LARC's first Quarter FY 2004 report on contract awards in the RTF program, $6.6 million in contracts were awarded with 96% of the dollars going to small businesses. However, a visit by NASA's Minority Business Resource Advisory Committee (MBRAC) in December 2003 determined that none of the small business awards included those owned by minorities or women. The MBRAC had also made a strong recommendation that LARC work with the HQ Small Business Office to develop a process by which SDBs and other small business categories could make known their capabilities to the NESC.

STRENGTH:

9. Utilization of the Center Small Business Office:

The team's review revealed that the center small business specialist (SBS) is now assigned to the Office of Procurement staff answering directly to the Procurement Officer with 100% of his time spent on small business. Previously, the SBS also served as a Branch Chief with only 60% of his time on small business. In addition, the head of the Industry Assistance/Industry Relations Office spends 50% of his time in assisting with some of the small business responsibilities.

STRENGTH:

Subcontract goal achievement is very important to NASA as an agency because most of the dollars that the agency spends on small businesses are in the subcontracting arena. A random statistical sample of six companies was drawn from 44 contracts that currently are required to develop subcontracting plans with goals. All contracts were at 60% to 80% of completion. Two of the contracts had zero funding. Of the remaining four, two had met the goal for small business and one was very close. Two of the four met the women-owned goal. None of the four contracts achieved the small disadvantaged goal.

Four contracting officers were interviewed by the team to see how the CO handles the preparation for award fee distribution. All the CO's perform in a professional manner and have an excellent process for ensuring that the interests of small businesses are represented. Preparation includes working with the SF 294, SF 533, company self-evaluations, and inputs from the small business specialists.

Since the prior review there have been numerous changes/challenges to the Agency's small business program - President's Management Agenda, the CAIB Report, President's Small Business Agenda, contract de-bundling, Freedom-to-Manage (F2M), A-76, and an increase in GSA/FSS buys and Credit Card Purchases. The coming years will probably bring many new initiatives to the small business program. To that end, it is imperative that emphasis be placed on "in-reach," which requires that NASA and contractor personnel continue to seek small businesses that can contribute to the NASA mission. The team observed that LaRC has taken aggressive initiatives to focus on this area, as follows:

DON ABRAMS (HQ - Code HC)	Survey Team Program Manager; procurement and technical staff interviews
KARL BEISEL (HQ - Code HC)	Pricing Topics
KENT COCKERHAM (GSFC)	Pre-Award, Simplified Acquisition
JERRY EDMOND (HQ - Code HC)	Bankcard Program, IFMP/SAP Issues, Self-Assessment, Pricing
MONICA MANNING (HQ - Code HC)	Metrics, Policy, Training Issues
SUZAN MOODY (NMO/HQ)	Grants & Cooperative Agreements, SBIR/STTR, Past Performance (NF-1680)
LINDA RIVERIA (JSC)	Post-Award
TONY DIAMOND (HQ - Code K)	Small and disadvantaged business utilization
SHIRLEY PEREZ (HQ - Code K)	Small and disadvantaged business utilization

DONNA SPRINKLE	Procurement Data Support
BRIDGET BOND	Headquarters Administrative Support
VIRGINIA C. WYCOFF	LaRC Point of Contact
MARY DUELL	LaRC Point of Contact
AUTUMN PIMPEL	LaRC Administrative Support

	FY2000	FY2001	FY2002
Small Bus.	134%	111%	109%
8(a)	108%	121%	108%
Small Disadvantaged	187%	124%	147%
Women Owned	234%	165%	116%

	FY2000	FY2001	FY2002
Small Bus.	139%	130%	119%
Small Disadvantaged	149%	135%	103%
Women Owned	138%	141%	106%

Procurement
Management
Survey Report

Langley Research Center

OFFICE OF PROCUREMENT
HEADQUARTERS
WASHINGTON, D.C.

OVERVIEW

ORGANIZATION - MANAGEMENT

PRE - AWARD

POST - AWARD

PRICING - FINANCIAL - AUDITS

GRANTS, COOPERATIVE AGREEMENTS, SIMPLIFIED ACQUISITIONS, AND OTHER ISSUES

SMALL AND DISADVANTAGED BUSINESS (SDB) UTILIZATION

Procurement Management Survey Report

Langley Research Center

OFFICE OF PROCUREMENT HEADQUARTERS WASHINGTON, D.C.

OVERVIEW

ORGANIZATION - MANAGEMENT

PRE - AWARD

POST - AWARD

PRICING - FINANCIAL - AUDITS

GRANTS, COOPERATIVE AGREEMENTS, SIMPLIFIED ACQUISITIONS, AND OTHER ISSUES

SMALL AND DISADVANTAGED BUSINESS (SDB) UTILIZATION

Procurement
Management
Survey Report

OFFICE OF PROCUREMENT
HEADQUARTERS
WASHINGTON, D.C.