The NASA Headquarters Office of Procurement conducted the procurement
management survey at the Goddard Space Flight Center (Goddard) under
the authority of NASA Procedures and Guidelines 1000.3, The NASA
Organization. The survey was conducted from July 9 - 20, 2007.
The report contains the survey strengths, weaknesses, and
considerations.
An exit briefing was held on July 20, 2007 to discuss the survey findings.
This report serves as a basis, in part, for fulfilling internal control requirements in accordance with the Federal Manager's Financial Integrity Act of 1982 (P.L. 97-255).
Yolande B. Harden/Ronald Backes
Survey Program Managers
Office of Procurement
Analysis Division
SECTION II ORGANIZATION — MANAGEMENT
SECTION III PRE - AWARD PROCESSES AND DOCUMENTATION
SECTION IV POST - AWARD PROCESSES AND DOCUMENTATION
SECTION V PRICING/FINANCIAL/AUDITS
SECTION VI GRANTS, COOPERATIVE AGREEMENTS, SIMPLIFIED ACQUISITIONS, AND OTHER ISSUES
SECTION VII SMALL BUSINESS PROGRAMS
The Goddard Procurement Operations Division (Goddard Procurement) is providing meaningful support to their technical and program customers.
The Procurement Management Survey consists of two primary components: 1) interviews with technical, procurement and legal personnel regarding the effectiveness of the procurement division and 2) compliance reviews that consist of a review of contracting actions focused on adherence to procurement statutes, regulations and procedures. The primary emphasis of the compliance portion of the survey is on systemic procurement processes rather than individual file anomalies. Current procurement innovations, both Agency-wide and Center specific are also reviewed.
The results of both the interviews and compliance reviews are compiled into narrative summaries with strengths, weaknesses and areas of consideration identified as appropriate. Strengths are generally defined as best practices utilized in support of the procurement system. Weaknesses are defined as problems, typically systemic, that require corrective actions. Considerations are defined as issues that: 1) if not corrected could turn into a problem or problems that are not necessarily systemic but should be corrected or 2) would result in better business practices if corrected.
To
promote the exchange of successful lessons learned and innovative
procurement methodologies between Centers, the team sought to identify
Goddard processes or initiatives that may be beneficial to other
Centers and conversely sought to identify suggested approaches utilized
by other Centers that may be beneficial to Goddard.
The exit
conference at the conclusion of the survey typically consists of a
direct exchange of observations and ideas between the participants. To
emphasize Center ownership of the resolution of any identified
weaknesses or considerations, the survey follow-up process will focus
on actions or initiatives undertaken by the Center to address survey
findings. At an appropriate interval (approximately six months after
this report is issued) the Goddard Procurement Officer will brief the
Assistant Administrator for Procurement and the survey team leader on
Center achievements in these areas.
Below is a list of team members and the areas they reviewed:
YOLANDE HARDEN |
Survey Team Manager, Interviews (Legal, 1102, & COTR), and Organizational Structure and Staffing |
RON BACKES |
Survey Team Co-Manager, Interviews, Self-Assessment Program, Award/Incentive Fee Contracts and CMM Issues |
JERRY EDMOND (HQ) |
Internal Policies and Procedures, Justification for Other Than Full and Open Competition (JOFOCs), UCAs, Deviations and Waivers and Master Buy Plan Actions |
VERONICA LANSEY |
Purchase Card Program, Environmental Issues, Adequacy of documentation resulting from Broad Agency Announcements (BAAs), and Contract Safety Requirements |
JEFF LUPIS |
HSPD-12 Compliance, SBIR/STTR Program, and Inter-Agency Agreements |
JOSE GARCIA |
Cost/Price Analysis, Financial Management Reporting, Audit Follow-Up, Pre/Post Negotiation Memorandums and Technical Evaluations |
MARY KISS |
Exercise of Options, Acquisitions in Process, Government Furnished Property, and Closeout and Unliquidated Obligations |
JALANE JOHNSON |
Construction and A&E contracts |
DARLA CLIFTON |
Grants and Cooperative Agreements, Commercial Acquisitions, Simplified Acquisitions, and Competition under Multiple Award and IDIQ Contracts |
CHRISTINE MONROE |
1102 Career Development and Training, COTR Delegations and Surveillance, COTR Training, and Contractor Performance Evaluations NF-1680 |
DAVE GROVE |
Small Business Programs |
The survey could not have been accomplished successfully without the support of the following individuals:
| JIM BECKER |
Goddard Point of Contact |
| NANCY LOCKARD | Goddard Point of Contact |
| VERONICA OKAI | Goddard Administrative Support |
| ANITRA TUCKER | Goddard Administrative Support |
1. Organization Structure and Staffing:
The Goddard Procurement organizational structure remains substantially unchanged from previous procurement management surveys. Recent additions to the organizational structure include the GOES-R Procurement Office supported by an Associate Chief and a small staff of contracting officers. This office was established to provide dedicated support to the GOES-R program office. An Organizational Development Specialist from the Office of Human Resources was recently added to the division staff to assist with personnel planning, issues and concerns.
Goddard Procurement still consists of five offices managed by an Associate Chief and two Procurement Managers. The Procurement Officer, Deputy, Senior Staff, Procurement Support Services Team, Industry Assistance Office, Organizational Development Specialist, Administrative Support Staff and Associate Chiefs all reside in the Center’s main administrative building. The Procurement Managers and the remainder of the procurement personnel are distributed throughout the Center and are either co-located or in buildings in close proximity to the technical organizations that they support.
Goddard Procurement is comprised of personnel with various skill sets and positions. Some of these positions are phasing out as a result of the transfer of certain types of activities to the NASA Shared Services Center (NSSC). A small number of contract administrators, grants specialists and procurement technicians remain in the organization. There is no plan to replace these positions as the individuals retire, leave the Center or transition into other positions.
2. Procurement Staff Interviews:
Interviews were conducted with several members of the Goddard procurement workforce to gain a greater insight into the effectiveness of the procurement organization. Interviews were conducted with a wide range of individuals varying from the most junior levels to senior personnel with more than 20 years procurement experience. The majority of people interviewed have 20 years or more tenure at the Center; however, it is noted that there are also a large number of employees with five or less years’ tenure. Most individuals in the latter category transferred from other government agencies (primarily Department of Defense), some were former participants in the NASA Contracting Intern Program.
The overall response from procurement personnel regarding the status of Goddard Procurement was mixed. Most felt that things were ‘okay’, many perceived an improvement in the status over the last few years. Co-location seems to insulate the various groups. When asked, people were able to discuss the environment within their individual organizations but did not have an appreciation for the environment within other organizations or the organization as a whole.
Most people interviewed perceived morale as good (particularly in their respective areas) and improved over the past. Few felt that morale was low. Based on the interviews, morale at Wallops is perceived as high. The perception of workload volume is that things are manageable. Generally, there are peaks and valleys in terms of the level of activity. A couple of people felt that their individual workload assignments were less than challenging and would like more complex work or assignments in different areas. The individuals indicated that their respective managers were aware of their desire for more complex or challenging opportunities.
The type of relationships with the technical organizations varied. In many instances, particularly on the larger programs and projects, procurement personnel are included in key meetings. Some still felt that relationships with the technical community are non-existent. One contracting officer’s technical representative (COTR) was specifically mentioned as exceptionally difficult to deal with and unresponsive to repeated requests for information.
Goddard Procurement management is perceived as effective in the areas of communications, management feedback, accessibility and flexibility. Training opportunities are available for Headquarters sponsored courses and courses sponsored by the Center. There appears to be a perception that the courses required for contracting certification are not available. Many individuals do not apply for the Headquarters sponsored courses and opt to enroll in courses offered locally by the Defense Acquisition University and the Federal Acquisition Institute. Slots in these courses are not always available; in contrast the slots are generally available in the Headquarters sponsored courses. The inability to travel (typically for personal reasons) is cited as the primary reason for preference for the local courses. As a result, many individuals do not attend the required courses in a timely manner.
The majority of people interviewed expressed their ability to communicate with the Headquarters Office of Procurement when necessary. Others, however, indicated that the nature of their workload did not require the need to communicate with the Office of Procurement. Many in this group could not identify the Headquarters Office of Procurement Program Operations Division Analyst for the Center.
Several individuals mentioned a survey conducted by the Office of Equal Employment Opportunity (EEO) in accordance with its Management Directive 715 (MD 715). The survey was conducted of procurement personnel in 2006 to determine whether certain barriers were present in the areas of opportunities for advancement, access to management, etc. The results of the survey were discussed with the Procurement Officer who recently conducted a series of meetings with procurement personnel to share the findings and discuss management’s plan of action to address them. Survey team interviews with procurement personnel did not reveal the mood or perception reflected in the EEO survey results – that management was inaccessible, opportunities for advancements were limited to certain individuals, etc.
STRENGTH:
The Goddard Procurement management team is commended for establishing an open line of communications with procurement personnel and proactively addressing the weaknesses cited in the EEO MD 715 survey. Examples include the series of small group discussions between the Procurement Officer and procurement personnel to candidly discuss the MD 715 survey findings and the recent initiation by some Associate Chiefs of regular newsletters addressing a variety of issues related to Goddard Procurement.
3. Technical Customer Interviews:
Representatives from a variety of technical organizations were interviewed regarding the effectiveness of Goddard Procurement and their level of satisfaction. Individuals interviewed represented various programs and projects at Goddard and Headquarters. Most COTRs interviewed had over 20 years experience. Many were not actively delegated as COTRs but served as technical monitors/officers or program managers with responsibility for other COTRs.
The majority interviewed indicated that the relationship with the procurement organization was good to excellent. Many stated that representatives from procurement were included in their staff, organization and strategic planning meetings. There were a few with limited interaction with procurement. Many perceived the co-location of procurement personnel within their respective organizations as a positive action and those without co-location expressed a desire to have it. COTRs at Headquarters perceived the distance from the procurement organization as a slight but not insurmountable disadvantage. It was recommended that contracting officers or contract specialists visit Headquarters periodically to make a direct connection with the COTRs and strengthen the relationships.
Overall, from the viewpoint of the technical organizations, Goddard Procurement is doing an effective job to support their respective organizations and the quality of procurement products is good. Those interviewed received the appropriate training and regular reminders are sent by procurement personnel for refresher courses. Some expressed a desire for more training regarding the role of the technical monitor/officer in addition to COTR responsibilities.
In general, technical organizations are receiving the appropriate level of support, advice and counseling from procurement. Most interact directly with the contract specialists and contracting officers; however, some had opportunities to interact with procurement managers and associate chiefs. This was particularly evident at the program level. Education regarding the role of procurement and an open dialogue between the procurement and the technical organizations were cited as critical components to successful relationships between the two organizations. In one particular case, a COTR described a situation where the relationship between the two organizations was virtually non-existent until both sides became more knowledgeable about their respective roles and responsibilities. As a result the working relationship blossomed; now there is constant interaction and both sides benefit.
CONSIDERATION:
Contracting officers and specialists within the Headquarters Operations Division should consider scheduling periodic visits to the Headquarters COTRs to discuss contract administration, responsibilities and other pertinent issues.
Interviews were conducted with two representatives from the Goddard Office of Chief Counsel responsible for reviewing procurement documentation. Review of procurement documents was a part time duty for both individuals and both had 5 years or less tenure at the Center.
The relationship between procurement and the legal office is perceived as positive. There are regular communications with procurement senior staff. The relationship is very open, quarterly luncheon meetings are held between legal office representatives and procurement managers to discuss common issues and problems. The quality of the documentation reviewed is generally perceived as good. There are exceptions; documents produced by individuals from certain organizations tend not to be the best quality. There are instances where the documents are not well written and it doesn’t appear that management reviews were conducted prior to requesting the legal office review. These circumstances are the exception rather than the rule.
The review levels are perceived as adequate. One comment or request is that the procurement personnel honor the 3 day turn around policy established for legal review. Documents are often submitted with requests for ‘rush’ reviews or regular reviews with calls to inquire about status after only one or two days.
CONSIDERATIONS:
1. Goddard Procurement should ensure that the appropriate level of documentation review is conducted internally prior to submission to the Office of Chief Counsel for review.
2. Goddard Procurement should ensure that sufficient time is provided to the Office of Chief Counsel for document review to the maximum extent possible.
5. Procurement Career Development and Training:
The procurement training coordinator provides outstanding support to the Procurement Career Development and Training Program. An examination of the training files and database was conducted along with an interview with the training coordinator.
The procurement training coordinator maintains a database which tracks personnel by branch, grade, degree status/number of business course hours; all required courses by certification level and the year of the person's latest Individual Development Plan (IDP). The procurement training coordinator also keeps track of all courses taken at the division level. However, each person within the division is responsible for tracking their own continuous learning points (CLPs). The procurement training coordinator maintains a spreadsheet with the information that is available to employees upon request. In addition, personnel are encouraged to utilize the SATERN database for tracking courses. Over half of the staff has IDPs. The attrition rate for Goddard Procurement this year is currently 6% which is lower than the past few years. Currently there are 182 individuals within Goddard Procurement comprised of 158 in the GS 1102 (contract specialist) series; 7 in the GS 1105 (purchasing agent) series and 2 in the GS 1106 (procurement clerk/assistant/technician) series. The remainder of the staff includes administrative/clerical personnel, administrative systems analysts and student employees.
The database is well-maintained and provides valuable information regarding the required course training needs. Goddard procurement management has assessed the warrant level of personnel to ensure that warrant levels are commensurate with their certification levels. Currently there are 90 warranted contracting officers. A random review of the backup documentation to support the database was also performed and the appropriate information was contained in the files. It is noted that the current Goddard Procurement policy regarding the issuance of warrants is inconsistent with the agency policy. The Procurement Officer is aware of the situation and advised the survey team that the appropriate changes will be made to the Center policy.
Goddard’s management team meets every six months to review personnel profiles. During this review the procurement training coordinator documents any actions generated during the meeting regarding an individual’s performance and forwards the information for each employee to the respective Procurement Manager. After the review, the Procurement Manager meets with employees to compare developmental progress to their IDPs and develops a plan for each employee to assist in achieving the IDP goals. This process enables Goddard procurement professionals to obtain a variety of experience in both procurement and non-procurement areas. The Goddard Procurement team has acknowledged that there is a constant need for on-the-job training particularly since there is a high turnover of procurement personnel at the Center.
The Procurement Officer established a “Learning Group” session originally comprised of attendees from the December 2006 Procurement Training. Membership has grown to include other interested individuals. Once a month members meet with the Procurement Officer to discuss issues, concerns and ideas relevant to Goddard Procurement.
Goddard Procurement has both formal and informal mentoring programs and there is also a center-wide program. These programs encourage employees to pursue higher education courses and degree programs. Opportunities to obtain continuous learning points are made available through workshops conducted by Goddard Procurement personnel as well as Center course offerings. The local National Contract Management Association chapter also offers training sessions.
There are several
employees at the GS-12 and higher level who need the training required
for the Level III Federal Acquisition in Contracting-Certification
(FAC-C). The procurement training coordinator is currently
updating the files and plans to send reminders to the employees to
ensure they are advised of the upcoming courses. A Center website
is also being created that contains guideline information regarding
certification requirements and upcoming courses
STRENGTH:
Goddard Procurement is commended for the overall quality of the training database and backup documentation.
CONSIDERATION:
Goddard Procurement should ensure that employees are aware of the opportunities to attend requisite training for contracting certification.
6. Contracting Officer’s Technical Representative (COTR) Training:
The NASA FAR Supplement (NFS) requires all COTRs and alternate COTRs to attend mandatory training prior to appointment. Further, COTRs must attend a refresher training every five years. Goddard Procurement Policy Office maintains a local COTR database for both Goddard and Headquarters. The majority of individuals listed in the database are current with their COTR training. The database contained 263 Headquarters COTRs and 964 Goddard COTRs.
The survey revealed that a separate master list of COTRs did not contain fields to indicate the contract number, contract completion date, or whether the COTR/technical Officer were still serving in that capacity on any Goddard or Headquarters contract, grant or cooperative agreement. While this is not a regulatory requirement, it would make the database more user-friendly and add overall value to the information contained therein. A number of the individuals listed in this particular database had not received COTR training in several years. This database should be removed from the website and replaced with the List of Active COTRs maintained by the Goddard Procurement Policy Office. The Policy Office should ensure that all fields in the Active COTR database are populated and amend the list to include the last year and/or the two most recent training sessions completed.
The NFS requires that the mandatory COTR training cover the following core topics: 1) contracting authority and contract modifications; 2) inspection and surveillance; 3) changes and performance-based contracting; 4) contract financial and property management and 4) disputes.
The basic COTR training course is conducted by contractors. It contains comprehensive coverage of the topics required by the NFS as well as other pertinent topics relevant to procurement and COTR responsibilities. The basic course is offered semi-annually or upon request. In addition, during the interviews conducted with the COTRs, they indicated that they were advised about the responsibility to submit contract files to the contracting officer for inclusion as part of the final contract file for closeout purposes.
CONSIDERATIONS:
1. Goddard Procurement should amend the COTR database to incorporate fields to include contract number (i.e. contract or grant), contract completion date, active or inactive COTR designation and the dates of the two most recent training sessions.
2. Goddard Procurement should consider either amending the COTR course curriculum to include information that specifically addresses task manager, SBIR COTR and technical officer responsibilities or conduct separate training sessions for these individuals to ensure a full understanding of those specific roles.
7. Internal Policies and Procedures:
The Goddard Procurement system of procurement policies, regulations, information circulars, and work instructions is managed by a procurement analyst within the division’s senior staff. All of these regulatory issuances are available electronically through the Goddard Procurement Library; as a result the policies, procedures, information, and instructions are immediately available and useful to the procurement workforce at the Center. The Goddard Procurement guidance is traced back to policies and procedures based on the importance of the documents that institutes those polices (e.g. Federal Acquisition Regulation, NASA/FAR Supplement, NASA Procurement Information Circulars, etc). This ensures Center policies are consistent with both Federal and NASA procurement policies and procedures.
Detailed guidance from the Air Force regarding technical evaluations is available on the Headquarters Procurement Office (Code 210.H) policy website. However, more straight forward technical evaluation guidance is available on the Goddard Programs and Project Management website which includes an example of a technical evaluation. It is suggested that Goddard Procurement include a link on its policy website to this technical evaluation guidance.
Review of the guidance revealed that the Goddard Procurement contracting officer warrant policy conflicts with the Agency policy. The current Goddard Procurement policy grants individuals with Level II certification and training contracting officer warrants with authority up to $50M. The Agency policy only grants individuals with Level II certification and training warrants with authority up to $10M for Commercial and Non-Commercial Items and unlimited warrant thresholds for incremental funding actions.
STRENGTH:
Goddard Procurement is commended for the overall quality of the Center's procurement policies and procedures.
CONSIDERATION:
Goddard Procurement should incorporate a web-link to the Programs and Project Management guidance on developing and writing Technical Evaluations into its procurement policy website.
WEAKNESS:
Goddard Procurement should amend their warrant policy for Level II certified contract specialists to ensure compliance and consistency with the Agency warrant policy.
The self-assessment program is designed to ensure that the procurement function throughout NASA is operating at a low level of vulnerability and that weaknesses identified in NASA Headquarters procurement surveys are tracked to closure. Self-assessments should focus on the implementation of corrective action plans developed to remedy weaknesses identified during the latest procurement management survey.
A Procurement Management Survey was conducted of Goddard Procurement from August 15th - 26th, 2005. Three self-assessments were subsequently conducted: April 2006, February 2007, and April 2007. The self-assessments focused on random compliance reviews of a representative sample of procurement actions from fiscal years 2005 and 2006. The actions selected for review reflected a representative sample of procurement offices and contract specialists as well as a variety of contract types and actions with small and large businesses, universities, nonprofits, other governmental organizations, and foreign entities. The self-assessment reports contained a summary of the findings and recommendations and a comparison of the current self-assessment to the last self-assessment and the most recent Headquarters survey. Recurring issues were identified and corrective actions were monitored for effectiveness. Where corrective actions were not effective, additional corrective actions were recommended.
Goddard Procurement Circular 03-01 dated July 23, 2003, contains additional guidance on performing self-assessments. The Circular requires reviewing team members to furnish a copy of the summary and findings for each contract action reviewed to the contract specialist at the conclusion of the file review. Additionally, the contract specialists are required to provide a written response to any final review findings where a need for corrective action is identified. The Procurement Manager must concur in the response and corrective action.
Discussions were conducted with the previous and current self assessment team leaders to further clarify the team formation and self-assessment processes at the Center. The self-assessment leader selects a mix of volunteers from Goddard Procurement based on experience and level of expertise. Self assessment team membership is also extended to individuals who will benefit from the experience as a targeted training opportunity. Goddard Procurement files are reviewed by the assessment leader including files from Wallops Flight Facility and Headquarters. The team leader schedules a kick-off meeting, assigns topics and files for review, and establishes assessment timelines. Corrective action plans are discussed individually in topic staff meetings with senior staff and managers.
STRENGTH:
The Goddard Procurement self-assessment process is highly effective at identifying issues at the Center and developing corrective actions to address shortfalls. The process includes an exceptional feedback mechanism to ensure individual accountability and management oversight.
The Agency’s Master Buy Plan requirements are stated in NFS 1807.71. In accordance with NFS 1807.7100, “The Master Buy Plan provides information on planned acquisitions to enable management to focus its attention on a representative selection of high-dollar-value and otherwise sensitive acquisitions”. In recent months, NASA senior management’s focus on the Master Buy Plan (MBP) Database has increased.
The Office of the Administrator now requires the Headquarters Office of Procurement to provide an Annual MBP Report of ALL MBP records submitted by the Centers. Due to this increased focus, Goddard Procurement (as well as other Centers) is advised to become more diligent regarding the level of quality and detail of data entered into the MBP database. The Headquarters Office of Procurement (Program Operations Division) has noted that in most cases, the data fields that often lack sufficient detail include: (1) Description and (2) Status Schedule.
Description - In many cases the MBP Description field reflects a one-line description of the planned acquisition. This type of description provides little more information about the acquisition than the title. The MBP description should be sufficiently detailed to provide NASA management a clear understanding of the acquisition requirements (e.g., similar to the level of detail normally provided in the description section of an ANOSCA).
Status Schedule - Planned dates for critical milestones such as JOFOC Approval (if applicable), RFP Release, Source Selection, and Contract Award must be supplied within the Status Schedule field.
As a related advisory, the information provided in the Acquisition Forecasts regarding planned procurements will receive the same level of scrutiny (for ALL Centers). The descriptions of these planned procurements must be sufficiently detailed for the same reasons stated above. The necessity for detailed descriptions is particularly applicable to Acquisition Forecasts in the “White” or “Yellow” categories.
Currently the Status/Schedule field in the MBP User Operations Guide (Release 3.6) is not mandatory, however, the latest focus on MBP by agency senior management may create a need to reclassify the field and make it mandatory.
CONSIDERATIONS:
1. Goddard Procurement must ensure that MBP Record submissions are sufficiently detailed and that particular attention is given to the Description, and Status/Schedule Data Fields.
2. NASA Headquarters Office of Procurement should consider making completion of the Status/Schedule Field a mandatory requirement.
Deviation requests were reviewed for compliance with FAR Subpart 1.4 and NFS Subpart 1801.4. All Goddard Procurement submissions met the requirements of the FAR and NFS and for the most part were well-documented. Only on a few occasions was additional data or clarification needed. When additional data was needed, Goddard Procurement responded quickly and in an appropriate manner.
An inquiry was made to the Procurement Policy senior staff to determine if a file of approved deviations and waivers is maintained in the office. The Procurement Policy senior staff does not currently maintain a list of pending or approved deviations and waivers. While not a requirement, the establishment of such a document or database is recommended as a good business practice.
CONSIDERATION:
The Goddard Procurement Policy senior staff should establish and maintain a file of approved deviations and waivers.
3. JOFOCs/Competition Advocacy:
Noncompetitive contract files were reviewed for compliance with FAR Subpart 6.3 and NFS Subpart 1806.3 regarding other than full and open competition. Dollar values ranged from $.378M to $13M. Statutory authorities included (c) (1) - only one responsible source and (c) (2) - unusual and compelling urgency. All of the JOFOCs were well-documented and contained sufficient rationale to support the sole source. All cited the appropriate authority and all were reviewed and approved at the appropriate level. It is noted that Goddard Procurement policy for all JOFOCs with a value of $500K and above requires the Procurement Officer’s concurrence and Competition Advocate approval. The Center Deputy Director is the Competition Advocate. JOFOCs greater than $750K require legal office concurrence. This policy ensures that JOFOCs undergo extensive reviews before approval. Goddard Procurement also uses an internal Procurement Circular, issued in 1997, as an effective tool for ensuring all of the FAR and NFS requirements are addressed in the documentation. This Circular is periodically updated to include current requirements and includes a JOFOC template.
Two JOFOC files from
contracts NNH05CC15C and NNG07FA09C are highly commended for being
exceptionally well documented and well written.
STRENGTH:
Goddard Procurement is commended for the overall quality of the JOFOCs reviewed.
The primary emphasis of this review was to determine if the awards were made in compliance with current procurement regulations and policies. In general, the procurement files selected for review contained the appropriate level of information, were well documented, and followed the appropriate procedures. This was particularly true for larger acquisitions, including those utilizing Source Evaluation Board procedures. With the exception of files utilizing simplified acquisition procedures, all files reviewed utilized the NASA Form 1098, Checklist for Contract Award File Content, which helped to ensure that all required documentation was both included in the award file and filed in the appropriate location.
For acquisitions of all sizes, the reviews indicated that the overall source selection process was effective and resulted in timely awards of procurements. For non-competitive actions, the sole source documentation was generally thorough and followed the required FAR format. For large competitive acquisitions, source evaluation teams were appointed prior to receipt of proposals. Evaluation documentation was also found to be very thorough and comprehensive. When not performed by the contract specialist, pricing support for competitive actions was provided by dedicated pricing support analysts in support of these activities. Additional supporting documentation for large competitive acquisitions indicated that policies and procedures were followed and that each step was thoroughly documented. This documentation provided an acceptable audit trail from acquisition planning through final award. In addition, source selection statements, which considered the findings of the board, were thorough, well documented, and provided clear rationale for the basis of selection.
Reviews of the files selected indicated no systemic problems, regardless of acquisition type or level of complexity. These reviews indicated that contract specialists were, in general, familiar with the policies and procedures governing the performance of their job responsibilities.
Non-systemic and isolated instances relating to a lack of proper documentation were noted, though these instances were often mitigated by other documentation provided elsewhere in the contract file. Other minor non-systemic findings included occasional lack of VETS100 certifications, lack of market research documentation in a sole source justification, lack of CCI consultation, and lack of debarred list consultation.
5. Pre-Negotiation and Post-Negotiation Documentation:
Files were reviewed to ascertain the adequacy of pre-negotiation position memorandums (PPM) and price-negotiation memorandums (PNM). Utilizing the standard format prescribed in Goddard Procurement Circular (PC) 02-01, Pre-negotiation Position and Price Negotiation Memorandums, all eleven areas specified in FAR 15.406-3 and NFS 1815.406-170 are addressed consistently.
A review of Pre-negotiation Position Memorandums and "stand alone" Price Negotiation Memorandums revealed the documentation was not always consistent or comprehensive. When the Goddard Form 210-94 (stand alone document) is used, it is important that it is truly stand alone and the documentation addresses all the required FAR material. It is also important that the Goddard Procurement review team ensure that the requirement of PC 02-01 is satisfied, particularly as it relates to the following:
The contract file must contain a separate memorandum documenting sound rationale for not preparing a separate PPM and requesting permission to prepare a “stand-alone” PNM. The request for “stand-alone” memorandum shall contain a summary of the cost and fee/profit differences as well as any other negotiation issues.
In a majority of the contracts reviewed, subcontractor data was not reviewed for adequacy. There appears to be a heavy reliance on the technical evaluation to review subcontractor costs. In most cases, the technical evaluation was silent in addressing subcontractor costs; however, the PPM stated that the technical evaluation accepted the subcontractor costs. The files reviewed did not appear to consistently include adequate documentation related to the adequacy or status of a contractor’s systems. The utilization of Goddard Procurement forms is helpful since the forms include a comprehensive list of items in accordance with the FAR requirements; however, contract specialists and contracting officers appear to rely too much on the form alone and do not always include the supporting narrative rationale.
One file included a negotiated amount greater than the objective approved in the PPM. Although the amount was not significant, the file should have documented the reason for exceeding the objective position. Additionally, no maximum position was developed to ensure that the appropriate range was available for negotiation.
CONSIDERATION:
Goddard Procurement should emphasize the importance of adequate documentation to contracting specialists and officers. PPM and PNM files should contain all supporting documents. The file documentation should be comprehensive, including proposed subcontractors and adequate narrative rationale.
WEAKNESS:
Goddard Procurement should emphasize the development of both an objective and maximum in circumstances where two negotiation positions for cost elements and the profit/fee may be appropriate. (Reference PIC 00-04)
The previous two procurement surveys conducted in 2003 and 2005, noted that technical evaluations lacked sufficient detail and supporting rationale. A review was performed during this survey to ascertain the acceptability and improvement of technical evaluations and their compliance with the requirements stated in FAR 15.404.
The review revealed very similar findings to those during the prior surveys. Most of the technical evaluations reviewed accepted the proposed labor skill mix and hours without indication of the performance of any critical analysis. There were no indications in either cost analysis reports or Pre-negotiation Position Memorandums that the contract specialists or contracting officers conducted any follow-up actions to obtain more information to support the technical evaluator's recommendations. On smaller procurements, the proposal submitted by the offeror or contractor did not contain the data in a format necessary to facilitate an appropriate evaluation. Proposals did not always time phase hours and skill mix or provide hours by tasks.
In the previous surveys it was recommended that the contracting officer tailor requests for technical evaluations to reflect the essential information needed to conduct effective technical evaluations. It appears that the requests for technical evaluations were tailored; however, not all were consistent with the information requested nor did the requests address the FAR minimum requirements. This includes an evaluation of the types and quantities of labor and materials. Also, very few requests actually required the technical evaluation to include the rationale for acceptance or exceptions taken to proposals. On a positive note, the technical evaluation report usually complied with those elements requested by the contracting officer.
Contract NAS5-99089 utilizes a matrix style format to capture the technical evaluation. This format was very effective since it captured the proposed and objective positions and contained the rationale for each of the Government’s positions. The utilization of this type of matrix ensures that all critical and significant elements are evaluated and documented.
CONSIDERATIONS:
1. Goddard Procurement should consider including language in solicitations that requires the offeror or contractor to provide a proposal that time phases the resources and pricing data by year. The utilization of templates with specific instructions to capture the required data are helpful, provides consistency across proposals and allows the technical evaluators to receive the appropriate type of data and information necessary to perform quality technical evaluations.
2. Goddard Procurement managers should consider developing a standard technical evaluation request template that includes at a minimum the FAR requirements. The standard request template should also state that the evaluation needs to reflect an analysis of the proposed hours and skill mix from a time phase perspective. Additionally, the standard request template should emphasize that written documentation accompany the analytical approaches and results of the evaluation. Finally, the contract specialist should be allowed to amend the standard template with additional or specific areas such as facilities, equipment, training, consultants, etc., as appropriate.
3. Goddard Procurement should consider not only sending the request to the technical evaluator but also meeting with the evaluators to discuss technical issues and concerns prior to negotiations. Direct communication between the procurement and technical representatives ensures that: a) the technical community understands the objective of the action and increases the quality of the analysis and documentation and b) the procurement professional receives the necessary data to develop negotiation positions.
WEAKNESSES:
1. Goddard Procurement managers should ensure that emphasis is placed on the content and quality of technical evaluations by both procurement and technical professionals. Managers should ensure that contract specialists and contracting officers are familiar with the pertinent FAR requirements and that the appropriate information is included in the request for technical evaluations (see consideration). Procurement personnel should also ensure that the quality of the technical evaluation is consistent with the financial risk and complexity associated with the procurement action. Follow-up measures should be taken if the appropriate technical input is not received.
2. Goddard Procurement should ensure that the technical evaluators comply with the technical evaluation request. All critical and significant areas should be evaluated with analytical approaches to determine if the proposed resources are consistent with the proposed approach. The evaluation should include supporting rationale for technical determinations. Sufficient documentation should be provided in narrative form to allow the contract specialist adequate insight to develop negotiation positions. (REPEAT FINDING)
7. Contractor Safety Requirements:
Several contract files (non-construction) were reviewed to verify compliance with NASA FAR Supplement (NFS) Subpart 1823.70, entitled “Safety and Health”. All of the contracts reviewed required either NFS Clause 1852.223-70 “Safety and Health” or NFS Clause 1852.223-72 “Safety and Health (Short Form)” depending upon the input received from the technical organization and the Safety and Environmental Branch.
All of the files reviewed contained the “Goddard Initiator's Acquisition Safety Checklist for Procurements Form” and the required NFS Clause 1852.223-75. Every “Goddard Initiator's Acquisition Safety Checklist” reviewed was approved by the Safety and Environmental Branch. Overall, files reviewed contained the required safety and health clauses and, if applicable, the approved contractor’s safety and health plan.
1. Contractor Performance Evaluation:
Several contract files were reviewed for compliance with FAR 42.15 and NFS 1842.15, including several evaluation periods. All files contained the required NF 1680 forms. Two files contained the NF1680 forms, with some for more than one period.
NFS 1842.1502 requires contracting officers to conduct interim evaluations of performance on contracts within 60 days of the anniversary date. Although some of the reports in the files reviewed were initiated in a timely manner, none were completed with the same timeliness, and some reports were completed several months after the required dates. However, several files contained documentation of attempts to contact the COTRs and their non-responsiveness was present.
Many of the forms contained narrative explanations to support the adjective ratings, and evidence of discussions of the evaluation with the contractors was present in all files. All of the evaluations were properly signed by the contracting officer upon completion. Only half of the files with NF1680 documentation were listed in the Past Performance Data Base (PPDB), and posting of these reports in PPDB was sporadic.
CONSIDERATIONS:
1. The Goddard Procurement Officer should ensure that contracting officers comply with the FAR and NFS requirements regarding contractor performance evaluations, including meeting the 60-day lead time for completion of the evaluation, and providing substantive narrative to justify the ratings.
2. Headquarters Office of Procurement should establish a mechanism that alerts the contract specialist to ensure that they are informed that the data in PPDB was not posted as complete.
WEAKNESS:
Goddard Procurement should ensure that past performance data is entered into the PPDB on a consistent and timely basis and is saved as permanent records so that it does not remain in draft form. Permanent records from PPDB are utilized not only by NASA procurement personnel but also integrated into the government-wide Past Performance Information Retrieval System for use in source selection activities.
2. Award/Incentive Fee Evaluation:
The NASA FAR Supplement (NFS) requires the Procurement Officer’s approval for the use of an award fee incentive contract along with a discussion of the other types of contracts considered and the rationale for selecting an award fee incentive. Goddard Procurement Circular 00-04, revised March 2006, requires the Procurement Officer approval in the form of a separate memorandum, prior to or concurrent with the approval of the procurement plan or acquisition strategy meeting documentation. The policy also requires written concurrence by the cognizant resources analyst at the Project or the Directorate level in the cost benefit analysis.
Several files were reviewed for compliance with FAR, NFS, and Goddard policy. In addition, Past Performance Database (PPDB) entries were reviewed for consistency with Performance Evaluation Board (PEB) evaluation documentation. During the entrance conference, it was noted that fifty eight PEBs were monitored by Goddard Procurement. Only seventeen award/incentive fee contracts were identified through the Contract Management Module (CMM). The disparity in the active incentive contracts reported through CMM and the PEBs monitored was not clarified.
The Procurement Officer and senior staff monitor PEBs closely for success, including timely execution of PEB assessments and timely issuance of Fee Determination Officer (FDO) letters. Upcoming PEB’s are reported to Procurement Managers for planning purposes, and completed PEBs are monitored to identify trends. Timeliness is monitored by directorate, and specific mission areas are examined to determine the source of any delays and corrective action is taken.
The General Accountability Office (GAO) recently issued a report entitled “NASA Procurement Use of Award Fees for Achieving Program Outcomes Should Be Improved” that reviewed three award fee contracts at Goddard: EOSDIS, Landsat-7, and Mechanical System Engineering Services (MSES). Of these contracts, one follow-on contract was awarded since the GAO report was issued, for the MSES II. The other two follow-on actions are in the acquisition planning phase. The survey team reviewed the MSES II contract for corrective actions taken, if any, as a result of the GAO audit findings on the previous contract.
Acquisition strategy meeting and/or acquisition plan documentation identified the proposed contract type and in most cases provided a basis for selection. The majority of files reviewed did not contain a Procurement Officer’s determination or cost benefit analysis (CBA). The single CBA found excluded the costs of personnel assigned full-time to the project. Performance evaluation data for approximately 50% of the contracts was contained in the PPDB. The PPDB entries were consistent with PEB evaluations. Award/incentive fee contracts were found to be generally well administered, with excellent monitoring of fees awarded to fee ceilings.
Goddard Procurement uses the Task Order Management System (TOMS) to manage task orders issued under large volume Indefinite Delivery Indefinite Quantity (IDIQ) contracts. A task monitor (TM) is delegated authority by the Contracting Officer’s Technical Representative (COTR) to monitor and report on contractor performance on specific task orders. At the conclusion of each six-month reporting period, the COTR submits a questionnaire through the TOMS to all TMs with active task orders to evaluate and rate contractor performance during the preceding period. The COTR consolidates all questionnaires and presents the results to the PEB to recommend the appropriate award fee for the period. The TOMS provides data and reports to the COTR and Contracting Officer to facilitate the administration of large scale, high-volume, award fee task order contracts. The TOMS also provides reports reflecting the performance rating and amount of fee awarded on all tasks awarded under a contract.
STRENGTH:
Goddard Procurement is commended for improving the timely completion and approval of award fee plans and the effective execution of the award fee process. As a result of these efforts, the average days late for PEBs and FDO Letters has decreased from approximately fifteen days to approximately five days.
CONSIDERATION:
Goddard Procurement should ensure that the cost benefit analysis developed for the establishment of award fee contracts identifies all administrative costs (including the costs of all personnel assigned to the PEB, procurement personnel and other staff as appropriate). It should also capture the cost of time spent by all personnel performing the additional responsibilities required by the PEB to provide a reasonable basis for assessing the relative value of the award fee process.
WEAKNESS:
The Goddard Procurement Officer should ensure that pre-award contract files contain documentation required by NFS and Goddard Policy concerning the selection of award fee contracts.
3. Closeout and Unliquidated Obligations (ULOs):
The review of closeouts and unliquidated obligations consisted of an examination of the Closeout workbook of spreadsheets created in support of a Corrective Action Plan that teams Goddard Procurement, the Goddard Regional Finance Office, the Goddard Resource Management Office, and the Goddard Chief Finance Office together to reduce the number of actions awaiting closeout and those with un-disbursed obligations.
The closeout function is predominantly decentralized at the Center. The Agency closeout contractor, Legacy, provides support to Goddard Procurement. Legacy is responsible for closing out grants and a small number of less complicated contracts.
Goddard Procurement maintains a database with information on expired but not closed files. These files are reviewed by the corrective action plan team members to resolve outstanding issues. In addition, the Goddard Procurement systems team conducts “Closeout Days”. The close-out system experts visits the various division offices, works with the individual specialists to reconcile any ULO’s or other issues on a real time basis and conducts timely follow-up actions as appropriate. The approaches taken by Goddard Procurement to address close-out and unliquidated obligation concerns are positive steps.
The implementation of the Contract Management Module (CMM) created an additional hurdle to overcome in the close-out process. Goddard Procurement developed a process to facilitate close-out of some simplified acquisitions by directly populating CMM and SAP with the information and accession numbers using the Closeout Checklist in CMM.
STRENGTHS:
1. The Goddard Procurement Officer is commended for establishing a strong relationship with the Finance organization and working as a team to reconcile and reduce the number of records awaiting closeout and reducing the dollar amount of undisbursed obligations. The initiation of the Corrective Action Plan is also a positive step.
2. The close-out contractor’s procedure to maintain records of all actions taken and the ability to provide a report upon request to the cognizant contracting officer or systems team member provides timely insight into close-out activities.
3. Goddard Procurement is commended for taking the initiative to successfully work with the system interface limitations in the processing of simplified acquisition closeouts in CMM.
CONSIDERATION:
Goddard Procurement should continue to take steps to reduce the significant volume of files (over 14,000) awaiting closeout.
4. Undefinitized Contract Actions (UCAs):
The majority of the UCAs identified in the May 2007 UCA report submitted to the Headquarters Office of Procurement were reviewed during this survey. The file review focused on age, dollar value, appropriate approval level, justification and file documentation in accordance with NFS 1843.70. For the most part there is prudent justification and file documentation prior to the issuance of UCAs. During the previous survey there were 20 open UCAs valued at greater than $400M. There were only 4 open UCAs valued at $9M at the time of this survey. All of the currently open UCAs are well within the 180 day limit for definitization. Goddard Procurement has done an excellent job of definitizing their UCAs within 180 days, which is an improvement from the previous survey.
One of the two UCAs definitized since the May 2007 UCA report did not use the proper form to document the price negotiation in accordance with Goddard Procurement policy. Also, a cost analysis was not included in the file. The UCA was negotiated below the contractor’s proposed amount due to rate differences; however, the DCAA rate verification was not included in the file.
STRENGTH:
Goddard Procurement is commended for the reduction in the number of UCAs from the previous survey and successfully definitizing their open UCAs within 180 days.
CONSIDERATION:
Goddard Procurement should ensure that the proper cost analysis is performed when implementing the modification to definitize UCAs.
5. Competition under Multiple Award Task and Delivery Order Contracts:
Blanket Purchase Agreements (BPAs) awarded against GSA Schedules (single and multiple award), open market BPAs for services, delivery orders awarded under GSA and Goddard Procurement multiple award contract vehicles, and delivery orders awarded under NASA SEWP were reviewed for compliance with requirements for competition.
Four of six BPAs reviewed and awarded against GSA multiple award schedule contracts resulted in single awards and two resulted in multiple awards. The file documentation indicates that the BPAs were competed in accordance with the procedures in FAR 8.4, Federal Supply Schedules. However, there was no file documentation for the determination to establish a single award BPA as required by FAR 8.405-3(a)(1).
Several delivery/task orders awarded against multiple award BPAs (both GSA and open market) were reviewed. One (GSA delivery order) was awarded as a sole source and the files for the competed actions contained documentation that each vendor was afforded an equal opportunity to bid on the orders.
Since the Solutions for Enterprise-Wide Procurement (SEWP) contracts are awarded and maintained by Goddard Procurement, delivery orders awarded against SEWP were evaluated separately. Only two of the files reviewed included documentation that all vendors within the category were provided fair opportunity to compete and contained a well documented award decision. Many files included a SEWP Request for Quotation which was sent to four or five vendors, but not to all vendors within the same category; two files indicated that the order was awarded against a single-award contract when the contract was actually a multiple award contract. In some cases the SEWP search tool was used determine sources, but in some cases supplier part numbers were used for the search rather than manufacturer part numbers.
A SEWP memorandum is available that verifies fair opportunity consideration for each contractor in a particular class. A Summary of Quotation/Selection Criteria for SEWP Multi-Award Classes/Single Award Class form is available for utilization; however, space to document vendor quotes is limited.
CONSIDERATION:
1. Goddard Procurement should ensure that GSA single award BPA files document the determination to establish a single BPA in accordance with the requirements of FAR 8.405-3(a)(1).
2. Goddard Procurement should consider modifying the Summary of Quotation/Selection form to include a section that reflects all vendors in a category. A section with an option to select a particular category and inclusion of a copy of the RFQ displaying all vendors may be helpful.
WEAKNESS:
Goddard Procurement should ensure that all awardees are provided a fair opportunity for consideration on each SEWP task order. The SEWP search tool should not be used to restrict sources. Additionally, the current version of the SEWP memorandum to the file should be included in the task order file along with the contract selected for award.
6. Options to Extend Performance:
Option exercise packages were reviewed to determine whether contract modifications to extend the period of performance were supported with the documentation required by FAR 17.207 and NFS 1817.207. Of the option packages reviewed, all contained a determination and findings document that: 1) verified funds were available, 2) the requirement covered by the option fulfilled an existing need; and 3) stated the need for continuity of operations. All of the determinations also indicated the option exercise was conducted in accordance with the terms of the contract and the requirements of FAR 17.207 and Part 6.
Improvement over the last survey results are noted – specifically, evidence of analysis by the contracting officer and the requiring organization to determine if the option was the most advantageous method was present in the files and dated prior to the notice of intent to exercise an option in accordance with FAR and NFS. In one case, however, the file contained an e-mail from the COTR to the contractor congratulating them on the planned exercise of the option a week prior to execution of the contract modification. There was one case where a bi-lateral modification was required because the letter of intent was not forwarded to the contractor in accordance with the contract terms and conditions. One exceptional file was reviewed that fully documented the research undertaken by the contracting officer in conversations with the COTR to validate the requirement prior to forwarding the notice of intent to the contractor.
One contract from Wallops Flight Facility contained four executed option modifications during the course of one year. Further review revealed that the numerous modifications were the result of on-going source evaluation board activities for the follow-on effort. The necessary documentation was present in the file backup and the notices of intent were sent as required. They were well documented files and in accordance with the FAR and NFS clauses.
The review of option modification documentation for contracts in the Headquarters Procurement Office (code 210.H) revealed some improvement since the last survey, however, some concerns were still noted. A variety of errors and/or omissions were present in several files reviewed. Specifically, in one instance although the contracting officer and the requiring organization determined that option exercise was appropriate in a timely manner, the notice of intent was not forwarded to the contractor in the appropriate time frame. Another example included a contract where the contract type was changed from firm fixed price to time and materials in the option modification with no explanation in the file.
Three contract modifications reviewed raised the most concern – they were executed after the commencement of the option period, and one contained no evidence of review by the requiring organization. Additionally, it appears that issues were present regarding contractor payment and cancellation of a previous modification to correct errors, but neither explanation of the errors nor their impact was evident. Further, the contracting officer did not sign and date all file documentation. The contracting officer also referenced that a 30-day notice was required; according to the clause in the contract, a 60 day notice was required. Since the notices were not issued in either case, the modifications were bilateral. One of the referenced modifications was issued in 2005, however, two were issued in 2007.
CONSIDERATIONS:
1. Although improvement in documentation is noted since the last survey, areas of concern are still present. The Goddard Procurement Officer should remind contracting officers to sign and date documents in the files when they are prepared.
2. The Goddard Procurement Officer should ensure that contracting officers document the analysis supporting the determination to exercise the option prior to providing notice to the contractor and that contracting officers execute the option modification before the option period begins.
7. Government Furnished Property:
The files reviewed contained the proper clauses in accordance with the FAR and NFS. The files were forwarded for the Supply and Equipment Management Officer (SEMO) review as required by NFS 1845.102-71(b). The files contained the list of property to be furnished to the contractor when appropriate. NF 1018 reporting is conducted electronically when required by NFS 1852.245-73.
There appears to be improvement in regards to the SEMO reviews conducted since the last survey, and the files contained proper documentation and reports. A representative in the SEMO office stated that the relationship and cooperation between Goddard Procurement and the SEMO office has never been better and has strengthened over the last couple years. Goddard Procurement has submitted timely packages to the SEMO for review and recommendations are incorporated into solicitation documents. The recent changes to FAR Part 45 has had some impact on the SEMO office’s ability to review packages, however, they are cooperating with Goddard Procurement and communication is good. Advance communications regarding planned procurements for cost reimbursement contracts that do not contain Government Furnished Property have improved.
STRENGTH:
Goddard Procurement is commended for the relationship developed and maintained with the SEMO office.
8. COTR Delegations and Contractor Surveillance:
The review revealed that COTR delegation forms (NF 1634) were not always present in contract files or issued in a timely manner as required by NFS 1842.270. The submission of a contract surveillance plan is one of the key components of the COTR delegation. When required, the COTR must establish and provide a contract surveillance plan to the contracting officer. A random selection was conducted in reviewing surveillance plans. The review revealed that some of the NF 1634s included the requirement for the presence of a surveillance plan and the corresponding on-site surveillance in accordance with the plan. However, in a few cases surveillance plans were not present as required by the delegation. Discussions with one contract specialist and the COTR revealed that the omission was an oversight.
WEAKNESSES:
1. Goddard Procurement should ensure that COTR delegations are issued in a timely manner and that all files contain a COTR delegation in accordance with NFS 1842.270.
2. Goddard Procurement should ensure that all COTR delegations (NF 1634) requiring surveillance plans and contractor surveillance include the appropriate documentation in the file as well as evidence of surveillance. If surveillance plans are not required, COTR delegations should be modified accordingly.
1. Financial Management Reporting:
The review focused on compliance with NFS Subpart 1842.72 regarding NASA Contractor Financial Management Reporting. During the previous survey it was recommended that a standard checklist be used to document the results of the analysis. A checklist to document the analysis of financial management reports was developed. The Goddard Procurement 2007 self assessment report stated that the contract files do not appear to include evidence that an independent procurement analysis is being documented. Review of the contract files revealed the same findings as the self assessment report. Very few files reviewed contained written narrative supporting a variance analysis and subsequent disposition when there were variances between planned and actual expenditures.
STRENGTH:
Goddard Procurement is commended for the incorporation of the checklist to document analysis of financial management reports. The checklist appears to be a controlled internal document (Goddard Form 210-49). The checklist is comprehensive in nature and acts as a good reminder to the contracting professional that they must perform a review.
CONSIDERATION:
Goddard Procurement management should remind procurement professionals that narrative rationale must accompany the Form 210-49 when variances between planned and actual expenditures exist. Variances should be adequately documented and resolved.
Goddard Procurement currently has two outstanding DCAA audits both are associated with the same audit report. The same plan for resolution and disposition exists for both. These audits are associated with a terminated contract. NASA personnel completed fact-finding of the termination proposal in March 2007. This fact-finding exercise resulted in a tentative resolution of the DCAA audit findings. Negotiations are complete and the termination settlement is scheduled for July 2007. The anticipated final disposition date is October 2007. An interview with the Goddard Audit Liaison Representative (ALR) indicates that schedule is on track to close the audits by the stated date.
A discussion with the Goddard ALR for GAO and OIG issues indicated that these types of audits are monitored and resolved adequately. The ALR also indicated that all open OIG recommendations are monitored in an independent manual system outside of the Corrective Action Tracking System (CATS) to ensure that a backup file is maintained. The ALR requests closure approval from the OIG; validates that all actions are completed; and documents the files with copies of any supporting documentation and other proof as appropriate. The OIG usually requests copies of these documents prior to approving closure. No issues were found in this area.
The last procurement survey revealed that Goddard Procurement had a number of DCAA system level audits that were not monitored in the CATS II System. A consideration was issued to implement a process to track and monitor system level audits on a quarterly or semi-annual basis. The ALR for DCAA audits indicated that development of a vehicle to monitor system audits would be useful but stated that the process has not yet been implemented. It appears that a new person has responsibility for this function since the last survey and was not aware of the survey consideration.
CONSIDERATION:
Goddard Procurement should implement a process to track and monitor system level audits on a quarterly or semi-annual basis to ensure that they are closed in a timely manner and that closure notices are sent to the appropriate Agency. The process does not have to be overly complicated or burdensome and can be achieved using an Excel spreadsheet. (REPEAT FINDING)
Numerous contract files were reviewed for adequacy of cost and price analysis. The scope of the review included competitive, noncompetitive, fixed price and cost reimbursable contracts.
DCAA audits were cited as justification for accepting proposed rates or factors in many of the files reviewed. Often DCAA was cited in the PPM as the sole rationale for accepting rates in the development of the negotiation position. Rarely was an independent analysis of the proposed rates conducted by Goddard Procurement. On occasions, the contract specialist included independent source data such as ‘Salary.com’ direct labor rate information in the file but failed to trace those rates back to the proposal and document the reasonability of proposed rates. Some smaller dollar value procurements (i.e., SBIRs) referenced an audit in the PPM; however, a copy of the audit was not found in the file. Some files contained a copy of the audit but the PPM did not always disposition the audit finding with adequate rationale. The DCAA audit is an advisory document and the contracting officer is still responsible for dispositioning the results.
The files reviewed appeared to rely heavily on the results of the DCAA audit during the cost analysis. On many occasions, the audit report was the only item included in the cost analysis area of the contract file. The PPM documentation primarily used the DCAA recommendations as a reason for acceptance of rates and factors. Additionally, the recommendations in the technical evaluation were used as the reason to accept proposed resources (labor, skill mix and ODC) even though most technical evaluations did not provide a detailed breakout of hours and skill mix time phased over the period of performance. Subcontractor data was rarely supported with cost or technical analysis. It appears that Goddard Procurement is performing research such as collecting Salary.com data to validate rates; however, the documentation does not always appear to reflect the results of that research.
Price analysis was not used to determine price reasonableness in any of the actions reviewed. Cost analysis appeared to be documented for all files reviewed, even fixed priced procurements. However none of the files reviewed included documentation to support price analysis. In one contract action, the DCAA indicated that the proposal was not adequate for certification of cost or pricing data or auditable. Based on that recommendation the price analysis should have been the appropriate tool to further validate the results of cost analysis.
Cost realism analysis did not appear to have traceability from the pricing report to the Source Evaluation Board (SEB) mission suitability findings. The cost and price section of the SEB report uses the technical evaluation as the reason for acceptance of resources; however, there is no requirement for a technical report other than the mission suitability findings and no separate technical report was found in the contract file. It was also noticed that the SEB normally used the proposed ceiling rates as probable rates. There was no evidence of analysis to support accepting the offerors’ proposed ceiling rates for probable cost purposes. The probable cost was heavily focused on DCAA audit results without an independent analysis. The DCAA is not always aware of the technical requirements or resources that may affect the development of rates; therefore, Goddard Procurement should ensure the DCAA results are valid, particularly after technical probable cost adjustments are made.
In all required situations, the requirement for cost and pricing data was appropriately identified in the solicitation document and the accompanying properly executed Certificate of Current Cost or Pricing Data was contained in the contract file.
CONSIDERATIONS:
1. Goddard Procurement should ensure that price analyses are performed on procurement actions when appropriate. Although price analysis is only required when cost or pricing data is required, this type of analysis is very effective in validating price reasonableness. Goddard Procurement should consider performing price analysis on actions where cost analysis may not suffice.
2. Goddard Procurement should consider establishing a process for the SEB to have traceability between the probable cost adjustments and the mission suitability findings. This will ensure that the all mission suitability findings result in appropriate probable cost adjustments. Also consider performing an analysis of proposed indirect rates and ceiling rates to determine whether probable cost adjustments up to the ceiling are appropriate. Further, consider requiring the contractor to provide previous contract actual experience on this subject as well as request DCAA to provide insight, if available.
3. Goddard Procurement should consider reviewing Cost and Price Analysis in future self assessments particularly since this area has had weaknesses in the past. The focus should be on reliance of cost or pricing data, utilization and disposition of DCAA audit findings, independent cost analysis of prime and subcontractors, utilization of price analysis and adequate documentation.
WEAKNESS:
The Goddard Procurement Officer should ensure that contracting personnel perform an independent cost analysis of the cost proposal rather than totally rely on the DCAA audit to satisfy the requirement for performing cost analysis. The audits are usually focused on rates and factors and do not perform cost analysis. They also do not perform a technical analysis so are not intimately familiar with the technical requirements. Requirements and associated resources (labor, materials, equipment, etc) may affect the development of rates, particularly if the procurement is sizeable. In order to develop adequate negotiation positions, the need for comprehensive cost analysis is required. This includes incorporating the DCAA results along with an independent critical cost analysis and should leverage from relevant in-house historical experience and/or legacy systems knowledge. (REPEAT FINDING)
4. Structured Fee Approach (NF 634):
Guidance for determining profit or fee objectives and the required use of Structured Approach to Profit/Fee Objective (NF 634) are provided in NFS 1815.404-470 and 1815.404-471. Files were reviewed for the appropriate use of and compliance with the requirements for NF 634. Written justification for the selection of the Assigned Weighting, Assigned Value, Weight Range, and Weight Designated percentages has improved since the last survey. However, the rationale provided still appeared to lack analytical evaluation of the procurement risk.
Goddard Procurement conducted a self assessment in 2007 and indicated that the majority of the contracts reviewed included written justification. The review revealed that the trend since the last procurement management survey is in the right direction and the attached NF 634 does include written justification. However, some files did not include sufficient rationale for using certain variables. For example, a couple of files used a factor of 0.5% under “other considerations” in the “Social Economic, Environmental, Public Policy Implementation” area; however, no rationale was provided why these points were allocated to the calculation. One file reviewed included a NF 634 that was developed in pencil. Utilization of manual forms could lead to errors.
CONSIDERATIONS:
1. Goddard Procurement should consider writing the justification in a manner that addresses performance risk as each area is documented. The Goddard Procurement review team should consider reviewing NF 634 analysis and documentation to ensure that the rationale addresses performance risk and the rationale adequately provides the basis for using the values/weights.
2. Goddard Procurement should ensure that the NF 634 is created electronically to facilitate performing the analysis electronically and avoiding potential errors.
1. Grants and Cooperative Agreements:
All files were reviewed for compliance with the Grant and Cooperative Agreement Handbook and the documentation requirements listed on the Goddard Procurement Grants Homepage. The award and administration of new grants and cooperative agreements transitioned to the NSSC effective May 1, 2006. Transition of grant processing to the NSSC appeared to be well managed by Goddard Procurement.
Files reviewed included a training grant, grants issued based on unsolicited proposals, and grants awarded as the result of NASA Research Announcements (NRAs). Technical evaluations lacked any detailed analysis of proposed costs. It is noted that the instructions in the Grants Handbook do not address documentation of results of the evaluation of proposals submitted in response to NRAs.
Negotiation memoranda reviewed lacked any detailed analysis of proposed cost (see note above); the form utilized to document negotiation objectives does not provide an area to complete an analysis of the proposed costs. The unsolicited proposals reviewed did not demonstrate an innovative and unique method, approach, or concept as required by FAR 15.607(a)(4). Rather than actual research proposals, the proposals provided funding for travel of 20 non-NASA individuals to attend a conference; funding for management of a history fellowship program; and funding for organizational logistics of a meeting respectively.
CONSIDERATION:
The Headquarters Sponsored Research Business Activity should ensure that the Grants Handbook is updated to identify procedures for documenting the review of the proposer’s estimated costs. The documentation should reflect information from cost proposals for NRAs and unsolicited proposals that is not submitted on the Budget Summary Form currently included in the Grants Handbook.
WEAKNESSES:
Goddard Procurement should ensure that all unsolicited proposals qualify for award in accordance with the FAR requirements.
Goddard Procurement should ensure that all grants files include a detailed analysis of the proposed costs.
Several simplified acquisition and commercial contract files were reviewed for compliance with the Simplified Acquisition and Commercial Item Procurement regulations. The review revealed that four different simplified acquisition checklists are used in the various Goddard Procurement offices. Two of the checklists referred to simplified acquisitions over $2,500 rather than $3,000. Many files reviewed did not include a checklist of any type and were not well organized. Miscellaneous correspondence was filed in various locations throughout the file.
Many sole source documents for actions under $100K did not contain the signature of either the technical or contracting representative. Many sole source justifications included poor rationale. Price analysis for noncompetitive awards did not contain evidence of published price lists, commercial catalogs, etc. Additionally, the Certification and Special Approvals for IFM Purchase Requisitions (NF1707) was often missing from files.
Central Contractor Registration (CCR) verification by contract specialists was often difficult to determine. The method of verification varied greatly. Some specialists included a copy of the CCR printout, some included a notation in the file, some checked the box on the checklist, and some files appeared to have no verification. One particular file included a CCR printout that contained white-out changes to match the information contained in the Contract Management Module (CMM).
Many files reviewed appeared to contain evidence of “creative contracting” techniques on simplified acquisition that are not necessarily consistent with the FAR and NFS regulations. The use of these “creative” techniques raises the question of whether contract specialists are receiving appropriate training/oversight. Examples include:
The previous survey documented a strength for the creation of the website for simplified acquisition processes. The website did not appear to provide any information or access to resources beyond information that is already available through Virtual Procurement Office or existing guidance for the preparation of sole source documents.
The previous survey recommended that Goddard Procurement establish a policy requiring a memorandum to the file with rationale justifying that the award of a BPA is in the best interest of the government based on FAR 13.303-2 – there is still no evidence of this documentation.
WEAKNESS:
Goddard Procurement should take steps to improve the overall quality of the simplified acquisition files, specifically in the areas of sole source justifications and price analysis. Additionally, Goddard Procurement should ensure that contracting personnel receive the necessary training and oversight regarding file documentation and analysis. (REPEAT FINDING)
A review of Interagency Acquisitions (IA) was completed with emphasis on ensuring conformance with the requirements in FAR 17.5 and NFS 1817.70. Of special concern in the review was ensuring that the proper determination and findings was completed for each action justifying that use of an IA is in the best interest of the Government, and that the supplies or services cannot be obtained as conveniently or economically by contracting directly for the same or similar supplies or services. Additional emphasis was placed on NFS requirements for proper review of IA actions and inclusion of appropriate terms and conditions in the resulting orders.
A high volume of IA actions are performed by Goddard Procurement, many of which are the result of the selection of a Government organization to perform scientific research through a NASA Research Announcement (NRA). For such IA actions, the NFS does not require the inclusion of a determination and findings document. These IAs were generally found to be properly completed by the offices supporting Goddard and Headquarters actions. Copies of the NRA selection statement and original proposal were included in the file to ensure clarity that the action was the result of an NRA selection. A standard format for the determination and findings was generally included in the file for IA actions that required such documentation. It is also noted that all IA actions included a set of standard terms and conditions that generally (although not always completely) addressed the terms and conditions required for inclusion in IAs as specified by the NFS. These standard sets of terms and conditions helped to ensure that important terms and conditions were not overlooked by the individual offices processing the actions.
Although it is noted that standard terms and conditions are utilized within the individual offices completing IA actions, this same standardization does not exist across Goddard Procurement. Standard terms and conditions for Headquarters IA actions are different than those for Goddard IA actions. For example, standard terms for Goddard awards include numerous specific provisions for new technology and patent rights as an attachment to the award document and provide a full description of technical monitor duties, where the Headquarters awards do not. Alternately, the Headquarters awards address property considerations and the Goddard awards do not. Many of the Headquarters awards reviewed contained orders that were not issued until several months after the beginning of the period of performance which increases vulnerability.
Justification for the utilization of the IA rather than an outside award action was generally lacking; it would seem logical that such research was generally the result of special experience or expertise that the agency could provide in the subject research area. The standard determination and findings format appeared to focus on a “de facto” completion of the document in other aspects. Specifically, there were files with the determination and findings document “checked” stating that work would be completed only by the Government agency; however, the statement of work or other correspondence associated with the order indicated that the work would be completed by an outside commercial organization. Additionally there was no evidence of a cost-breakdown for the effort completed by the outside organization. For these instances and others involving effort conducted by the government organization, there was no evidence of any cost analysis or discussion, or as mentioned above any technical consideration requiring the need to contract with the agency.
A review was completed of GSA schedule awards. Each of the actions chosen for review involved a situation where a follow-on incumbent contractor was competing to provide support through a GSA schedule against other schedule holders. In two of the three situations reviewed, the other contractors determined to “no-bid” the requirement. In the third, although competitive proposals were received, the adequacy of the evaluation, which ultimately resulted in the selection of the incumbent contractor, raised concern. Specifically, a second contractor, which was originally evaluated as “excellent” was subsequently reevaluated via an e-mail correspondence as only “very good.” The evaluation stated that the incumbent’s costs were higher than other offerors’ but not high enough to justify award to another offeror; no actual pricing information was provided beyond a listing of proposed rates for all offerors, and there was no application of assumed hours or total anticipated cost by offeror. It is noted that in two of the three circumstances reviewed, review by higher level procurement personnel and legal office of the selection decision were waived due to “non-substantial change in the RFP.” However, this type of waiver does not address the need to ensure that competition issues were adequately addressed for the procurement.
STRENGTH:
Goddard Procurement is commended for developing standard formats for IAs and determination and findings documents within the individual offices.
CONSIDERATIONS:
1. Goddard Procurement should standardize IA documentation across the entire organization rather than within individual offices. Review at a level above the contracting officer completing the action could help to achieve standardization of documentation and would seem prudent as a risk management tool given the size of many of the IA actions. Goddard Procurement should also consider working with Headquarters Offices to ensure more timely receipt of IA purchase requests to minimize the need to place orders after the initiation of performance by the agency. Goddard Procurement should also comply with the requirement that all IAs be completed on NASA form 523 (vs. Optional Form 347).
2. Goddard Procurement should re-access the procedures used to ensure that adequate competition is obtained for GSA orders.
WEAKNESSES:
1. Goddard Procurement should ensure that sufficient rationale is provided in the determination and finding to justify placement of the order in lieu of an outside award for the required services. Additionally, sufficient cost analysis should be performed for effort proposed by the government agency and cost-breakdown of any effort proposed by outside organizations should be provided for analysis and review.
2. Goddard Procurement should ensure that determination and findings are reviewed by legal counsel as required by the NFS. It is particularly important that reviews are conducted at a level above the cognizant contracting officer, especially with the large number of high dollar value requirements and the potential for work performed by organizations (outside contractors) other than the servicing Agency.
The survey focused on a review of the internal controls, training, number of card holders, SAP bankcard module automated system, monthly reconciliation, delegations exceeding $3,000, violations, and limits (individual/ monthly) placed on card holders pursuant to the government bankcard regulations.
There are currently a total of 644 NASA purchase card holders and 245 approving officials at Goddard. The number of purchase card holders remained consistent; there were a total of 645 in 2005. Each purchase card holder has an approving official and an alternate approving official in the event one of them is unavailable.
The same Center Purchase Card Program Coordinator (CAPC) has managed the bankcard program since 1998, this continuity provides the Center with invaluable corporate knowledge and consistency with the purchase card program processes. The CAPC and Alternate are both knowledgeable of the monitoring and administration duties of the purchase card program recently updated in the NASA Purchase Card Internal Control and Audit Plan dated April 2007 and PIC 03-20 dated October 2003.
All cardholders and approving officials complete all required training in SATERN prior to making purchases or approving any transactions. The CAPC monitors and tracks existing training records/requirements through an excel database. PIC 03-20 requires recertification of all cardholders and approving officials every three (3) years. Currently, approximately 80% of the existing cardholders and approving officials are scheduled to complete refresher training; it was noted that the required refresher training is planned in September 2007. The required purchase card training is implemented and monitored within SATERN. Most recently, copies of rules and updated regulations were routed to all cardholders and approving officials with new delegation letters that updated the micro-purchase threshold.
There are a limited number of individuals authorized to place individual bankcard purchases above the micro purchase threshold: five (5) warranted Contracting Officers/Small Purchase Agents within Goddard Procurement ($5K), five (5) Associate Division Chiefs/Contracting Officers for emergency purposes ($25K), two (2) non-warranted individuals ($5K), and one (1) Health & Safety office individual ($100K). All cardholders completed the necessary Simplified Acquisition training.
The CAPC performs a random overview of the system to identify any inappropriate purchase card activities during the monthly reconciliation process performed in the SAP Bankcard module. A three (3) strike policy is utilized for purchase card violations. In each instance, the violators are held accountable and the approving official/supervisor is notified of the violation(s). If deemed appropriate, the individual loses all purchase card privileges and must be recertified. Additionally, the CAPC continuously reviews inactive accounts; these accounts are either suspended or closed in their entirety. In conclusion, all current purchase card policies, procedures, and records regarding the purchase card program were reviewed and found to be extremely well managed and documented.
STRENGTH:
Goddard Procurement is commended for performing an internal desk audit of all existing purchase card cardholders, this audit reviewed/verified several transactions, receipts, and supporting documentation that occurred over the past three (3) years. Also, the CAPC and alternate are available to program users through the Center purchase card phone help-line to discuss problems, provide guidance, and provide clarification on a variety of issues. In addition, Goddard Procurement provides an outstanding website that contains extensive information such as the user’s guide, instructions on specific bankcard issues, responsibilities, and many other areas.
CONSIDERATION:
The Goddard Procurement Officer should carefully monitor the re-certification program and take the necessary steps to ensure that all required purchase card holders complete the refresher training in a timely manner. (REPEAT FINDING)
5. Construction and Architect and Engineer (A&E) Contracts:
Construction
Contract file review included construction efforts under IDIQ contracts and single discrete construction projects. Awards were made as a result of both sealed bids and negotiated procurements. Files reviewed contained for the most part the appropriate clauses, complied with bonding requirements, safety plans, invoices, certified payrolls, and support documentation requirements.
Safety Plans were included in the contract files but no evidence that the plans were reviewed by the Safety Office for approval was found. The Safety Plans were not incorporated into the contracts by modification. It is noted that the safety plan in two contracts reviewed contained an Attachment J annotated “Reserved for Safety and Health Plan to be incorporated at contract award” however no subsequent modification incorporating the plan was located. Another contract contained an email stating “revised safety plan is acceptable and contractor must submit safety plan to Contracting Officer for approval within 10 calendar days after Notice to Proceed”. There was no evidence that the contractor resubmitted the plan to the contracting officer nor was there evidence that any action was taken to incorporate the revised safety plan into the contract. Several contracting officers indicated during discussions that when the safety plan was submitted with solicitation/proposal it was understood that it was also incorporated into the resulting contract. The Attachment J reference was explained as an oversight. It is noted that only one contract reviewed incorporated the safety plan by modification and this contract should be recognized as exceptional since it exceeded the requirements regarding proper documentation in all areas. (NNG05AZ25C).
The review revealed that contracts adhered to the performance and payment bonds in accordance with FAR 28.102-1 securing the dollar amount of the contract value. Some files contained modifications increasing the total contract value but contained no documentation to reflect that the bonds securing the increased contract amount were revised.
CONSIDERATIONS:
1. Goddard Procurement should ensure that Safety Plans are coordinated and documented with approval from the Safety Office. Safety Plans should be incorporated into the contracts by modification after award and before Notice to Proceed is issued.
2. Goddard Procurement should extend the bond coverage if the period of performance is extended in accordance with the modified contract.
WEAKNESS:
Goddard Procurement should ensure that performance and payment bonds are increased if the total contract value is increased.
Architect and Engineer (A&E) Contracting
The majority of contract files reviewed contained the appropriate clauses. Most of the files contained copies of the A&E Selection Statement and the recommendation of the A&E Board. Only one file reviewed did not contain the required FAR Clause 52.244.4, Subcontractors and Outside Associates & Consultants (A&E Services) (Aug 98), which is found in all A&E contracts. It is noted that while most files contained a Safety Plan, only one file incorporated the safety plan by modification. Contract files did not provide any documentation with evidence of review and approval from the Safety Office. Also, only one file reviewed contained a Safety Plan. As was noted for construction contracts, several contracting officers indicated during discussions that when the safety plan was submitted with solicitation/proposal it was understood that it was also incorporated into the resulting contract.
STRENGTH:
Goddard Procurement is commended for its contract administration efforts on A&E contracts. Contract administration files were well organized and contained detailed government estimates, technical evaluations, and price negotiations memorandums.
CONSIDERATIONS:
1. Goddard Procurement should ensure that Safety Plans should be coordinated and documented with approval from the Safety Office. Safety Plans should be incorporated into the contracts by modification after award and before Notice to Proceed is issued.
2. Goddard Procurement should ensure that all proper FAR clauses for A&E are included in the solicitation/contract.
A number of contracts were reviewed for the inclusion of required environmental clauses. In accordance with PIC 01-27 and NASA Procedural Requirement (NPR) 8530.1 under Section 3.2.2, the NF 1707 – “Certification and Special Approvals for IFM Purchase Requisitions”, should be completed and included in the Procurement Requisition (PR) to indicate whether the items being purchased are on the EPA’s Comprehensive Procurement Guideline Lists. NF 1707 covers three (3) required certifications for all new acquisitions which include: NPR 7120.5, "NASA Program and Project Management Progress and Requirements”, "Electronic and Information Technology (EIT) Accessibility Compliance," and "Affirmative Procurement – Environmentally Preferable Products".
The majority of the contract files reviewed contained a completed NF 1707. A few of the contract files reviewed lacked the required documentation. Compliance with the requirement for utilization of the NF 1707 appears to be inconsistent and is not clearly understood by the technical and procurement organizations. Although only a few contract files did not contain the NF 1707, this is a concern since the NF 1707 is now a multi-purpose form that contains other required certifications.
All contracts reviewed contained one or more of the following environmental clauses: 52.223-3 Hazardous Material Identification and Material Safety Data, 52.223-5 Pollution Prevention and Right to Know Information, 52.223-9 Estimate of Percentage of Recovered Material Content for EPA-Designated Products, 52.223-10 Waste Reduction Program, 52.223-11 Ozone Depleting Substances, or 52.223-12 Refrigeration Equipment and Air Conditioners. None of the files involved the acquisition of EPA-designated products not meeting applicable minimum recovered material content recommended by EPA guidelines. In general, all of the contract files included the appropriate environmental clauses.
CONSIDERATIONS:
1. Goddard Procurement should ensure that the NF 1707 is completed and maintained in the contract files. The Simplified Acquisition Contract Checklist contains a box to annotate the presence of the NF 1707.
2. Headquarters Office of Procurement should consider amending the NF 1098 - Checklist for Contract Award File Content to include the requirement for the NF 1707 documentation.
7. Small Business Innovative Research (SBIR) Awards:
A review of files noted that the required contract clauses were included per the NASA FAR Supplement requirements. Contracts were generally well documented and complete. Special attention was paid to the policy aspects associated with SBIR/STTR awards.
It is noted that SBIR/STTR procurements, similar to NRAs or Announcements of Opportunity awards, are evaluated and selected by personnel outside of the procurement organization. However, unlike these Broad Agency Announcement areas, there is no similar policy guidance in the NFS outlining evaluation, selection, and debriefing policies for SBIRs/STTRs. Responsibility for the development of policy guidance for SBIR and STTR requirements resides at the NSSC.
CONSIDERATION:
The NASA Headquarters Office of Procurement should research and resolve any ambiguity regarding the roles and responsibilities for the development of policy guidance for SBIRs/STTRs.
8. Metrics Review and Documentation:
Goddard Procurement collects an array of management reports to ensure that the organization as a whole is functioning at a low level of risk. The organization grades itself on the reported data using red, yellow and green indicators as an objective way to define organizational performance. The organization also collects metrics on workload for major acquisitions and other actions to ensure schedules are met. The metrics data that are collected are reported to the Management Operations Board of Directors (MODBOD) and to the Goddard Management Council. The following metrics are collected: procurement schedules for major acquisitions, UCAs, workforce stability numbers, other actions (which includes modifications on major programs), and schedule information on Performance Evaluation Boards. The schedule information on Performance Evaluation Boards is a new metric that was added since the previous survey.
The ULO tracking mechanism discussed in the previous survey has not been fully implemented. However, the Office of the Chief Financial Officer in conjunction with Goddard Procurement and the resources community are placing greater emphasis on the tracking of ULOs. Several tools are used to track ULOs particularly one tool that currently tracks them by procurement line item.9. Homeland Security Policy Directive (HSPD) -12 Compliance:
Goddard Procurement initiated a proactive stance in the implementation of the requirements of HSPD-12. Actions completed included briefings to both its contractor and COTR communities. Additionally, Goddard Procurement developed separate standard Center unique clauses to address implementation of HSPD-12 for contracts supporting Goddard and Headquarters requirements. Also, by using its existing Goddard “List” System, a comprehensive list of on-site contractors requiring access to Center facilities was developed and used as a reference point for the incorporation of necessary HSPD-12 contract clauses in contracts by the required January 31st Headquarters Office of Procurement mandated deadline.
Although much has been done to satisfy the HSPD-12 requirements, several problems still exist, and all contracts were not modified in a timely manner. Despite the mandate that all impacted contracts be updated by January 31, 2007, there was evidence that the required changes to the contracts were not completed until several months after the required date. Non-compliance occurred despite the assurance from Goddard Procurement (as well as other NASA Centers) that all modifications were completed within the timeframe.
Additional dialogue between Headquarters and the Centers prior to the establishment of compliance deadlines regarding implementation may have helped to create an improved HSPD-12 clause and provided a more reasonable timeframe with which to meet the requirements.
It is clear from reviews of contract work requirements in the statements of work, or the presence of Goddard Clause 52.204-99 “On-site Contractor Personnel – Identification, Reporting and Checkout Procedures” in contracts that HSPD-12 clauses may need to be incorporated in some contracts not previously identified. It is noted that use of the “List” system does not provide a comprehensive identification of all contracts subject to HSPD-12. The system does not identify contracts that only require access to information systems rather than physical access to the facility.
STRENGTH:
Goddard Procurement is commended on its proactive approach in working with the Goddard Security Office and other offices to implement the HSPD-12 contract requirements for onsite contractors and for working proactively with the Center and Contractor community to highlight the need to complete Personal Identity Verification (PIV) badging requirements.
CONSIDERATION:
1. Goddard Procurement should reassess the requirement for incorporation of HSPD-12 requirements for all contracts whether or not initially determined to include on-site Goddard performance per the LIST system. Special consideration should be taken to address contracts that require contractor access to information systems.
2. Headquarters Office of Procurement and Office of Security and Program Protection (as necessary) should take more proactive measures to work with Goddard Procurement (and other Centers) to discuss policy implementation requirements such as HSPD-12 prior to their unilateral imposition.
10. Documentation for Awards Resulting from Broad Agency Announcements:
Contract files that resulted from NRAs were reviewed with emphasis on contract file organization, content, and adequacy of documentation. All contract files reviewed contained NF1098 – “Checklist for contract Award File Content”. All the files reviewed were appropriately labeled with numbered tabs corresponding to individual items listed on the forms.
The resultant contracts were well-documented and included all necessary supporting documentation in the files. There were no substantial anomalies noted. Goddard Procurement did a great job ensuring that the acquisition packages were completed, announcements were synopsized, successful offer's proposal including representations and certifications were maintained, price negotiation memorandums were prepared, and COTR delegations were executed.
11. Contract Management Module (CMM) Issues:
CMM received mixed reviews during the course of the survey. Some users reported positive experiences with CMM, while most reported frustration and increased stress. Major drivers of the stress are the format of documents generated by the CMM, numerous and complex “work-arounds”, and the increased time to complete procurement actions due to system limitations or learning curve adjustments. CMM has directly impacted Goddard Procurement managers’ ability to measure and assess their workload activities and it impedes their ability to effectively assign and manage work. Workload reports are either not available or difficult to run, and tend to produce inaccurate information. Documents generated in CMM are perceived as unprofessional in appearance, requiring extensive re-formatting outside of the system prior to release (i.e., paragraphs do not line-up correctly; clauses are not located in the appropriate contract sections; forms do not populate correctly; and numbers in columns do not line up). Examples of formatting issues were furnished to the CMM support staff, Competency Center, and the survey team.
From the individual user’s perspective, the transition to CMM has not been easy. The number of steps required to complete simple actions such as incremental funding have increased in volume and complexity. Once acclimated to the system, it is reported as relatively easy to use. Training is a continuing challenge, particularly in situations where users have workloads that do not require frequent utilization of the system. Most employees reported they have not initiated solicitations within CMM yet, but have heard second-hand that it is a difficult process and that the documents must be reformatted extensively. One user reported that they were told not to use CMM to generate their solicitation, but had to create the document outside of the system. Users who have attempted to generate solicitations and contracts reported frustration at not being able to locate local clauses and prescriptions within CMM.
The process for generating CMM documents is reported as more time consuming than previously experienced. Goddard Procurement personnel previously used the “Auto Word” system. Auto Word provided a menu for the user to select the contract type, produced a clause listing showing available clauses, and stated whether they are mandatory or optional, for that type of contract. Auto Word also contained local prescriptions for use, generated solicitation and contract documents, and produced a checklist to assist in the conduct of file reviews. CMM does not contain this functionality and is seen as more cumbersome to use, requiring additional administrative effort. As a result, some users reported generating their initial documents in CMM and downloading them to a Microsoft Word document to make changes to “clean it up”. Documents edited in this manner are not re-entered into the CMM system. Other users reported creating documents exclusively outside of the CMM system using Auto Word or previous contracts and modifications as templates, creating a “shell” within CMM, then uploading the document(s) as attachment(s).
Pressure to accomplish the day-to-day mission is primary in the minds of procurement personnel. By-passing CMM processes and procedures in favor of locally developed solutions can have a detrimental impact across NASA. For example, four BPAs reviewed were located in CMM, but do not exist in SAP. One task order awarded as a BPA in CMM was created as a “TO/DO Other” in SAP, and the external award did not track back to the BPAs. These issues create problems tracking financial obligations on contracts. If the “TO/DO” is not linked to the basic award, the agency is potentially overstating contract value in the system. It also creates confusion regarding where to post costs and disbursements. Users are reminded to create outline agreements in SAP; that Task Orders and Calls created under the outline agreement/BPA/Contract be created as “TO/DO NASA” rather than “TO/DO Other” for all orders under NASA contracts.
Initial CMM training was completed with 162 people. The training was generic on the PRISM Software and not conducted in the integrated PRISM-SAP environment. Numerous Service Requests (SRs) were applied to the CMM system immediately following go-live, during the stabilization period. The generic nature of the training combined with changes in the look and feel of the system increased user frustration and hindered familiarization. New employees hired after the initial training period have not had the opportunity or awareness of training on the CMM. The Procurement System Support Team (PSST) provides one-on-one hands-on sessions to work with individual users. Feedback on the support provided by the PSST is overwhelmingly positive. Contract specialists and contracting officers specifically acknowledged the assistance provided by Steve Lloyd, Terry Keane, Nancy Lockard, and Wanda Benhke. A tip sheet was developed, circulated, and periodically updated to identify ways to work more effectively within the CMM system.
The purpose for the Office of Small Business Programs (OSBP) participation in this Procurement Management Survey is to conduct a detailed review of Goddard’s small business program to ensure that it has adequate processes and that these processes are fully implemented to meet the goals established by the Agency and the Center. The review also identifies any issues that may prevent a small business at the Center from accomplishing its mission. The review consists of interviews, contract file reviews, metrics analysis and covers socio-economic goals, procurement planning, subcontracting, award fees, reporting, outreach programs, and training programs. Particular attention is focused on the various categories of small businesses (SB), small disadvantaged businesses (SDB), woman-owned small businesses (WOSB), veteran-owned small businesses (VOSB) and service-disabled veteran-owned small businesses (SDVOSB), Historically Underutilized Business Zone small businesses (HUBZone), and minority serving institutions(MSI).
2. Organizational Structure and Staffing:
The Goddard Small Business Office / Industry Assistance Office organization is an integral part of the Center’s business activities. This office consists of four people. It is headed by the Small Business Specialist who resides in the Procurement Operations Division and reports directly to the Procurement Officer. This position is assisted by three people: the Headquarters Small Business Specialist, the Goddard Small Business Specialist, and a Program Support Assistant. All three positions comprise the Industry Assistance Office (IAO).
Personnel within the IAO are cross-trained and the specialists are able to perform either Headquarters or Goddard workload responsibilities. As a result, the division between Headquarters and Goddard responsibilities noted in previous surveys is diminishing. The IAO staff is now more knowledgeable about all areas of Goddard procurement. One of the important tools within the IAO is a written set of Standard Operating Procedures (SOP). The SOPs allow the specialists to ensure that each process is accomplished correctly. As procedures change, the SOP book is updated to reflect those changes.
3. Industry Assistance Priorities:
The IAO is responsible for the implementation and management of all aspects of the Small Business Program, in accordance with the FAR, NFS, and as directed by the Assistant Administrator for NASA’s OSBP. Programmatic priorities include (1) counseling both large and small firms interested in conducting business with Goddard and providing advice regarding the importance of teaming relationships; (2) engaging in acquisition planning, analyzing statements of work, and providing guidance on small business participation in NASA business opportunities; (3) monitoring periodic progress and annual results of the agency’s and prime contractors’ achievement of negotiated socioeconomic business goals; (4) providing oversight of prime contractors’ subcontracting programs, to ensure compliance; and (5) participating in a broad range of outreach activities.
4. Center Prime Contractor Socioeconomic Goals:
In FY2006, Goddard achieved three of six NASA assigned goals and four of six congressionally-mandated goals. Combined Headquarters and Goddard procurements exceeded the congressionally mandated small business goal of 23% by achieving a 31.5% award rate in that category. Additionally, Goddard procurements exceeded the congressionally mandated-goals in the following categories: 1) SDVOSB goal of 3% - actual awards totaled 3.18%; 2) SDB goal of 5% - actual awards totaled 16.47%; 3) WOSB goal of 5% - actual awards totaled 5.51%.
Goddard and Headquarters Figures for Prime Contracts
| FY 2006 Actuals | FY 2007 Target | FY 2007 Actuals Through June 30, 2007 |
|
|---|---|---|---|
| Total Dollars | $2,039,173,352 |
$1,934,000,000 |
$1,494,276,991 |
| SB $ | $642,174,342 |
$617,900,000 |
$394,406,821 |
| SB % | 31.49% |
31.59% |
26.39% |
| HUBZone SB $ | $3,593,982 |
$4,700,000 |
$1,700,073 |
| HUBZone SB % | 0.18% |
0.24% |
0.11% |
| SDB $ | $335,859,280 |
$368,300,000 |
$230,114,222 |
| SDB % | 16.47% |
14.0% |
15.4% |
| WOSB $ | $112,403,038 |
$114,700,000 |
$45,624,021 |
| WOSB % | 5.51% |
5.9% |
3.05% |
| SDVOSB $ | $64,781,674 |
$2,400,000 |
$41,526,899 |
| SDVOSB % | 3.18% |
0.12% |
2.78% |
| HBCU/MI $ | $0 |
$1,360,000 |
$0 |
| HBCU/MI % | 0.0% |
0.07% |
0.0% |
5. Center Subcontractor Socioeconomic Goals:
The subcontracting award numbers are a combination of Goddard and Headquarters procurements. Goddard manages in excess of 180 subcontracting plans. The figures were obtained from the Electronic Subcontracting Reporting System (eSRS), a relatively new federally mandated database. Initial start-up and data conversion problems were experienced across the government. The numbers reflected below show a large increase in subcontracting dollars reported. This could be a result of inadvertent reporting of second and third tier subcontractor data or possibly data from contracts at other NASA Centers. The issue was reported to the Headquarters OSBP for resolution. Until resolution, no conclusions can be made from the figures reported.
Goddard and Headquarters Figures for Subcontracts Through March 31, 2007
| FY 2006 Actuals | FY 2007 Target | FY 2007 Actuals
|
|
|---|---|---|---|
| Total Dollars | $701,776,747 |
$660,000,000 |
$589,867,946 |
| SB $ | $252,230,781 |
$231,000,000 |
$180,525,792 |
| SB % | 35.9% |
35% |
30.5% |
| HUBZone SB $ | $7,462,301 |
$8,500,000 |
$1,778,131 |
| HUBZone SB % | 1.6% |
1.3% |
0.3% |
| SDB $ | $144,578,740 |
$81,200,000 |
$58,328,642 |
| SDB % | 20.6% |
12.3% |
9.9% |
| WOSB $ | $33,103,896 |
$23,300,000 |
$24,86,603 |
| WOSB % | 4.7% |
3.5% |
4.9% |
| SDVOSB $ | $14,900,000 |
$1,500,000 |
$327,595 |
| SDVOSB % | 2.1% |
0.23% |
0.06% |
| VOSB $ | $29,089,036 |
$1,500,000 |
$12,932,082 |
| VOSB % | 4.2% |
0.23% |
2.2% |
| HBCU/MI $ | $2,352,447 |
$1,600,000 |
$0 |
| HBCU/MI % | 0.3% |
0.24% |
0.0% |
The duties of the small business specialists are documented in a monthly report to the Headquarters OSBP. Each small business specialist documents specific actions such as outreach, counseling, PR review, travel, protests, and telephone calls in this report.
Uniform Methodology for Determining Small Disadvantaged Business Goals
NPD 5000.2A – is used to establish a uniform method for determining the small business goals incorporated into solicitations for full and open competition valued at $50M or more. The methodology entails a review of the solicitation’s statement of work to identify areas with subcontracting opportunities; review of the subcontracting history of the contract if the solicitation is for a follow-on requirement and/or review of the subcontracting history of similar contracts; market research to assess the availability of small businesses with the capability to perform the effort in the statement of work requirements; and a determination of the percentage of the effort that can be subcontracted to small businesses. The goals established are incorporated into the solicitation and the awarded contract and are tracked both on a six month and cumulative basis.
The following programs used the NPD 5000.2A methodology during this survey period and no issues were found:
2007 Tracking and Data Relay Satellite (TDRS) K Program Space Communications Networks Services Program
Geostationary Operational Environmental Satellites (GOES)
2006 Geostationary Lighting Mapper (GLM)
Landsat
Construction of Space Science Building
Environmental Test & Integration (ET&I) support Services
Small Business Administration (SBA) Procurement Center Representative (PCR):
An SBA PCR is assigned to Goddard and is also responsible for four other agencies in the area. The PCR has resident offices within the Goddard facility; however, time is divided between the agencies as workload permits. The PCR reviews all procurement requests over $100,000 and coordinates comments with the small business specialists. The PCR also reviews all subcontracting plans and provides recommendations to the small business specialists. Issues are addressed with the contracting officer for resolution during negotiations.
Center Small Business Technical Advisor
Goddard has an assigned Small Business Technical Advisor, however, this individual was unavailable for an interview during the survey.
Subcontracting plans are required for prime contracts over $550,000 unless the contract is awarded to a small business. Plans are sent to the small business specialists for review. Any omissions or problems with the plans are worked until they are approved by the specialists, PCR and contracting officer. Copies of approved subcontracting plans are maintained in the IAO and performance is monitored on a bi-annual basis. The plans are reviewed in accordance with FAR 19.704 and 52.219-9. During this survey, five subcontracting plans were reviewed. Two of the plans contained initial deficiencies that were later corrected. The typical deficiencies include failure to meet one of the eleven elements required by the FAR.
The challenging part of approving any subcontracting plan is developing realistic goals that the prime contractor can achieve while working to attain the agency goals established by SBA. The goals established in the contracts reviewed ranged from 26% to 83.33%. Goddard seems to have the appropriate level of experience and knowledge of industry capability to establish realistic yet challenging goals.
3. Award Fee/Incentive Fee Contracts:
The small business specialists provide input into award fee determinations. The Performance Evaluation Board (PEB) requests a written evaluation of the contractor’s performance in achieving its small business and SDB subcontracting goals including an adjectival score of either: excellent, very good, good, satisfactory, poor or unsatisfactory.
Goddard has a very strong process for awarding set-aside contracts. A Review of Acquisition Package (PR Review) form is completed for all procurements over $100,000. The small business specialists provide a set aside recommendation, NAICS code, size standard, contractor source list, and justification if no set-aside is recommended to the contracting officer. The form also contains information regarding subcontracting plan requirements as appropriate.
The small business specialists sign the form and forward it to the contracting officer for concurrence. It is also then reviewed at one level above the contracting officer and by the PCR.
Small Business Set-Asides
Small business set-asides are used when at least two or more small businesses possess the capabilities to perform the work required on a contract. During the last two years, Goddard awarded a total of 20 contracts as small business set-asides.
| Fiscal Year | SB Set-Asides Awarded | SBIRs Awarded | STTRs Awarded |
|---|---|---|---|
3 |
1 |
0 |
|
16 |
0 |
0 |
SBA Section 8(a) Program:
Goddard awards contracts to 8(a) firms whenever possible. The Section 8(a) awards for each of the last two fiscal years are provided in the following table:
| Fiscal Year | SBA Section 8(a) Contracts Awarded |
|---|---|
10 |
|
The IAO conducts an aggressive review of all Individual Subcontracting Reports (ISR) and rejects any reports that contain discrepancies. Additionally, the small business specialists conduct one-on-one training with the prime contractors to explain the new eSRS reporting system. The eSRS system creates challenging problems for the small business specialists, however, none are insurmountable.
Small Business Forum Roundtable
The Small Business Forums are held on a quarterly basis. These IAO sponsored forums are normally hosted by two Goddard program managers, two Goddard prime contractors and the small business specialists. The Forums are free to the small businesses and limited to 50 participants in order to facilitate networking opportunities. The number of participants may be expanded in the future to 75. The Forums are well received and feedback is obtained via email.
The March 21, 2007 Small Business Forum Roundtable agenda included the Flight Programs & Project Director, Heliophysics and Program Analysis and Control II Divisions and the Flight Programs and Projects Division Business Office. The prime contractors represented included were Stinger Ghaffarion Technologies (SGT) and Bastion Technologies.
In addition to the Forums, Goddard IAO representatives also participated in:
Goddard has a comprehensive and robust counseling program. Small businesses are provided a counseling package at the beginning of each session. The package contains a “Marketing to NASA” guide, an organization chart, a list of the top ten NAICs codes used at the Center, a list of prime contractors, a list of small business contractors, and a checklist of topics addressed during the counseling session. The small business specialists identify relevant contracting officer and program office points of contact to match the small business’ type of supplies or services. The small business specialists conduct follow-up sessions with each small business to ensure that the appropriate information and advice was provided to the small business and document any success stories involving contracts awarded to the small businesses.
In summary, the Goddard IAO processes and procedures are very effective in meeting the Agency and Center goals.
STRENGTHS:
Program Management: The Goddard IAO is commended for its utilization of a workload tracking form; it is an excellent way to document the accomplishments of the small business specialists.
PR Reviews: The Goddard IAO is commended for its PR review process for contracts over $100,000. The process allows the specialists, PCR and contracting officer to support small businesses participation.
Counseling Package: The Goddard IAO is commended for its utilization of a counseling package during sessions with small businesses. The packages contain detailed information regarding conducting business with Goddard which facilitates the promotion of maximum opportunities for small businesses. Additionally, soliciting feedback from the small businesses including negative feedback is an excellent tool.
Small Business Forum Roundtable: The Goddard IAO is commended for its very strong forum program with Program Management and prime contractor support. Low cost and small size events allow more one-on-one interaction and networking.
Subcontracting: The Goddard IAO maintains good records and has the ability to track subcontract performance despite the issues with eSRS. The review the subcontracting plans and contract files allow the specialists to validate the numbers, work with the prime contractors and correct issues before formal submission of the data into the eSRS system.
|
Owner: Yolande Harden |
Technical Support |
NASA Privacy Statement Section 508 Compliant | Last revised: April 16, 2008 |