Virtually every year, the Government Accountability Office’s (GAO’s) Bid Protest Annual Report includes “flawed technical evaluations” as one of the top five most common grounds for successful protests. Simply stated, this means that if a protest is to be sustained at the GAO, there is a good chance the Government watchdog will find that the agency failed to evaluate the protester’s and/or awardee’s technical proposal in accordance with the solicitation’s disclosed evaluation methodology. It follows, of course, that more complex evaluation schemes (i.e., those with a multiplicity of factors, sub-factors, and weighting systems) carry a commensurately higher level of risk that agency evaluators will get it wrong. The GAO’s recent decision in AT&T Mobility, LLC provides one such example and is a useful case study for contractors.

The balance between streamlining the evaluation process and adequately performing a qualitative analysis in a best-value tradeoff (BVTO) procurement is at the heart of this case. Here, the GAO sustained AT&T Mobility’s protest and found that the U.S. Secret Service relied upon a record devoid of any meaningful qualitative analysis under the technical factor, and instead evaluated offerors based upon a pass/fail matrix that had been included under an earlier version of the solicitation but which was removed by the Agency as the acquisition evolved. The Secret Service’s evaluation criteria included additional non-price factors (transition and corporate experience), which should have been evaluated on a BTVO basis but were also assessed on an acceptable/unacceptable basis. AT&T Mobility also contended that there were price calculation errors. Although the GAO acknowledged the price factor errors may have been harmless given AT&T Mobility would remain the higher-priced offeror, it concluded that due to the flawed evaluation of the non-price factors, there was the possibility AT&T Mobility was prejudiced by the errors. Ultimately, the GAO found that the Agency disregarded its stated BVTO evaluation scheme and, instead, awarded the contract on a lowest-priced, technically acceptable (LPTA) basis.

The decision highlights important considerations that contractors should take into account when determining whether to file a protest.

1. Consider the Scope of the Evaluation: The manner in which a solicitation is written can increase the likelihood of a successful protest. Complicated procedures are likely to produce uncertain results that may be worth investigating post-award. The solicitation in question contemplated a technical factor evaluation of the ability to meet 112 different core technical requirements. A BVTO evaluation assessing 112 substantive technical requirements is a logistical nightmare for an evaluation team, and it’s not surprising that the Secret Service got it wrong. While FAR Subpart 8.4 —which was utilized here—provides agencies with a simplified process for procuring certain commercial supplies and services, agencies are still required to evaluate pursuant to the solicitation’s stated criteria.Often, an agency’s desire to ensure that contractors are able to meet all of their technical requirements is reflected in the solicitation’s evaluation criteria. However, a solicitation that requires the evaluation of a lengthy list of technical requirements—and an attendant inter-factor tradeoff based on those evaluative results—can make it more difficult for the evaluation team to perform an adequate qualitative analysis of offerors’ proposals. Thus, when reviewing an agency’s award decision (as disclosed in a debriefing for FAR Part 15 procurements or a “brief explanation” of contract award for FAR Part 8 procurements), a disappointed contractor should carefully consider the depth and breadth of the disclosed evaluation criteria against the stated evaluation results and ask one central question: “Does this make sense?” If the answer is “no” or “likely no,” consider filing a protest.Remember, the devil on an agency’s shoulder may convince the evaluation team that the agency can have it all. The agency may genuinely believe that complex and lengthy evaluation criteria are necessary to fulfill existing and emerging requirements and that its evaluators will administer flawless evaluations. As with most things that seem too good to be true, by the time the difficulties become apparent, the agency may be left with options that are less than ideal. The agency may be forced to figure out how to pull off a Hail Mary, in which it attempts to evaluate offerors in accordance with complex and lengthy evaluation criteria while also meeting its deadlines and adequately documenting the record. When an agency is forced to choose between competing priorities such as meeting internal timelines and performing a sufficient qualitative analysis, the temptation to streamline in ways not contemplated by the solicitation may become too much for it to resist.
2. Pay Attention to Amended Requirements: When an agency amends a solicitation – and particularly when the agency overhauls its award methodology – contractors should ensure that the agency evaluated proposals in accordance with the final version of the solicitation. Here, the solicitation originally articulated a pass/fail matrix would be utilized. The final version of the solicitation removed the pass/fail evaluation matrix and stated the ability to meet 112 core requirements would be considered in the technical factor evaluation. The source selection decision noted that the Agency considered the pass/fail matrix in its decision and in the basis for award—a clear red flag that the Agency did not evaluate in accordance with the most recent version of the solicitation.Agency evaluations conducted pursuant to outdated solicitation requirements can be attributed to various causes. Responses to questions for clarification may amend the solicitation, but the evaluation team may not take the responses into account during the evaluation. Changes may be made by the contracting team but not adequately communicated to the evaluation team. There may be document version control issues, or there may be personnel turnover that leads to an institutional loss of knowledge. Contractors should thus look out for the use of outdated language (or explanations that rely on outdated requirements), as this may be an indication that the agency did not evaluate in accordance with the most recent version of the solicitation.
3. Understand Debriefing Documents: When reviewing debriefing documents, contractors should be cognizant of the acquisition procedures utilized and ensure that the information provided meets the minimum regulatory requirements. FAR Part 15 debriefings should detail: (1) any assessed significant weaknesses or deficiencies; (2) the overall technical rating of the successful and debriefed offeror; (3) the overall evaluated cost or price of the successful and debriefed offeror; (4) the overall ranking, if applicable; (5) a summary of the rationale for award; (6) for commercial products, the make and model to be delivered by the awardee; and (7) reasonable responses to relevant questions about whether source selection procedures contained in the solicitation, applicable regulations, and other applicable authorities were followed.In addition to FAR Part 15 procurements, the Defense Federal Acquisition Regulation Supplement (DFARS) requires enhanced debriefings for certain procurements, and FAR Subpart 8.4 requires a brief explanation of the award decision. Regardless of the operative regulations at issue, the debriefing documents should provide the contractor with the agency’s rationale for the award at some basic level. If the debriefing provides only conclusory and/or superficial information, the lack of detail may indicate that the agency failed to adequately perform and/or document its decision.Here, the GAO determined that the Agency did not adequately explain and document the basis for its determination. In making its award decision, the Agency was required to consider the underlying bases for the ratings and the advantages and disadvantages associated with the competing proposals. The Agency’s determination that the competing proposals were technically equivalent should have been based on a documented qualitative assessment of proposals, but one simply was nowhere to be found in the evaluative record.
4. Corrective Action: When a protest is sustained, the GAO typically recommends that the agency take corrective action. Here, the GAO gave the Agency the choice between (1) reevaluating the proposals in accordance with the current solicitation terms and (2) reassessing its requirements and method of source selection. The GAO also recommended the protestor be reimbursed for protest costs, including attorney’s fees.When determining whether to file a protest, consider the range of potential remedies at play. Protest arguments that, if successful, can be addressed by correcting only minor issues are far less likely to result in a change in the award decision, and therefore contractors may wish to forego the battle in the first place. That said, consider moving forward if the alleged deficiencies are more fundamental in nature and could conceivably alter the award decision once corrected.
5. A Note About Compliance: While not a key issue in this case, the GAO’s decision states that a third offeror was eliminated from competition due to compliance problems. Agencies often utilize minimum mandatory criteria as a way to reduce the number of proposals that must be evaluated, and this in turn allows the agency to streamline the evaluation process. Contractors should ensure their proposal demonstrates adherence to all affirmative compliance requirements and remember the old adage that “an ounce of prevention is worth a pound of cure” when developing their proposals. An agency is typically unforgiving when an offeror does not meet basic requirements. If a compliance requirement does not make sense, contractors should ask questions within the time period specified in the solicitation. Any ambiguities should be brought to the agency’s attention prior to the solicitation closing.

At bottom, this case provides yet another example of the potential utility of the protest process and further confirms that contractors should never be afraid to ask an agency to “show its work” when providing its final answer— i.e., the award decision—in a complex procurement. Remember, 1+1 never equals 3.