Playbook: How to Build an Agency RFP That Measures Decision Quality

Most RFPs answer one question: "What can this partner do?" But capabilities tell you nothing about decision quality. You can hire a partner with every capability on your checklist and still end up with analysis nobody acts on.

The problem isn't the partner. It's the evaluation architecture. When you evaluate for capabilities rather than decision quality, you select for the wrong thing. You get feature density. You do not get strategic clarity, accountable recommendations, or work that survives the handoff from procurement to execution.

Gartner research indicates that poor vendor selection accounts for 46% of project failures. The failure rarely traces back to the vendor. It traces back to the evaluation process that selected them.

This playbook restructures your agency evaluation around the metric that matters: will this partner improve the quality and velocity of your strategic decisions?

Phase 1: Map Your Decision Architecture (Week 1–2)

Define the decisions the partner will support before writing a single RFP line. Most organizations skip this — then wonder why the selected partner produces intelligence nobody acts on.

Decision Architecture Map

The Partner Role column forces specificity: "access to data" is not a role. "Synthesize market data into a go/no-go recommendation" is.

RACI for Evaluation

This prevents the most common governance failure: procurement owns the process, strategy owns the regret.

Phase 2: Replace Checklists With Decision-Quality Criteria (Week 3)

Standard RFPs use binary capability checkboxes. This reveals nothing about decision quality. Organizations using structured evaluation methods are 33% more likely to select the right vendor.

Decision Quality Scoring Rubric

Weight dimensions to match your Phase 1 map. If nobody owns the recommendations you already buy, weight accountability at 30%. If speed is critical, move speed-to-decision to 25%. Organizations that streamline procurement operations see a 42% increase in transformation success.

Phase 3: Run Structured Evaluation Gates (Week 4–6)

Without structured gates, the process drifts toward the most polished pitch.

Gate 1: Written Scoring (Week 4)

Evaluators score independently against the rubric before group discussion, preventing anchoring. Scores of 4–5 must cite response evidence; 1–2 require a deficiency citation.

Gate 2: Working Session, Not Pitch (Week 5)

Don't ask partners to present. Ask them to demonstrate: "Here is a strategic question we're evaluating. Walk us through your approach — what you'd gather, how you'd synthesize, what recommendation you'd produce."

Gate 3: Decision Quality Reference Checks (Week 6)

Replace "Were you happy?" with three questions:

Can't answer question one? No decision-quality output. Pause on question three? Accountability gap.

Gate 4: Governance Review

The committee presents the top-ranked partner(s) to the accountable executive with the completed rubric, reference findings, and a risk assessment. The accountable executive makes the final call — the person who owns the outcome owns the selection.

The Metric That Matters

Track one metric for 90 days: decisions enabled per engagement. Not outputs produced. Decisions enabled. If the partner can't identify specific strategic choices their work informed, the evaluation architecture failed — no matter how well they scored.