A robust vendor management scorecard tracks 15 KPIs across five dimensions: delivery (5 KPIs), commercial (3 KPIs), risk (3 KPIs), relationship (2 KPIs), and innovation (2 KPIs). Most scorecards over-weight delivery and under-measure commercial integrity and risk; the result is scorecards that look comprehensive but miss the dimensions where vendor relationships actually decay.
Why scorecard design matters.
The KPIs you choose decide what gets managed. Vendor management functions that score only delivery — SLA compliance, ticket volumes, change-success rate — manage delivery and lose the rest. Vendors learn quickly which dimensions are measured; the dimensions that aren't, drift.
The five-dimension scorecard described here is the structure we apply across our engagements. The dimensions are not arbitrary — each represents a category where vendor relationships have failed often enough to deserve a measured channel.
Dimension 1 — Delivery (5 KPIs).
1. SLA compliance rate.
Percentage of contracted SLAs met in the reporting period. The headline metric, but only useful when independently verified rather than vendor-reported. Without verification, this KPI measures vendor reporting, not vendor performance.
2. P1/P2 incident resolution time vs. SLA.
Median and 95th-percentile time-to-resolution for the highest-severity incidents. The 95th percentile matters more than the median; it captures the tail risk that medians hide.
3. Change-success rate.
Percentage of changes deployed successfully without rollback or post-deployment incident. Below 95% is a quality signal worth investigating; below 90% is structural.
4. First-time-fix rate.
Percentage of incidents resolved without re-opening or re-assignment. A leading indicator of skills and tooling depth on the vendor side.
5. Service-availability against contracted target.
For services with availability SLAs, actual availability measured at the architectural points the contract specifies — not just at the vendor's monitoring boundary.
Dimension 2 — Commercial (3 KPIs).
6. Run-rate vs. baseline.
Current annualized run-rate as a percentage of the original contract baseline, with cumulative change-request uplift broken out. This is the single most under-measured KPI in IT vendor management. Most scorecards track invoice accuracy but not run-rate drift; the drift is where commercial value erodes.
7. Benchmark variance.
Vendor's pricing as a percentage of current market benchmark for equivalent services. Refreshed annually for tier-1 vendors. Without this, "the contract is in line with what we agreed" is the only commercial truth available — which is not the same as "the contract is in line with the market."
8. Invoice accuracy.
Percentage of invoices reconciling without dispute against contracted run-rate plus approved change requests. Below 97% indicates an upstream commercial-discipline problem.
Dimension 3 — Risk (3 KPIs).
9. Open material risks (count and trajectory).
Number of risks at material-level severity in the seven-category register, with trajectory over the last six months. Static or growing material risks are themselves a signal.
10. Security-incident count.
Number of vendor-attributable security incidents in the reporting period. Includes vulnerability disclosures affecting the vendor's stack that have not yet been remediated within agreed windows.
11. Audit-finding closure rate.
For regulated industries: percentage of audit findings (internal or external) closed within agreed remediation windows. The single fastest-eroding KPI when governance attention shifts elsewhere.
Dimension 4 — Relationship (2 KPIs).
12. Stakeholder satisfaction (NPS-style).
Quarterly survey of buyer-side stakeholders on relationship health. Trend is more informative than absolute score; a steady decline of 5+ points over two quarters is a signal worth investigating.
13. Account-team stability.
Turnover of named vendor account team members over rolling 12 months. High turnover usually precedes service degradation by 3–6 months.
Dimension 5 — Innovation (2 KPIs).
14. Joint-roadmap commitments delivered.
For strategic vendors: percentage of agreed joint-roadmap commitments delivered against agreed timelines. Tracks whether the strategic relationship is actually producing strategic value, or has decayed to transactional service delivery.
15. Vendor-initiated value contributions.
Count of unsolicited vendor-initiated proposals that were accepted by the buyer in the reporting period. A leading indicator of the strategic vendor's engagement with the buyer's roadmap rather than just their service catalogue.
The KPIs we deliberately don't include.
Several KPIs that show up routinely in vendor scorecards do not tell you what they appear to:
- Ticket volume. Tells you something about activity; tells you nothing about value or quality.
- Average response time. Easily gamed by acknowledgement vs. resolution semantics. P95 of resolution time is the more honest metric.
- Customer satisfaction at ticket close. Almost always above 95%; stakeholders click "satisfied" to close the survey. Useful for detecting catastrophic problems; not useful for measuring quality.
- Vendor-self-reported availability with no verification. Measures the vendor's reporting, not the vendor's performance.
How often each KPI is reviewed.
| Cadence | KPIs |
|---|---|
| Weekly (Tier 1) | P1/P2 resolution time, change-success rate, SLA compliance, security-incident count |
| Monthly (Tier 2) | All 15 KPIs in summary form; trends across last 6 months |
| Quarterly (Tier 3) | Run-rate vs. baseline, benchmark variance, joint-roadmap delivery, stakeholder NPS, material-risk trajectory |
| Annually | Full benchmark refresh; account-team stability summary; relationship strategic review |
The thresholds that matter.
For a tier-1 strategic vendor, the action thresholds across the dimensions:
- SLA compliance below 97%: Tier 2 review and remediation plan.
- Run-rate above 110% of baseline (excluding approved CRs): Tier 3 commercial review.
- Benchmark variance above 8% adverse: structured renegotiation initiated.
- Open material risks growing for two consecutive quarters: executive sponsor escalation.
- Stakeholder NPS down more than 10 points QoQ: relationship-health review.
The Aventario perspective.
"The five-dimension scorecard is what separates managed vendor relationships from theatre. Most scorecards we inherit measure four delivery KPIs in great detail and miss commercial integrity entirely — which is exactly the dimension where vendor relationships actually break."
— Margit Györfi, CPO, Aventario
FAQ.
How many KPIs should a vendor scorecard track?
For strategic vendors, 15 across five dimensions is the structural answer. Fewer leaves blind spots in commercial, risk, or relationship dimensions. Substantially more produces fatigue and dilutes attention.
What is the most under-measured vendor KPI?
Run-rate vs. baseline, with cumulative change-request uplift broken out. Most scorecards measure invoice accuracy but not run-rate drift, which is where commercial value erodes over the contract life.
How often should KPI thresholds be reviewed?
Annually at minimum, and whenever there is a material change in the contracted scope or service. Static thresholds drift away from current reality; the scorecard becomes less informative over time without refresh.
