The problem with choosing an MSP isn't that the information isn't available — it's that you're comparing three proposals structured differently, weighted toward each provider's strengths, and delivered by salespeople who know how to make their offering look best. A scoring framework removes the salesmanship and lets you compare what actually matters.
The Five Scoring Categories
Category 1: SLA Quality (25% weight)
Score each provider 1–10 on:
- P1 response time (15 min = 10, 30 min = 8, 1 hour = 5, no defined SLA = 0)
- P2 response time (1 hour = 10, 2 hours = 8, 4 hours = 5, "best efforts" = 0)
- After-hours coverage quality (dedicated on-call staff = 10, monitored + callback = 6, voicemail = 0)
- SLA remedies (specific credit formula = 10, "we take it seriously" = 0)
Category 2: Security Stack (25% weight)
Score each provider on what's included in the base price:
- EDR (true EDR with MDR/monitoring = 10, EDR software only = 7, antivirus = 3, nothing = 0)
- Email security (advanced filtering + anti-phishing = 10, basic spam filter = 5, none = 0)
- Managed backup (tested restores included = 10, backup software only = 5, none = 0)
- MFA enforcement (policy + enforcement = 10, MFA available but optional = 5, not mentioned = 0)
- Patch management (SLA-backed patching = 10, best efforts = 5, no defined process = 0)
Category 3: Industry Experience (20% weight)
Score based on verifiable industry expertise:
- Number of clients in your specific industry (5+ = 10, 2–4 = 6, 0–1 = 2)
- Can name clients and provide references (yes = 10, no or "confidential" = 0)
- Compliance program for your framework (documented, specific = 10, "we handle HIPAA" = 5, not mentioned = 0)
- Industry-specific certifications or training (relevant = 10, generic = 5, none = 0)
Category 4: Total Cost of Ownership (20% weight)
Don't score on sticker price — score on total cost including add-ons:
- Calculate the all-in monthly cost including every required security component
- Score relative to the other proposals on a 1–10 scale (lowest cost = 10)
- Penalize proposals that move key components to extra-cost (deduct points for each necessary add-on)
- Include one-time onboarding/migration cost amortized over contract term
Category 5: Contract Terms (10% weight)
Score on contract flexibility and fairness:
- Contract length (12-month = 10, 24-month = 7, 36-month = 4)
- Auto-renewal notice requirement (30 days = 10, 60 days = 7, 90 days = 4)
- Performance exit clause (yes = 10, no = 0)
- IP/documentation ownership (client = 10, MSP = 0, unclear = 3)
Scoring the Reference Calls
After reference calls, adjust scores based on what you learned:
- If references confirm incident response is as described: keep SLA score
- If references describe SLA misses without remedies: drop SLA score 2 points
- If references describe high staff turnover affecting service: drop industry experience score 2 points
- If references can't describe specifics: neutral (don't adjust)
Interpreting the Results
A provider with the highest weighted score is usually the right choice — but run a sanity check: if the highest-scoring provider is also the most expensive by a large margin, make sure the score gap is large enough to justify the cost premium. Security stack and SLA quality are the categories where cutting corners creates the most real-world risk; a provider who scores poorly on these shouldn't win on price alone.
The full scorecard template — formatted and ready to fill in — is in the MSP Evaluation Guide.