Methodology

Last updated: June 2026

Signed by: Pulkit, MCP Verdict

This page describes how MCP Verdict researches, evaluates, and publishes content. It exists so you can assess our work against our own stated standards — and hold us to them.

What gets profiled and why

We profile MCP servers and clients that meet at least one of three thresholds: significant adoption (measurable install base or search demand), vendor maintenance (the tool’s own company ships and maintains the server), or ecosystem importance (reference servers, protocol-level infrastructure, or servers that define a category). We do not profile every server in the registries. A directory does that. We profile the ones that matter for a decision.

The decision to profile a server is editorial, not commercial. No vendor can pay to be profiled, and no vendor can pay to be excluded. If a server meets the thresholds, it is eligible. If it does not, it is not — regardless of who asks.

How we evaluate

Every server profile and comparison uses a consistent evaluation framework. The criteria are set before the evaluation begins, not shaped by the results.

Server profiles are evaluated on five dimensions:

Maintenance status. Commit recency, issue response time, alignment with the current MCP specification. A server last updated before the most recent spec revision gets flagged. We check the repository directly — not registry metadata, which may be stale.

Capability and token cost. What the server actually exposes (tools, resources, prompts), how many context window tokens the tool definitions consume, and how many tokens typical responses return. Token costs are sourced from testing or documentation and cross-checked where possible. Where we cannot test directly, we state the source and note it as reported rather than measured.

Authentication and security. What credentials the server requires, how those credentials are stored and scoped, whether OAuth 2.1 is supported, and any known security considerations or incidents documented in the repository or public vulnerability databases.

Client compatibility. Which MCP clients the server works with, which transports it supports, and any known client-specific issues. We test against the clients we cover (currently Claude Desktop and Cursor) and note untested clients explicitly.

Setup and failure modes. How the server is installed, what goes wrong first, and what the error messages actually say — not what the README says they say. Setup guides are written from actual installation attempts, not synthesized from documentation alone.

How comparisons work

Comparisons and rankings state their criteria explicitly at the top of the page. The criteria are chosen before any server is evaluated against them. The verdict follows from the criteria. This order is non-negotiable: criteria first, then evaluation, then verdict. We do not set criteria to justify a predetermined conclusion.

If you disagree with a verdict, check the criteria first. If the criteria are reasonable and the evaluation is accurate, the verdict follows. If you think the criteria are wrong for a particular use case, that is a legitimate disagreement — and one we want to hear about.

Source quality rules

We use a source hierarchy, and we cite where claims come from:

Primary sources (preferred): official specification documents, vendor documentation, GitHub repositories (commit history, issues, release notes), and published changelogs. These are verifiable and timestamped.

Secondary sources (acceptable with attribution): vendor blog posts, official announcements, conference talks with published recordings or transcripts. These are useful for context but may contain marketing framing.

Tertiary sources (used cautiously): third-party blog posts, tutorials, community discussions. These can surface real-world experience but may be outdated or inaccurate. We cross-check tertiary claims against primary sources before publishing.

Not used: unverifiable claims, anonymous tips without supporting evidence, social media posts without corroboration, and registry metadata without repository verification.

When a claim cannot be verified against a primary source, we either omit it or state explicitly that it is unverified.

Dating and freshness

Every profile and comparison carries a “last verified” date. This date means someone reviewed the content against current primary sources on or after that date. It does not mean the content was written on that date.

Our target re-check cadence is quarterly for active profiles. Servers with high rates of change (frequent spec updates, active development, known instability) may be re-checked more frequently. Servers that are archived, deprecated, or unmaintained are noted as such and re-checked less frequently.

Between verification cycles, information may become outdated. The “last verified” date is your signal for how much to trust the current content. If the date is more than six months old, treat the profile as potentially stale and check the repository directly.

How we handle errors

When we get something wrong — a factual error, an outdated claim, a miscategorized server, a broken setup instruction — we fix it. Our process:

The error is confirmed against primary sources. The content is updated with the correct information. A correction notice is added to the page noting what changed and when. The correction is logged on our public corrections page.

We do not silently edit published content. If the change is substantive (a factual correction, a changed verdict, a revised ranking position), it gets a correction notice. Minor edits (typos, formatting, link fixes) do not get correction notices but are still tracked in page revision history.

To report an error: email contact@mcpverdict.com with the page URL, the specific claim, and your source. Corrections are prioritized over all other inquiries.

Editorial and commercial separation

MCP Verdict may generate revenue from newsletter sponsorships, affiliate links, and lead-generation partnerships. When any of these are active, they are disclosed on the relevant pages and on our affiliate disclosure page.

The separation rule: revenue sources never influence editorial content. No one pays to change a verdict, move up a ranking, alter a comparison, or suppress a negative finding. Affiliate links, when they exist, appear on profile pages as a convenience. They do not appear on ranking or comparison pages. The link existing never changes what we write about the product.

Sponsored content in the newsletter is clearly labeled. Sponsored placements on the site, if offered in the future, will be visually distinct from editorial content and labeled as sponsored.

This is the line that keeps the site worth reading. We will not cross it.

Holding us accountable

This methodology is public so you can check our work against our own standards. If you believe we have deviated from what is described here — criteria set after the verdict, an undisclosed conflict, a sourcing failure — tell us. contact@mcpverdict.com.