MCP status: methodology

0. Headline

IsItStillUp publishes measurements of publicly reachable MCP endpoints and an opinion, computed by the published formula on this page, about how much evidence stands behind those measurements. Nothing here is a statement about a company, its people, or its intentions.

These numbers are IsItStillUp's opinion, computed by a published, versioned formula from probe results we disclose on this page. They describe what our probes observed from named vantage points during a stated window; they are not a guarantee, a security assessment, or a statement about the vendor. Product names identify the endpoint measured; no affiliation or endorsement is implied.

1. What we measure, and what we never do

For each listed endpoint we send the two documented, side-effect-free discovery requests of the Model Context Protocol — initialize and, where the server answers without authentication, tools/list — from every network we operate, at a stated cadence, and we record what came back: the HTTP status, the time to the first complete answer including TLS setup, whether the answer was a valid MCP response, the protocol version, the number and names of tools advertised, and a hash of their declared shape.

For endpoints that require authentication we record only the front door: the 401 challenge, and whether the two documents an OAuth client needs — the protected-resource metadata and the authorization server’s discovery document — can be fetched.

We never call a tool, never send credentials, never attempt to bypass authentication, never send an Origin header, and never store any content a server returns beyond the fields listed above, each cut to a fixed length. Server-supplied names are shown only in a box marked “self-reported” and are never used to identify a server.

2. Networks, cadence and quorum

Two networks: Azure Central India (tools-centralindia), from a location and address we detail under Probe identity below, and a Cloudflare Worker (cf-edge) whose location varies and is recorded per probe.

Open endpoints are probed every 5 minutes; authenticated front doors every 30 minutes. Both networks claim probes for a listing against the same published cadence, so it is one number per tier, not a separate rate per network.

An endpoint is called unreachable only when the latest counted probe from every configured network has failed (2 of 2 networks with two configured) and 2 consecutive probes have failed; it is called reachable again on the first success.

When one network goes silent, its last vote remains valid for up to ten probe intervals (never less than 10 minutes), so the reachability state can lag: 3,000 s (50 minutes) at most for an open-tier endpoint, 18,000 s (5 hours) at most for a gated-tier one. The page shows “some probes failing” when the remaining network alone reports two consecutive failures — the score-time degraded state (§3) covers exactly this lag.

Latency from the edge network is shown but never scored, because its location varies.

3. Health

Health is a number from 0 to 100 computed from four inputs with fixed weights, never renormalised even when an input cannot be observed:

Where an input cannot be observed — a front door hides latency and schema — we substitute a published prior and say so on the page as “prior used”. The literals below stand until at least 10 rated open-tier targets exist; after that the prior is the daily population median of rated handshake-depth targets, republished in methodology.json every day.

Weights are never redistributed to compensate for a missing input — see the header comment on WEIGHTS in the scoring code for why that matters: renormalising once let an authenticated front door, which by construction hides two of the four inputs, outrank a server measured end to end. The ranking on the leaderboard (§8) uses Health computed with the availability lower bound (§7), so thin evidence ranks below solid evidence.

A drift event’s severity, before the 50% halving described above:

4. Outcome classes

Every probe is classified into one of the outcomes below. Counted classes contribute to availability — including every failure. No-evidence classes do not contribute to availability and are shown as facts. A refusal is not an outage. An endpoint that refuses every network we operate for seven days is listed as “opted out (refused)” and no longer probed.

5. Trust signals (provenance)

A second number, 0 to 100, describes how much independent, verifiable provenance we found — it is not a rating of the vendor. Five terms, weighted as points out of 100:

A term we could not observe is set to 0.5 and listed as “imputed”; the page states how many of the five were measured. Publishing a status feed can only keep or raise this number. Claiming a listing changes nothing in it. Trust is not computed for endpoints observed for fewer than 14 days.

6. Evidence grades and the unrated floor

We publish a score only after at least 100 counted probes over at least 3 days; a second network is needed for grade A only. The Evidence grade (A–D) describes our data, not the endpoint.

Coverage is measured per network against that network’s own cadence, so a missed edge cycle lowers it. If field reports (§9) disagree with our probes, the grade is capped at C and the page says “field reports disagree with our probes”. Listings under review, delisted, frozen or refused carry no score and say why. A score is published after 100 counted probes over 3 days.

7. Availability intervals

Beside every availability figure we print a 95% interval computed with the Wilson method on an effective sample size that accounts for the recency weighting (Kish’s formula). Example: 99.5% from one network at a 15-minute cadence over 7 days has an effective sample size of about 273 and an interval of 97.7% to 99.9%. We do not print a single “confidence” number anywhere.

8. Ranking

Within each tier the order is:

• rated before unrated;
• reachable before unreachable;
• Health computed with the availability lower bound (§7);
• Evidence grade;
• name.

Open and authenticated endpoints are never ranked against each other. Unrated endpoints are listed last in their own block. Colours follow the reachability state first and the score second, and every colour has adjacent text.

9. Field reports

Anyone may report the outcome of a real call as a counter — ok, error, timeout, auth error, schema mismatch — without content. Reports never change a score. They can lower the Evidence grade when many distinct reporters disagree with our probes, which is the only way we can detect an endpoint that treats our probe differently from real clients. Reporter addresses are pseudonymised with a monthly key and deleted after 14 days.

10. Probe identity

User-Agent, exactly:

Requests per endpoint: one initialize and, for open endpoints, one tools/list; for authenticated endpoints two additional GETs of public well-known documents. Cadence as in §2; on a 429 we double the interval for 24 hours, then quadruple, then stop.

Central India egress address: we have not yet pinned and published a fixed outbound address — the deployed replica does not carry one today, and we are not going to print a number here we cannot stand behind. When one is pinned it will be published on this page and at /probe.txt, not before.

Opt out at any time: a DNS TXT record _isitstillup.<host> = optout, a robots.txt rule for our User-Agent or for all agents on the endpoint path (checked daily), the dispute form on any listing, or an email to tools@isitstillup.com. We honour it within one daily cycle and list the endpoint as “delisted at the operator’s request”, never silently. Full detail: /probe.txt.

11. What we keep

Per-probe results for the last 35 days; daily aggregates for 90 days; disputed periods, incidents, grade drops and legal notices are archived as evidence and kept for at least three years. Those archives are released under Data: CC BY 4.0 alongside any correction that cites them.

12. Sources, licences, other public sources

OpenSSF Scorecard results (CDLA-Permissive-2.0, attributed); server.json as published to the Official MCP Registry (CC0); vendor status feeds read as published facts; our daily data, snapshots and released evidence are Data: CC BY 4.0.

Other public sources of MCP server health

• MCP Queen — registry-wide grading of MCP servers, published under CC BY, on its own cadence and from its own vantage points.
• openstatus — private, multi-region uptime monitoring offered to its own customers.

We cite them and do not re-serve their data.

13. Known weaknesses

The part of this page we would most want to read about someone else. These are real, current, and listed because you would find them anyway.

• Canned answers are detectable only through field reports. An operator can special-case our User-Agent or our address and serve our probe a canned answer that a real client would never see; we have no way to catch that from the probe side alone.
• One network only until enough history accumulates. While a listing has only one network reporting, its Evidence grade is capped at B — grade A requires both networks (§6).
• Edge latency is not scored — the Cloudflare Worker’s location varies, so its timing is shown, never graded.
• Authenticated endpoints are measured at the front door only. An outage behind a healthy login page is invisible to us.
• Cloudflare-fronted targets share our edge network’s infrastructure. A Cloudflare-wide incident can move both the target and the vantage that measures it at once.
• We probe the legacy handshake first. Almost every live server still speaks the pre-2026-07-28 initialize revision rather than the current one, so our own MCP servers are measured the same, legacy-first way until we finish migrating them.
• Reachability state can lag a silent network. Up to 3,000 s (50 minutes) for an open-tier endpoint, 18,000 s (5 hours) for a gated-tier one — see §2.

14. Corrections, disputes, delisting, court orders

Every listing has a dispute form. Disputes from the endpoint’s operator receive a first response within 2 business days; other reports are reviewed in order. A dispute does not change a listing by itself. When we review a listing at the operator’s request or on a legal notice, the page shows “Listing under review — a correction request is being checked; no score is shown while we verify.” Every resolution — including a review that found no error — is logged on the listing.

On receipt of a court order we delist within 24 hours, before any review. On receipt of a written legal notice from a person with apparent authority for the listed endpoint we mark the listing under review within 24 hours, archive the evidence, and answer in writing within 5 business days.

Legal notices and grievances: legal@isitstillup.com. Personal data in disputes: an optional email, used only to reply, deleted after one year or on request (privacy@isitstillup.com).

15. Reproduce a number

Fetch https://www.isitstillup.com/api/v2/tools/deepwiki/tool.json. Take availability.pct (99.5), latency_ms.p50 (412 → Latency = 1 − (412 − 300) / 2700 = 0.959), last_probe.classification (up, tools_ok true → Protocol 1.0), schema.changes_30d (one additive change 3 days ago → Stability = 1 − 0.15 × 0.5^(3/7)/2 = 0.944).

Health = round(100 × (0.6 × 0.995 + 0.2 × 0.959 + 0.15 × 1.0 + 0.05 × 0.944)) = round(100 × 0.9860) = 99 — the page prints this same line under “How the scores are made”. Ranking uses the same formula with availability.ci95[0].

16. Glossary

17. Changelog

“Last updated” at the top of this page is the date of the newest entry above, read directly from this table — it cannot be set by hand.