How to tell a broken link from a protected endpoint

A link checker requests a URL, reads the status line, and writes a row in a report. Three rows come back looking identical:

Only one of those is likely a defect a visitor will meet. One is access control firing, which is what a protected route is supposed to do — though whether it was supposed to fire here is a question the row cannot answer. And one is not a result at all: it is the server declining to answer the scanner, right now, and saying nothing about the link.

Hand that list to a client and you have spent their attention on two non-problems. Worse, you have taught them that your findings need re-checking before they are believed.

The response code starts the investigation

A status code is a fact about one HTTP exchange: this client, at this moment, with these credentials, from this IP, got this answer. It is evidence. Turning evidence into a verdict takes a second step most link checkers skip — the question who was supposed to open it?

Three things vary, and all three change the meaning of the same number:

• Who asked. An anonymous crawler is not a signed-in customer.
• When. A rate limiter and an outage produce different codes for the same healthy resource.
• What was asked for. A page, an API endpoint, and a button-shaped anchor are different kinds of thing.

The rest of this is the response classes worth separating, and what each one is actually good for.

404 and 410: the closest a probe gets to certainty

A 410 is the strongest assertion a server makes about a link: intentionally gone, and likely permanently.

A 404 is weaker than it looks. RFC 9110 defines it as did not find a current representation for the target resource, or is not willing to disclose that one exists — the specification deliberately refuses to let 404 mean "does not exist." It goes further: a server that wants to hide a forbidden resource may answer 404 on purpose. It is still the best an anonymous probe gets, but the assertion is about what the server will serve you, not about what is there.

Route /careers/2019-intern-program×Confirmed

HEAD /careers/2019-intern-program HTTP/1.1 404 Not Found

Three caveats keep even this from being automatic:

1. A 404 on a URL nobody linked to is not a finding. Probing a guessed URL and calling the miss a defect is manufacturing work.
2. A soft 404 — HTTP 200 with "page not found" in the body — will not appear here at all. Status-only checking misses it entirely.
3. A deliberate 404 can be concealment. Some servers answer 404 rather than 403 precisely so you cannot tell the difference.

401 and 403: not the same claim

These are where most false positives come from, and they do not mean the same thing.

A 401 does mean credentials: the specification requires a 401 to carry a WWW-Authenticate header saying which ones. A 403 means only that the server understood and refuses. RFC 9110 is explicit that a request may be forbidden for reasons unrelated to credentials — a geo-block, an IP allowlist, a user-agent rule. Signing in might change nothing.

Route /account/settings?Needs confirmation

HEAD /account/settings HTTP/1.1 403 Forbidden

Note what the plate does not say. It does not say the link works, and it does not say the access control is correct. The honest verdict is neither "broken" nor "fine" — it is needs confirmation, carried to the person who can sign in and look.

There is a narrower case worth separating. When the path itself is shaped like a machine endpoint — a segment of api, export, or download — an anonymous refusal is usually the architecture behaving correctly, and asking a client to confirm every protected API route is noise. But a path shape is a weak signal, not a statement of intent: /downloads/brochure.pdf matches the same rule. The right treatment is to stop requesting an action while still not claiming the endpoint is correct.

429, timeouts, bot challenges and 5xx

These four look like failures and are mostly failures of the measurement.

Two distinctions in that table do real work.

Rate limiting and timeouts are usually the scanner's problem, not the client's. The right response is to re-run the measurement rather than file a finding. But not always: RFC 6585 leaves the counting scope undefined — a limit may be per-resource or server-wide — so if a 429 re-fires at a polite rate, that is a visitor-facing condition, not scanner impatience.

A bot challenge is not a rerun candidate. Running the same automated probe against the same wall produces the same wall. Only a human opening the link settles it. The headers worth recognising are Cloudflare's cf-mitigated and cf-chl-*, and AWS WAF's x-amzn-waf-action: challenge.

And 5xx deserves its own line. A single 500 is hard to distinguish from a deploy that happened to be mid-flight. The same 500 across independent attempts is a different claim — and even then it should cap at probable, never confirmed, because the fault may sit with a third-party host or an incident window the probe happened to span.

href="#" is a question, not an answer

An anchor with href="#" or href="" is what RFC 3986 calls a same-document reference: it resolves to the page you are already on, and dereferencing it is defined not to produce a new retrieval. javascript:void(0) is not an HTTP URL at all.

So probing any of the three tells you nothing about the control. Worse — a checker that naively resolves the first two against the base URI will request the current page, get a 200, and score a dead button as a working link.

Inspection ticket№ AP-2026-1318

Medium priorityTechnical

Primary CTA has no destination

FILED ON · /pricing

The evidence is that the anchor's href goes nowhere — #, empty, or a javascript: value. The verdict — dead button, or working modal trigger — needs a click, not a request.

Usually it is a control: a modal trigger, a dropdown, an accordion. Sometimes it really is a dead button that shipped without its handler. These two are indistinguishable from the markup alone. Separating them requires interaction evidence — load the page, click the thing, watch what happens — which is a different instrument from a link checker.

An evidence contract for link results

Everything above reduces to one discipline: decide in advance what each class of result is permitted to claim, and never let a report exceed it. That contract is worth writing down, because the pressure to call an ambiguous result "broken" is strongest at the moment you are assembling findings for a client.

Three properties make this a contract rather than a preference.

Only two rows may deduct on public evidence alone. A confirmed absence, and a failure that reproduced. Everything else is reported without moving the number.

"Inconclusive" is a real outcome, not a hedge. A verdict class that quietly means defect is worse than no class at all, because the client learns to discount the whole report.

Held-back results still appear. Hiding them would be its own dishonesty — coverage a reader cannot see is coverage they cannot trust. They appear with the reason they were held, and with no score impact.

Four links under an adjudicating contractIllustrative figures — method, not a real scan

Four links under an adjudicating contract — Illustrative figures — method, not a real scan

Contribution	Points
404 on a linked careers pageconfirmed	-6
403 on /account/settingsheld back — needs confirmation	0
429 on /api/exportheld back — re-measure	0
"#" on a pricing CTAheld back — needs a click	0
Total	-6

Figures are illustrative and do not come from a scan. The point is the shape: a checker without an evidence contract treats all four as broken, and without adjudication those observations can all contribute to a single deducting link-status finding. Separating them prevents measurement uncertainty from becoming unnecessary client work.

The vocabulary

These six labels are the ones this publication uses to mark a finding. They are the editorial subset of the dispositions AgencyProof's engine defines, and they are deliberately narrower than "pass and fail":

×ConfirmedObserved directly, and the observation is sufficient on its own. !ProbableThe evidence points one way, but a second signal would settle it. ?Needs confirmationCannot be resolved from public evidence. Someone has to check. ◆IntentionalThe behaviour is a control working as designed. –Not applicableThe check does not apply to this kind of site. ✓Verified strengthSomething the site does well, confirmed rather than assumed.

Two are hard to earn from a status code alone. Intentional is a claim about someone's decision, and a path shape is not a decision. Verified strength means a thing was checked and works — which for a gated route needs credentials an external inspection does not have. A link result that is none of the six is better called inconclusive than forced into the nearest label.

Two things to check about your own instrument

Whatever tool produces your report, two of its properties change how you should read it, and neither is usually on the summary page.

What user-agent does it send? Most scanners identify themselves as bots, because that is the polite thing to do. It also means a bot wall firing at them is expected and uninformative. If your report has a cluster of 403s from one host, check the headers before checking the links.

How many links does it probe? Every crawler has a budget. A link that was never requested is not a link that was cleared, and a trustworthy report says which is which rather than presenting partial coverage as complete.

What to do with your own report

You do not need any of this automated to get the benefit. Take the broken-link section of whatever tool you already run and sort it into four piles:

1. 404 and 410 on links that exist on your pages. Real. Fix or redirect.
2. 401 and 403. Sign in and try the same URL. Either it works — in which case the finding is noise — or you have found something a scanner never could.
3. 429, timeouts, and anything from a bot wall. Re-run, more slowly, or open it yourself in a browser.
4. # and empty hrefs. Click them. Every one.

That sorting is the difference between handing a client a list of problems and handing them a decision.