Quick answer
401 Unauthorized means the request needs valid authentication. 403 Forbidden means the server understood the request but will not allow it. Both are often correct for private areas; the right choice depends on the authentication protocol and the policy you want clients to follow, not on SEO preferences.
What the issue means
A 401 response normally tells the client that authentication is required and may include a WWW-Authenticate challenge. A 403 says that authentication is not enough, or that access is refused by policy. The wording on a custom error page does not change the status semantics.
Do not use either response to hide a public page that should be available. Conversely, do not make an account, admin, or private file public merely to satisfy a crawler.
Why it matters for SEO
Public pages need a response that allows users and crawlers to retrieve their content. Private routes should remain protected and should not appear in public navigation or sitemaps. A 401 or 403 on a page that was accidentally linked from public content can reveal an information-architecture problem without being a reason to remove access control.
An edge firewall may also return 403 to a crawler while allowing normal users through. That is a coverage and delivery issue, not proof that the page itself is private.
How NOTABIS detects it
NOTABIS records an HTTP error response when a crawled URL ends with a 401 or 403. If the response is produced by an edge challenge or bot-control page, the audit may also classify it as edge blocked and reduce coverage confidence. The crawler cannot infer your authorization policy from the status alone.
How to check it yourself
- Request the URL as an anonymous user and as an appropriately authenticated user.
- Inspect
WWW-Authenticate, cookies, redirects, and response headers. - Check whether a CDN, WAF, reverse proxy, or application route generated the response.
- Remove private URLs from public links and sitemaps unless they are intentionally discoverable.
- Compare the response with the browser behaviour your real users should receive.
How to fix it
Keep 401 when the client must authenticate and the endpoint follows an authentication challenge. Keep 403 when the identity is known or access is otherwise refused by policy. Correct the route or permissions when a public page is blocked accidentally. Adjust edge rules narrowly when they block legitimate crawlers, and verify the change from the relevant network and user-agent conditions.
When this is intentional or does not need fixing
Account pages, administration tools, private downloads, staging routes, and protected APIs commonly return 401 or 403 intentionally. Document the policy, avoid linking those URLs as public content, and assess edge blocks separately from page-level access control.
Related issues
Read HTTP status codes explained, HTTP error responses, edge and bot blocking, and crawl coverage and audit limitations.
Run a NOTABIS audit
Run a NOTABIS SEO audit to see which responses are protected, which are edge blocked, and how much of the site was measurable.