Status Codes and SEO
The HTTP status code is the single most important technical signal a crawler receives about a URL. 200 means "index me". 301 means "I moved, credit the new URL". 404 means "gone". 500 means "retry later — I'm broken".
Common SEO-Relevant Codes
- 200 OK — normal, indexable.
- 301 Moved Permanently — permanent move, passes full equity.
- 302 Found — temporary. Rarely the right choice in production.
- 404 Not Found — page doesn't exist. Soft 404s (200 with "not found" content) confuse crawlers.
- 410 Gone — page existed but is now permanently removed. Google deindexes faster than 404.
- 429 Too Many Requests — rate limiting. Crawler backs off and retries.
- 500 Internal Server Error — retry later. Persistent 500s drop URLs from the index.
- 503 Service Unavailable — use for planned maintenance, but include
Retry-After.
Headers That Matter
| Header | Purpose |
|---|---|
| Content-Type | Must be text/html; charset=utf-8 for HTML pages. |
| Strict-Transport-Security | HSTS — forces HTTPS for future requests. |
| X-Content-Type-Options | Should be nosniff. |
| X-Frame-Options | DENY or SAMEORIGIN to prevent clickjacking. |
| Referrer-Policy | Controls referrer leakage. |
| Cache-Control | Controls CDN and browser caching. Impacts Core Web Vitals on repeat visits. |
| Content-Encoding | Should include gzip or br for text assets. |
| X-Robots-Tag | Equivalent of meta robots but at HTTP level — applies to PDFs, images too. |
Why Headers Matter for SEO
HSTS reduces the risk of protocol downgrade attacks. Missing Cache-Control forces repeat visitors to re-download assets, directly hurting INP on return visits. Missing Content-Encoding can double your transfer size. A wrongly-set X-Robots-Tag can accidentally noindex an entire section of your site.
Soft 404s — The Silent Killer
A soft 404 is a page that returns 200 OK but shows "not found" content. Google has to guess whether the page is empty, dead, or simply a thin page. Persistent soft 404s waste crawl budget and can pull authority from the whole site. If content is gone, return a genuine 404 or 410 — never a 200 with a "nothing here" message.