Noindex Tag

The noindex tag is a directive that instructs search engines not to include a page in their search index, which means the page will not appear in search results even though the search engine may still crawl it. It is most commonly delivered as a value in the meta robots tag in a page's HTML head — — or, for non-HTML files and server-wide control, in an X-Robots-Tag HTTP response header. When a compliant crawler reads noindex, it drops the page from the index or refrains from adding it. Teams reach for noindex on pages that have no business showing in search: thank-you and confirmation pages, internal search results, thin filter or parameter pages, staging content, login screens, and duplicate or low-value pages that would clutter results or dilute a site's presence.

The noindex tag matters because indexing and crawling are separate decisions, and noindex is how you control the first while leaving the second alone. A search engine must be able to reach and read a page to see its noindex directive, so a page you want removed from the index should stay crawlable, not blocked. That is a subtlety that trips people up: blocking a page in robots.txt does not remove it from the index and can actually prevent the crawler from ever seeing the noindex that would. Used correctly, noindex keeps the index full of the pages you want found and free of the ones you do not, which sharpens how a site appears in search and prevents low-value pages from competing with your important ones.

The noindex tag lives inside the meta robots tag, so the two are related but not identical. The meta robots tag is the container — a single tag that can carry several directives (index or noindex, follow or nofollow, and others) — while noindex is one specific value it can hold. Saying "add noindex" usually means setting the meta robots tag's content to noindex. The distinction matters because the meta robots tag does more than noindex alone, and mixing up the container with one of its instructions leads to muddled implementations. Noindex controls indexing; the follow/nofollow value in the same tag controls whether links on the page are followed. They are independent, so a page can be noindex yet still pass link signals if it is set to noindex, follow.

Noindex is also frequently confused with robots.txt and with the canonical tag, and each does a different job. Robots.txt controls crawling — whether a crawler may fetch a URL at all — not indexing; a URL blocked in robots.txt can still end up indexed from external links, and blocking it can hide a noindex the crawler needs to see. The canonical tag handles duplication by naming the preferred version among similar pages, consolidating signals to it, but it is a hint, not a removal instruction — a canonicalized page can still be indexed. Noindex, by contrast, is an explicit instruction to keep the page out of the index. The rule of thumb: use robots.txt to manage crawling, canonical to manage duplicates, and noindex to actually exclude a page from search results — and keep a noindex page crawlable so the directive is seen.

Using noindex well means applying it deliberately to pages that should not appear in search — confirmation pages, internal search results, thin or duplicate pages, staging and admin screens — while making sure those pages remain crawlable so the directive is actually read. It means not combining noindex with a robots.txt block on the same URL, because the block can stop the crawler from ever seeing the noindex. It means pairing noindex with the right follow/nofollow value for the situation (often noindex, follow so link equity still flows), auditing which pages carry noindex so nothing important is excluded by accident, and removing noindex when a page should return to the index. Treated as a precise instrument, noindex keeps search results clean and focused on the pages that matter.

The failures are dangerous because they are quiet. Accidentally leaving noindex on important pages — a common accident when a staging directive survives a launch — removes them from search entirely, and nobody notices until traffic falls. Blocking a page in robots.txt while expecting noindex to remove it fails, because the crawler cannot read the directive it cannot fetch. Confusing noindex with canonical leaves duplicate pages indexed when you meant to exclude them. The discipline is to use noindex as the deliberate exclusion instruction it is, keep noindexed pages crawlable, keep it off pages you want found, and audit it regularly — because the whole point of the tag is that it silently removes pages from search, which is a great feature when intended and a costly bug when not.

Noindex tag — a robots directive delivered in a meta robots tag or HTTP header — tells search engines to keep a page out of their index, distinct from robots.txt (crawling) and canonical (duplication).

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where noindex tag is a core concern: