"noindex" directive

The noindex directive tells search engines: "you can crawl this page, but don't include it in the index." A page with noindex won't appear in search results, even if it's perfectly accessible to users and crawlers.

It's the standard tool for keeping pages out of search results from Google and other search engines while still allowing them to be publicly accessible.

How to apply noindex

The method depends on the file type (HTML vs PDF) and which bots you want to target.

In the webpage HTML ""

<meta name="robots" content="noindex">

In an HTTP response header (for non-HTML files like PDFs)

X-Robots-Tag: noindex

Targeting specific bots

The generic "robots" meta will instruct all bots to noindex the page, but this can also be done for specific bots only, e.g.:

<meta name="googlebot" content="noindex">
<meta name="bingbot" content="noindex">

In cases where a page has a bot-specific directive (e.g., googlebot) and the generic robots tag, the bot-specific directive will be followed by that bot.

When to use noindex

Pages that have no purpose being in search results

• Internal search results pages — thin, duplicative content that creates infinite URL variations.
• Account and checkout pages
• Thank-you and confirmation pages — they're only meaningful after a conversion, not for acquisition.
• Tag archive pages with little or no unique content (a common problem in WordPress, where the primary category archives already list all content appropriately).
• Author archives — pointless on single-author websites.
• Test, staging, or development URLs accidentally exposed to the public web.
• Print-friendly versions of articles.

When NOT to use noindex

• Pages that should be canonicalised elsewhere. noindex on a page that should be consolidated into another removes it from search entirely instead of merging signals.
• The whole site, accidentally. Misconfigured CMS settings or stray staging templates can apply noindex site-wide.
• Doubling up with robots.txt disallow. If a page is blocked by robots.txt, Google can't fetch the page and won't see the noindex tag. Blocked pages may still be indexed if they have enough inbound links (often as a "URL only" entry without a snippet). To deindex a page reliably, it should permit crawling to allow bots to read the noindex tag.

How long does it take for a page to be deindexed?

Google has to re-crawl the page to discover the noindex directive, so it depends on how frequently the content is currently being crawled. A few days to a few weeks is standard.

To accelerate a page's removal from Google results, the URL can be submitted to Search Console's URL Inspection tool to request a recrawl.

The "Removals" tool in Search Console will remove URLs from results faster, typically within a day; however, this is temporary (6 months). If noindex is not also applied to the page, it will return to the index after the removal expires.

Specific removal requests are intended more for emergency removals from search results, e.g., if sensitive information was accidentally published on a public page. There is no need to do this for general website housekeeping. Once noindex is applied, the pages will naturally drop out of search over time.

Common mistakes

1. Noindex + canonical to elsewhere

Conflicting signals. Pick one: - If you want a URL deindexed, use noindex. - If you want signals consolidated to another URL, use a canonical tag.

2. Noindex pages included in sitemaps

Sitemaps should only contain URLs you want indexed. Including noindex pages in sitemaps just wastes crawl budget.

3. Noindex + nofollow

While the full noindex, nofollow combination will deindex a page, on internal pages that have outbound internal links it also prevents the page from passing internal link equity. In nearly all cases, noindex, follow is what you want — the page won't be indexed, but its internal links may still pass signals.

4. Forgetting noindex during a staging migration

Staging sites are often configured with a site-wide noindex. If the staging environment is migrated to production without removing the directive, the entire site gets deindexed. This is a recurring real-world problem, and can be a disaster if not immediately corrected.

5. Noindex on resources blocked by robots.txt

Already covered above, but worth repeating: a page blocked by robots.txt can still appear in search as a URL-only entry. Allow crawling and apply noindex.

Indexing

Disclaimer: All information contained herein is for informational purposes only. It is not advice or instructional.