A directive requesting that a supporting search engine exclude a resource from its index. The crawler must access the directive to process it.
Noindex is a directive telling supporting search engines not to include a resource in their index. For Google, it can be delivered through a robots meta tag in an HTML page or an X-Robots-Tag HTTP response header. The method should match the type of resource and the controls available in the site’s publishing system.
The crawler needs to retrieve the directive to process it. If a URL is blocked in robots.txt or otherwise inaccessible, Google may not see the noindex instruction. Adding the directive therefore is not an instant removal button; a previously indexed page needs to be revisited and processed.
Search exclusion is separate from access control. A noindexed page can still be opened directly or followed from a link. Confidential client information needs authentication and authorization, not merely a search setting. Conversely, an unintended noindex on an important public page can conflict with the business’s goal of being found.
Takeaway: Use noindex deliberately for public resources you do not want appearing in search. Check important landing pages for accidental directives, and use real access controls for private content.
Example
A business has a public confirmation page shown after a general inquiry form. The team decides it has no independent search purpose and evaluates noindex for that page. A separate customer portal contains private records and requires a login; keeping that portal secure cannot depend on whether a crawler honors an indexing directive.
What to check
Inspect both the page markup and HTTP headers when diagnosing an unexpected noindex. Confirm that the intended directive is on the correct resource and can be read. Do not remove a sitewide privacy or staging setting without reviewing what else it protects.
For more detail, see Block Search indexing with noindex.