Robots Meta Tag Generator

How to use the robots meta tag generator

1. Indexing: keep index for pages you want in search, or choose noindex to keep the page out of results.
2. Links: keep follow so crawlers can discover pages linked from this one, or choose nofollow.
3. Snippet: "No limit" outputs max-snippet:-1 (Google chooses the length); "No snippet" outputs nosnippet.
4. Image preview: max-image-preview:large, standard or none.
5. Click Copy tag and paste it inside the page's <head>, or into the "advanced robots" field of your SEO plugin.
6. After publishing, open the URL in Google Search Console's URL Inspection tool and check the crawled HTML shows the tag. If the URL is blocked in robots.txt, Google never sees it (more on that below).

The syntax, explained with a real example

With the default settings the tool produces:

<meta name="robots" content="index, follow, max-snippet:-1, max-image-preview:large">

• name="robots" addresses every crawler. Use name="googlebot" to address only Google; Google also honours googlebot-news and ignores other user-agent names in this tag.
• index, follow are Google's defaults, so writing them changes nothing. They are still useful as documentation that the page is meant to be indexed.
• max-snippet:-1 lets Google pick the snippet length; max-image-preview:large allows a large image preview, which matters for visual results and Discover.
• Rules are comma-separated and case-insensitive. Where two rules conflict, Google applies the more restrictive one.

Worked example — a Noida real-estate developer. The site has a lead form that redirects to /thank-you.html. If that page is indexed, people land on it from search and every visit fires the "lead" conversion. At a ₹1,200 cost per lead, just 30 such false conversions a month make the campaign look ₹36,000 cheaper per month than it is (30 × ₹1,200). The fix is:

<meta name="robots" content="noindex, follow">

The page stays out of results, while links on it (back to projects) can still be followed. The same developer runs a festive "₹5 lakh off on booking" page that ends on 31 December 2026. The generator's four dropdowns don't include an expiry rule, so add it by hand:

<meta name="robots" content="max-image-preview:large, unavailable_after: 2026-12-31">

After that date Google stops showing the page, and it crawls it far less often.

Complete robots directive reference

Every rule below is documented in Google Search Central's robots meta tag specification. Other search engines may interpret some of them differently.

data-nosnippet: hide part of a page from snippets

To keep one block out of snippets (a price disclaimer, say) without affecting the whole page, add the boolean data-nosnippet attribute. Google accepts it only on span, div and section elements, and the HTML must be properly closed or the exclusion runs on into the rest of the page:

<div data-nosnippet>Prices exclude GST and registration charges.</div>

X-Robots-Tag header examples for Apache and Nginx

A PDF brochure or price list has no <head>, so the rule goes in the HTTP response header. Any rule valid in the meta tag is valid here. Apache (.htaccess or server config):

<Files ~ "\.pdf$">
  Header set X-Robots-Tag "noindex, nofollow"
</Files>

Nginx (inside the relevant server block):

location ~* \.pdf$ {
  add_header X-Robots-Tag "noindex, nofollow";
}

To target Google only, prefix the user agent: X-Robots-Tag: googlebot: noindex. Check the live header with curl -I https://example.com/brochure.pdf.

robots.txt vs meta robots: which one to use

Google's robots.txt introduction states plainly that robots.txt "is not a mechanism for keeping a web page out of Google". Never combine the two on one URL: if robots.txt blocks a page, Google cannot read its noindex.

Common mistakes and how to fix them

• Staging noindex goes live. WordPress's "Discourage search engines" setting outputs a noindex tag; a redesign launched with it on can drop a site from Google within weeks. Check the live source after every launch.
• Disallow plus noindex. Remove the robots.txt block until Google has recrawled and dropped the page, then re-block if you need to.
• Two conflicting tags. A theme outputs index and a plugin outputs noindex: the restrictive one wins. Keep a single source of robots tags.
• Using nosnippet to "protect" content. It removes your description from results and from AI Overviews input, which usually cuts clicks. Use data-nosnippet on the specific block instead.
• Expecting instant removal. noindex takes effect only when the page is recrawled. For urgent cases use Search Console's Removals tool as a temporary measure.
• Relying on noarchive. Google no longer uses it; it does nothing.

Practical use cases

• D2C store: noindex, follow on internal search results and thin filter combinations, so product pages keep getting discovered.
• Clinic or hospital: noindex on appointment-confirmation and patient-portal login pages.
• B2B SaaS: X-Robots-Tag noindex on gated whitepaper PDFs so the landing page, not the file, ranks.
• Publisher or blog: max-image-preview:large on articles to be eligible for large image previews.
• Education institute: unavailable_after on last year's admission-date pages.

For a full crawl and indexing check, see our technical SEO audit checklist; if thank-you pages are inflating your numbers, read how to fix duplicate conversions in GA4 and Google Ads. Writing titles too? Use the meta tag preview generator, or talk to our SEO team.

The generator runs entirely in your browser — nothing you select is sent to our servers.