Direct answer: An AI-search technical audit verifies that priority pages return a successful response, allow the intended search crawlers, remain indexable and canonical, expose useful text, permit appropriate snippets, load well and can be discovered through internal links. It also checks whether CDN, WAF or bot mitigation blocks legitimate AI-search crawlers despite correct robots rules.
Key takeaways
- Audit the response Googlebot or another crawler actually receives—not only browser source.
- Robots.txt controls crawling; noindex controls indexing when the directive can be read.
- Snippet controls can limit how content is used in Google AI Search features.
- Crawler access, content quality and conversion performance must all work together.
1. Build a priority URL inventory
List revenue pages, topic pillars, high-intent guides and proof pages. Record intended canonical, indexability, sitemap inclusion, internal-link source and conversion goal. An audit of thousands of low-value URLs is less useful than a precise review of the pages the business needs to win.
2. Test crawlability and indexing
| Layer | Test | Failure example |
|---|---|---|
| HTTP | 200 on preferred URL | Soft 404 or blocked 403 |
| robots.txt | Intended crawler allowed | Broad disallow rule |
| Indexing | No accidental noindex | Staging tag left live |
| Canonical | Consistent preferred URL | Points to unrelated duplicate |
| Sitemap | Only canonical indexable URLs | Redirects and errors included |
| Links | Priority page discoverable | Orphan article |
Use Google Search Console URL Inspection to compare the rendered page Google received. Public tools cannot confirm the property's selected canonical, last crawl or exclusion reason.
3. Review snippet and AI visibility controls
Google documents nosnippet, data-nosnippet, max-snippet and noindex controls for managing Search previews, including AI features. Restrictive settings may reduce how content appears. Apply them deliberately at page or element level and verify after recrawling.
4. Verify rendered content
- Main answer is present in the rendered DOM and not gated by interaction.
- Titles, H1s and canonicals are unique and stable.
- JavaScript errors do not remove content or links.
- Mobile layout has no hidden body, oversized media or horizontal page overflow.
- Tables remain usable through responsive layout or contained scrolling.
- Images use descriptive alt text where the image conveys meaning.
- Structured data is valid and matches visible content.
5. Audit internal information architecture
Every supporting page should link to its pillar, nearby siblings and an appropriate service or proof page. Use descriptive anchor text naturally within the explanation. Pagination must not make later articles unreachable to crawlers; keep real HTML links in the source even if JavaScript changes the visible page.
6. Test platform-specific crawler access
For ChatGPT Search, review current OAI-SearchBot guidance. For Perplexity, review PerplexityBot documentation. Check user-agent rules, published IP validation when required, server responses, WAF events, rate limits and JavaScript challenges. Do not allow a bot solely because an unknown request copied its name.
7. Check experience and performance
Review Core Web Vitals in field data where available, mobile usability, intrusive overlays, font loading, image size and interaction stability. Performance is not a substitute for useful content, but poor experience can prevent users from engaging or converting after discovery.
What the audit report should contain
- Issue and affected URLs
- Evidence: response, directive, screenshot or console output
- Impact on crawling, indexing, understanding or conversion
- Exact fix and responsible owner
- Priority based on business pages affected
- Verification method after deployment
Start with blockers, then templates affecting many pages, then content and conversion improvements. The Google AI guide and ChatGPT Search guide explain platform context.