Crawled, Currently Not Indexed: Run a Quality Audit

Google Indexing Quality Audit

Watch the 5-minute Crawled - currently not indexed audit workflow

A technical-first walkthrough for separating crawl, render, canonical, duplicate, internal-link, and quality problems before rewriting pages.

Google's John Mueller and Martin Splitt just gave SEOs a useful reminder about indexing: quality can affect how many pages Google chooses to crawl and index, but quality is not limited to the article text.

That last part matters. A page can have unique sentences and still be a bad search result. It can bury the answer below ads, load slowly, duplicate a dozen other URLs, hide the main content behind scripts, or exist only because an SEO content calendar needed another URL.

So the move is not to panic when Search Console shows a larger `Crawled - currently not indexed` bucket. The move is to stop treating that bucket as one problem. It is an evidence queue. Some URLs need a technical fix. Some need consolidation. Some need stronger internal links. Some should not be indexed at all.

What Google Said

The discussion came through a Search Engine Journal report on a conversation between Mueller and Splitt about the Page indexing report. Splitt described the difference between `Discovered - currently not indexed` and `Crawled - currently not indexed`: in the first case, Google knows the URL exists but has not visited it yet; in the second, Google visited the URL and still did not put it in the index.

Mueller's answer on quality was careful. When Splitt asked whether a not-indexed status is a quality sign, Mueller said it is sometimes. That qualifier should control the whole audit. Sometimes is not always.

He also explained that when Google's systems have strong concerns about overall quality, Google may crawl less and index less. In practice, that means a broad non-indexing pattern with no technical explanation can point to a site-level quality problem. It does not mean every excluded URL is a bad page.

The strongest part of the discussion was the definition of quality. Mueller did not frame it as word count or text uniqueness alone. He talked about the full page experience: ads, interstitials, filler, layout, performance, and whether users can reach what they came for.

Do The Technical Audit First

Before you diagnose quality, prove that Google can access and understand the page.

Start with a cohort, not a random URL. Pull a sample of excluded pages from Search Console and group them by template, page type, intent, directory, canonical target, sitemap inclusion, and internal-link path. You are looking for patterns that repeat.

For each cohort, check the basics:

• HTTP status and redirect chain.
• Robots.txt access and page-level `noindex` controls.
• Canonical target and duplicate cluster.
• Sitemap inclusion and lastmod accuracy.
• Internal links from crawlable pages.
• Rendered HTML, especially if the page relies on JavaScript.
• Template-level thinness, duplication, or boilerplate.
• Server errors, DNS instability, and crawl spikes.

This is where many indexing projects go wrong. They jump from "Google crawled it and skipped it" to "Google hates our content." Maybe. But if the canonical points somewhere else, the content appears only after a fragile script runs, the URL is an infinite faceted variant, or the page has no internal links, the quality debate is early.

That technical-first path also protects the team from fake fixes. You do not want a content rewrite project when the real issue is canonical conflict. You do not want a disavow project when the real issue is 40,000 low-value URL variants. You do not want to publish more copy when Google is already telling you the template is not worth much crawl attention.

When Quality Becomes The Right Diagnosis

Quality becomes the right diagnosis after the technical evidence gets boring.

If Google can crawl the pages, render the main content, see the intended canonical, reach the URLs through internal links, and still leaves a large share of a section out of the index, then the question changes. The audit is no longer "can Google index this?" It is "why should Google index this?"

That question is uncomfortable because site owners know the work that went into the site. Mueller made that point directly. Your own site is hard to judge because you remember the effort, the meetings, the draft history, and the business reason for every page. Google sees a URL competing against the rest of the web.

For SEOFrancisco audits, I like to make the quality review practical:

• Intent match: Does the page answer the query behind the URL, or does it circle around the topic?
• Original value: Is there field experience, data, examples, screenshots, a stronger framework, or a clearer answer than competing pages?
• Task completion: Can the visitor finish the job without fighting the layout?
• Template value: Does this template create useful pages at scale, or does it generate near-duplicates?
• Proof: Are claims supported by visible evidence, sources, tests, or specific examples?
• Page experience: Is the main content easy to access on mobile and desktop?

This is where AI content can become relevant, but only as an example. The problem is not that AI touched the draft. The problem is pages that sound interchangeable, add no proof, and give users nothing they could not get from ten other results. Human-written filler can fail the same test.

Quality Is The Whole Page

Google's people-first content guidance already pushes teams toward usefulness, reliability, and value. The indexing discussion adds a useful operational detail: the page wrapper matters too.

A page can lose the quality argument before the main paragraph starts. Intrusive ads, jumpy layouts, slow scripts, heavy media, confusing navigation, and filler before the answer all change the user's experience of the content.

That does not mean every Core Web Vitals issue explains non-indexing. It means page experience belongs in the quality audit rather than in a separate cosmetic bucket. Google's page experience guidance names signals and checks that help site owners assess whether users can access and use the page comfortably.

In a real indexing audit, I would inspect the excluded template in four views:

1. Raw HTML: what is present before JavaScript?
2. Rendered HTML: what does Google likely see after rendering?
3. Mobile viewport: how quickly can a person reach the answer?
4. Crawl graph: how does the URL earn discovery and priority inside the site?

That same lens connects with AI search work. If a page is hard for Google to fetch, render, classify, and quote, it is also weak for AI Overviews and agentic search. The foundation is still crawlable, indexable, useful content, which is why Google's AI search guidance keeps pointing back to serious SEO.

The Cohort Audit Workflow

Do not start with one URL and argue from a screenshot. Build a sample.

Export affected URLs from Search Console, then create cohorts. For a medium or large site, I would separate them like this:

Cohort	What To Check First	Likely Action
New URLs	Discovery date, sitemap, internal links, crawl status	Wait, strengthen links, or request indexing after validation
Template pages	Duplicate patterns, canonical targets, thin boilerplate	Consolidate, improve template value, or noindex low-value variants
Editorial articles	Originality, intent match, proof, internal links	Rewrite around a sharper answer and add evidence
JavaScript-heavy pages	Raw HTML, rendered HTML, blocked resources, hydration failures	Move critical content into crawlable HTML or fix rendering
Faceted or parameter URLs	Crawl traps, duplicate sets, canonical and robots behavior	Control crawl paths and consolidate duplicates

Then score each cohort on two axes: technical confidence and index-worthiness. A page with low technical confidence needs engineering or information architecture work. A page with high technical confidence and low index-worthiness needs editorial, template, or consolidation work.

The worst middle state is a technically accessible page that says nothing new. That is where teams keep requesting indexing, adding a few paragraphs, and refreshing the publish date. If the page does not give Google a reason to keep it, those rituals will not create one.

What To Fix Before You Request Indexing

Request indexing after the page has changed in a way that matters.

For a technical issue, that may mean a corrected canonical, crawlable internal link, fixed render path, clean status code, or removed accidental `noindex`. For a quality issue, it means the page has a stronger reason to exist: better intent coverage, original evidence, clearer structure, faster access to the answer, less filler, or consolidation with a better URL.

Use the URL Inspection tool for representative URLs, not as a bulk magic button. If one template has 4,000 excluded pages, inspect samples from each pattern, document the fix, validate the live URL, and monitor the cohort. The goal is not to force one page into the index. The goal is to make the pattern index-worthy.

For JavaScript-heavy pages, pair this with rendered HTML checks. Our JavaScript rendering audit shows why raw HTML and rendered output can tell different stories. For agent-facing pages, the agentic browsing and llms.txt discussion points to the same operational principle: machines need a page they can inspect without guessing.

If the affected pages are commercial or lead-generation pages, connect the audit to real business outcomes. Thin location pages, weak service variants, or comparison pages with no proof do not become valuable because they target a keyword. That is exactly where a technical SEO advisory audit should meet content strategy.

The Practical Takeaway

Mueller's comment should not turn every indexing issue into a vague quality debate. It should make the audit more disciplined.

Use Search Console to find the pattern. Use technical checks to remove false causes. Then judge whether the remaining pages deserve indexation from a user's perspective. If they do, make the evidence stronger and the page easier to use. If they do not, consolidate, noindex, redirect, or leave them out of the index on purpose.

That is the mature version of index coverage work. It is less satisfying than blaming one technical switch, but it is closer to how Google has described the system: crawl, render, classify, evaluate, and decide what deserves to be stored for retrieval.

Related Articles

• Google's AI Search Guidance Makes One Thing Clear: SEO Still Does the Work
• AI Assistants and JavaScript Rendering: What SEOs Need to Fix
• Google Says You Do Not Need llms.txt. Lighthouse Checks It Anyway.
• Google Universal Commerce Protocol: What SEOs Need to Do
• Technical SEO Advisory

FAQ

About the Author