Копия источника
I spent yesterday planning to delete the best part of my site. I run a calculators-and-tools site, solo, five months old. About 5,800 pages across four language sections. Google has 642 of them in its index.
Not great. The rest sit in two buckets: 1,153 "crawled, currently not indexed" and 4,161 "discovered, currently not indexed." Like everyone in that situation, I had a theory. Somewhere in those 5,800 pages I had generated 1,432 timezone pages — every URL a pair of cities, `/date-time/time-difference/london-and-buenos-aires`, that sort of thing. One calculation, different arguments, spread across a URL space.
It looks exactly like what Google's spam policy calls scaled content abuse, and I assumed it was dragging the whole host down. The plan was to collapse them into one hub page and 301 the rest away. Before doing it, I exported the list of what Google actually has. ## The export This is the part I had never bothered with, and it takes about four minutes.
In Search Console: **Indexing → Pages**, then the link **"View data about indexed pages."** At the bottom of the examples table, change rows-per-page to 500. The examples table caps around 1,000 URLs, so if your indexed count is under that, you get the complete list, not a sample. Mine was 642, and I pulled 638 unique URLs out of it. Then classify them.
I did it with a throwaway script that buckets by URL prefix: ``` import re, collections def family(url): p = re.sub(r'https://example\.com/[a-z]{2}', '', url) or '/' if re.match(r'^/date-time/(convert|time-difference|meeting-time|world-clock)/', p): return 'generated timezone pairs' if p.startswith('/recipes'): return 'handwritten recipes' if p.startswith('/cooking'): return 'cooking guides' if re.match(r'^/finance/[^/]+/.+', p): return 'generated finance scenarios' return 'tools and hubs' urls = [l.strip() for l in open('indexed.txt') if l.strip()] for name, n in collections.Counter(map(family, urls)).most_common(): print(f'{n:5d} {name}') ``` Enter fullscreen modeExit fullscreen mode Do the same for the "crawled, not indexed" and "discovered, not indexed" drilldowns and you get the whole picture in three exports. One trap if you automate the reading: the console reuses its DOM between drilldowns. My first pass at the "crawled, not indexed" list came back looking almost identical to the indexed list, and it took a set intersection to notice that 638 of those 660 rows _were_ the indexed list, still sitting in the page. Reload the drilldown URL directly before reading it.
## What came back | Family | Total | Indexed | Rate | Crawled, rejected | Never crawled | | --- | --- | --- | --- | --- | --- | | Generated timezone pairs | 1,432 | **428** | **29.9%** | 647 | 24 | | Generated finance scenarios | 180 | **74** | **41.1%** | 65 | 4 | | Cooking guides | 530 | 61 | 11.5% | 36 | 95 | | Tools and hubs | 1,377 | 65 | 4.7% | 45 | 407 | | Market-specific pages | 593 | 8 | 1.3% | 0 | 138 | | **Handwritten recipes** | **2,290** | **0** | **0%** | **0** | 195 | The two generated families I was about to delete are 502 of the 638 indexed pages. Seventy-nine percent of my index. They also have the two highest acceptance rates on the site. The 2,290 recipes — the ones with photographs, per-serving nutrition, ingredient scaling rules, written natively in two languages rather than machine-translated — have zero pages in the index. Not rejected.
Zero of them appear in the "crawled, not indexed" bucket either. Google has never fetched a single one. (The counts in the last two columns are from samples — those drilldowns cap out around a thousand example URLs, and my buckets are larger than that. The indexed column is complete.) ## Why this is not the story I expected The obvious reading is "Google likes generated pages better," which is nonsense.
Look at the last two columns instead. For the timezone family, Google is **finished**. It crawled essentially all 1,432, kept 428, threw out 647, and has almost nothing left in the queue. That is a completed judgement, and 30% is the verdict.
For the recipes, Google hasn't **started**. They are all sitting in "discovered, currently not indexed," which does not mean rejected. It means Google knows the URLs exist and has not spent a request on them. Those are two completely different failure modes and the summary number blends them into one scary total.
"Crawled, not indexed" is a quality verdict. "Discovered, not indexed" is a scheduling decision — Google deciding your host isn't worth more requests right now. My Links report says external links: 0. That's the whole story: crawl allowance got spent on whatever was linked from the hubs first, and the rest of the site never got its turn.
So the site does not have a content-quality problem in the family I suspected. It has a "nobody links to this domain" problem, which no amount of deleting pages will fix. ## What I would have lost Had I shipped the collapse-and-redirect plan, here is the sequence. Day one, the old URLs are still in the index. Someone searches for a city pair, finds the old URL, clicks, gets a 301 to the hub with the pair pre-filled.
Fine — the reader gets their answer. Week four, Google re-crawls, sees the redirects, drops the old URLs and shows the hub. Now one generic hub page has to rank for every city-pair query on its own. A hub with two dropdowns competes badly against a page whose title is literally the query.
Net effect: hand back 67% of the index in exchange for rankings the hub would have to earn from scratch. To fix a quality problem that the data says is not there. ## The actual takeaway If you are about to prune pages because they feel thin, export the indexed list first. Four minutes.
The report you need is one click below the chart most people stop at, and the answer may be the opposite of your theory. Mine was. And split those two "not indexed" reasons in your head permanently. If your pages are **crawled and rejected**, work on the pages.
If they are **discovered and never crawled**, the pages are irrelevant — go get links, because you are asking Google for more of its budget than your domain currently earns. I am doing the second one now. If it's useful to anybody, the recipe data those uncrawled pages are built on is published as an open dataset — 501 dishes from 127 countries, JSON and CSV, CC BY-SA 4.0, with per-serving nutrition and sourced cooking times: [theunitools.com/en/data](https://theunitools.com/en/data). Google hasn't read it either, but you might. [\\ The DEV Team](https://dev.to/devteam) Promoted Dropdown menu - [What's a billboard?](https://dev.to/billboards) - [Manage preferences](https://dev.to/settings/customization#sponsors) * * * - [Report billboard](https://dev.to/report-abuse?billboard=264047) [](https://hacktoberfest.com/host?utm_source=devto&utm_medium=billboards&utm_campaign=hacktoberfest2026_pre_promo&bb=264047) ## [Bring Hacktoberfest to your city](https://hacktoberfest.com/host?utm_source=devto&utm_medium=billboards&utm_campaign=hacktoberfest2026_pre_promo&bb=264047) Hacktoberfest is back with 300+ in-person Fests and online events worldwide, all focused on building with open-source AI.
Run a Hack Day with prizes and funding, or a Meet Up with talks and community time. Applications are confirmed within a week. [Host a Fest →](https://hacktoberfest.com/host?utm_source=devto&utm_medium=billboards&utm_campaign=hacktoberfest2026_pre_promo&bb=264047) Read More  [Create template](https://dev.to/settings/response-templates) Templates let you quickly answer FAQs or store snippets for re-use. SubmitPreview [Dismiss](https://dev.to/404.html) Are you sure you want to hide this comment?
It will become hidden in your post, but will still be visible via the comment's [permalink](https://dev.to/farcrak/google-indexed-428-of-my-1432-generated-pages-and-none-of-my-2290-handwritten-ones-8m8#). Hide child comments as well Confirm For further actions, you may consider blocking this person and/or [reporting abuse](https://dev.to/report-abuse) [\\ Sentry](https://dev.to/sentry) Promoted Dropdown menu - [What's a billboard?](https://dev.to/billboards) - [Manage preferences](https://dev.to/settings/customization#sponsors) * * * - [Report billboard](https://dev.to/report-abuse?billboard=240305) [](https://blog.sentry.io/smarter-debugging-sentry-mcp-cursor/?utm_source=devto&utm_medium=paid-community&utm_campaign=smarter-debugging-mcp-cursor-blog&bb=240305) ## [Smarter debugging with Sentry MCP and Cursor](https://blog.sentry.io/smarter-debugging-sentry-mcp-cursor/?utm_source=devto&utm_medium=paid-community&utm_campaign=smarter-debugging-mcp-cursor-blog&bb=240305) No more copying and pasting error messages, logs, or trying to describe your distributed tracing setup or stack traces in chat. MCP can investigate real issues, understand their impact, and suggest fixes based on the actual production context. [Read more →](https://blog.sentry.io/smarter-debugging-sentry-mcp-cursor/?utm_source=devto&utm_medium=paid-community&utm_campaign=smarter-debugging-mcp-cursor-blog&bb=240305) 👋 Kindness is contagious Dropdown menu - [What's a billboard?](https://dev.to/billboards) - [Manage preferences](https://dev.to/settings/customization#sponsors) * * * - [Report billboard](https://dev.to/report-abuse?billboard=236879) x Dive into this thoughtful piece, beloved in the supportive DEV Community.
**Coders of every background** are invited to share and elevate our collective know-how. A sincere "thank you" can brighten someone's day—leave your appreciation below! On DEV, **sharing knowledge smooths our journey** and tightens our community bonds. Enjoyed this?
A quick thank you to the author is hugely appreciated. ### [Okay](https://dev.to/enter?state=new-user&bb=236879)  We're a place where coders share, stay up-to-date and grow their careers. [Log in](https://dev.to/enter?signup_subforem=1) [Create account](https://dev.to/enter?signup_subforem=1&state=new-user) 