2026-05-24
Search Discovery Expansion
Added low-cost discovery surfaces that help search engines, feed readers, and AI crawlers find the public portfolio before there is enough traffic data to filter by quality.
Hypothesis
If the site exposes XML/text sitemaps, RSS/Atom feeds, robots sitemap references, AI-readable context, and IndexNow notifications, more crawlers can discover updates earlier than relying on one sitemap alone.
Action
Added /search-discovery, /sitemap.txt, /feed.xml, /rss.xml, /atom.xml, an IndexNow key file, and a sitemap-driven IndexNow submit script; robots.txt now advertises the discovery files.
Result
The site now has multiple public discovery paths ready for Google, Bing, Yandex, feed readers, and AI crawlers. IndexNow submission status is recorded separately after the deployed key file is verified.
Next test
Submit the discovery endpoints in webmaster tools, then compare discovered URL counts, crawl dates, and first impressions after search engines refresh.
Evidence
- New route: https://aigrowthbench.com/search-discovery.
- New feed routes: /feed.xml, /rss.xml, and /atom.xml.
- New fallback sitemap: /sitemap.txt.
- New IndexNow key file at the site root.
- robots.txt advertises sitemap.xml, sitemap.txt, feed.xml, and atom.xml.
Checked URLs
- https://aigrowthbench.com/search-discovery
- https://aigrowthbench.com/sitemap.xml
- https://aigrowthbench.com/sitemap.txt
- https://aigrowthbench.com/feed.xml
- https://aigrowthbench.com/atom.xml
- https://aigrowthbench.com/robots.txt
Wins
- Discovery no longer depends only on /sitemap.xml.
- The public discovery page makes the SEO operations layer visible as portfolio evidence.
- IndexNow is available without storing any API key, password, cookie, or session token.
- RSS and Atom feeds create another route for Google-compatible feed submission and feed readers.
Risks
- Discovery does not guarantee indexing, ranking, or useful traffic.
- Bing, Yandex, and Google webmaster dashboards still require Alex's logged-in accounts for verification and monitoring.
- IndexNow should be used for new or meaningfully updated URLs, not spammed repeatedly.
Next
- Submit sitemap.xml, feed.xml, and atom.xml in Google Search Console.
- Add the domain in Bing Webmaster Tools and check the IndexNow tab after submission.
- Add the domain in Yandex Webmaster if Yandex visibility is useful for this portfolio.