Google’s AI crawler, Google-Extended, the crawler it uses for Gemini and other related AI services, does render itself and can render JavaScript, just like Googlebot, Google’s main web search crawler.
Related Posts
Google Manual Actions For News & Discover Policy Violations Updated
A few months ago, Google quietly updated the manual action policy violations list for News and Discover policy violations. Google removed the adult-theme and artificial freshening sections and then added a slew of deceptive practices sections, while also updating the hateful content section.
Google Tests Discover More Places & Explore Places Nearby
Google is testing more local search features and sections including a section in the local finder named “Discover more places” and a local search box in web search named “Explore places nearby.”
Google Adds LLMs.txt To Search Developer Docs Portal
Google has been saying that no one uses the LLMs.txt file, that Google won’t use it, that it can be useless, and you probably should noindex it if you do use it. Well, Google itself uploaded an LLMs.txt file for the Google Search Central portal.
