Google: Cloudflare Content Signals Robots.txt Directive Has No Effects

John Mueller from Google said that the new Content Signals robots.txt directive invented by Cloudflare last year has “no effects whatsoever for any crawler or LLM.” He added that it “just adds bloat and future maintenance to your robots.txt file.” As far as he knows, he said, none of the crawlers or LLMs use the content-signal robots.txt directives.

Related Posts

Semrush To Be Acquired By Adobe

  • 2

Semrush will be acquired by Adobe, the two companies announced today. “Adobe and Semrush announced that they have entered into a definitive agreement under which Adobe will acquire Semrush, a leading brand visibility platform, in an all-cash transaction for $12.00 per share, representing a total equity value of approximately $1.9 billion,” the press release said.

Daily Search Forum Recap: July 30, 2024

  • 0

Here is a recap of what happened in the search forums today, through the eyes of the Search Engine Roundtable and other search forums on the web. Google Search Console has a bug with the search appearance filters…