{"id":12114,"date":"2025-01-20T13:45:18","date_gmt":"2025-01-20T13:45:18","guid":{"rendered":"https:\/\/craighordlow.com\/index.php\/2025\/01\/20\/google-dont-dynamically-update-robots-txt-file-multiple-times-per-day\/"},"modified":"2025-01-20T13:45:18","modified_gmt":"2025-01-20T13:45:18","slug":"google-dont-dynamically-update-robots-txt-file-multiple-times-per-day","status":"publish","type":"post","link":"https:\/\/craighordlow.com\/index.php\/2025\/01\/20\/google-dont-dynamically-update-robots-txt-file-multiple-times-per-day\/","title":{"rendered":"Google: Don&#8217;t Dynamically Update Robots.txt File Multiple Times Per Day"},"content":{"rendered":"<p>Google&#8217;s John Mueller said that since the robots.txt file is cached by Google for about 24-hours, it does not make much sense to dynamically update your robots.txt file throughout the day to control controlling. Google won&#8217;t necessarily see that you don&#8217;t want Google to crawl a page at 7am and then at 9am you do want Google to crawl that page.<\/p>","protected":false},"excerpt":{"rendered":"<p>Google&#8217;s John Mueller said that since the robots.txt file is cached by Google for about 24-hours, it does not make much sense to dynamically update your robots.txt file throughout the day to control controlling. Google won&#8217;t necessarily see that you don&#8217;t want Google to crawl a page at 7am and then at 9am you do want Google to crawl that page.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"h5ap_radio_sources":[],"cybocfi_hide_featured_image":"","footnotes":""},"categories":[61],"tags":[],"class_list":["post-12114","post","type-post","status-publish","format-standard","hentry","category-se-roundtable"],"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":false,"author_link":"https:\/\/craighordlow.com\/author\/craig\/"},"_links":{"self":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/12114","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/comments?post=12114"}],"version-history":[{"count":0,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/12114\/revisions"}],"wp:attachment":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/media?parent=12114"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/categories?post=12114"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/tags?post=12114"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}