{"id":9139,"date":"2023-04-20T13:52:22","date_gmt":"2023-04-20T13:52:22","guid":{"rendered":"https:\/\/craighordlow.com\/index.php\/2023\/04\/20\/search-the-15-7-million-websites-in-googles-c4-dataset\/"},"modified":"2023-04-20T13:52:22","modified_gmt":"2023-04-20T13:52:22","slug":"search-the-15-7-million-websites-in-googles-c4-dataset","status":"publish","type":"post","link":"https:\/\/craighordlow.com\/index.php\/2023\/04\/20\/search-the-15-7-million-websites-in-googles-c4-dataset\/","title":{"rendered":"Search the 15.7 million websites in Google\u2019s C4 dataset"},"content":{"rendered":"

Was your website or content used to help train AI systems as part of Google\u2019s C4 dataset? A new search tool from the Washington Post lets you find out.<\/p>\n

Why we care.<\/strong> The dataset includes the types of websites and content creators that generative AI could potentially negatively impact or even wipe out, such as news and media publishers, blogs and marketing. <\/p>\n

Search.<\/strong> The new search tool can be found in the Post\u2019s article Inside the secret list of websites that make AI like ChatGPT sound smart<\/a>. It created the list \u201cbased on how many \u2018tokens\u2019 appeared from each in the data set. Tokens are small bits of text used to process disorganized information \u2014 typically a word or phrase,\u201d the story explained.<\/p>\n

For example, Search Engine Land was used.<\/p>\n

As were Marketing Land (a brand that no longer exists, but did in 2019) and Marketing Land Events, which hosted our SMX and MarTech conference sites.<\/p>\n

And Search Engine Land\u2019s parent company site, Third Door Media.<\/p>\n

Also, Barry Schwartz\u2019s Search Engine Roundtable was used.<\/p>\n

\n

@kevinschaul<\/a> and @dataviz_szuyu<\/a> did all the hard work and built this great search tool for sites. A bunch of us already found their old personal blogs. Hope you’ll find the rankings as fascinating as I did https:\/\/t.co\/xckLl15ZaS<\/a> pic.twitter.com\/7Q7zmzDC6w<\/a><\/p>\n

\u2014 Nitasha Tiku @nitashatiku@mastodon.social (@nitashatiku) April 19, 2023<\/a>\n<\/p><\/div>\n

Only part of the data.<\/strong> As a reminder, the C4 (which stands for Colossal Clean Crawled Corpus) is only part of the data used by Google Bard<\/a> and other large language models. It also uses Wikipedia, Reddit and other sources.<\/p>\n

Speaking of Reddit.<\/strong> Reddit wants to get paid when any companies want to use its data to train AI models, the New York Times<\/a> reported. Reddit has updated its API terms<\/a> and will now charge some companies (e.g., Google, OpenAI) for access. Said Reddit CEO and co-founder Steve Huffman:<\/p>\n

\u201cThe Reddit corpus of data is really valuable. But we don\u2019t need to give all of that value to some of the largest companies in the world for free. Crawling Reddit, generating value and not returning any of that value to our users is something we have a problem with. It\u2019s a good time for us to tighten things up.\u201d<\/p>\n

Ironically, Reddit, itself, didn\u2019t even create any of that value. Its users did.<\/p>\n

The post Search the 15.7 million websites in Google\u2019s C4 dataset<\/a> appeared first on Search Engine Land<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"

Was your website or content used to help train AI systems as part of Google\u2019s C4 dataset? A new search tool from the Washington Post lets you find out. Why we care. The dataset includes the types of websites and content creators that generative AI could potentially negatively impact or even wipe out, such as news and media publishers, blogs and marketing. Search. The new search tool can be found in the Post\u2019s article Inside the secret list of websites that make AI like ChatGPT sound […]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"h5ap_radio_sources":[],"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-9139","post","type-post","status-publish","format-standard","hentry","category-best-seo-blogs"],"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":false,"author_link":"https:\/\/craighordlow.com\/author\/"},"_links":{"self":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/9139","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/comments?post=9139"}],"version-history":[{"count":0,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/9139\/revisions"}],"wp:attachment":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/media?parent=9139"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/categories?post=9139"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/tags?post=9139"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}