{"id":6705,"date":"2022-11-02T15:02:08","date_gmt":"2022-11-02T15:02:08","guid":{"rendered":"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/"},"modified":"2022-11-02T15:02:08","modified_gmt":"2022-11-02T15:02:08","slug":"how-to-get-the-most-out-of-the-google-search-console-api-using-regex","status":"publish","type":"post","link":"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/","title":{"rendered":"How to get the most out of the Google Search Console API using regex"},"content":{"rendered":"<p>Google Search Console is an amazing tool that provides invaluable search data by real users directly from Google. While the charts and tables are friendly to work with, a large part of the data is not accessible from the UI.\u00a0<\/p>\n<p>The only way to get to this hidden data is to use the API and extract all that valuable search data that is available to you \u2013 if you know how. This is possible with regular expressions.<\/p>\n<p>Here\u2019s how you can maximize the Google Search Console API using regular expressions, according to Eric Wu, VP of Product Growth at Honey, a PayPal Company, who spoke at SMX Advanced.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_81 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Diagnosing_SEO_issues_with_GSC\" >Diagnosing SEO issues with GSC<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Overcoming_GSCs_sampling_problem\" >Overcoming GSC\u2019s sampling problem<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Getting_more_GSC_data_with_regular_expressions\" >Getting more GSC data with regular expressions<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Regex_informational_queries\" >Regex informational queries<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Regex_branded_queries\" >Regex branded queries<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Regex_and_the_GSC_API_in_action\" >Regex and the GSC API in action<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/craighordlow.com\/index.php\/2022\/11\/02\/how-to-get-the-most-out-of-the-google-search-console-api-using-regex\/#Watch_Getting_the_most_out_of_the_Google_Search_Console_API\" >Watch: Getting the most out of the Google Search Console API<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Diagnosing_SEO_issues_with_GSC\"><\/span>Diagnosing SEO issues with GSC<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Working on a website experiencing stagnant or declining growth or a core update drop?<\/p>\n<p>Most SEO professionals turn to Google Search Console (GSC) to diagnose such issues.<\/p>\n<p>(Or if resources permit, you may even use a paid tool like Ryte or build your own platform.)<\/p>\n<p>Fortunately for the SEO community, there\u2019s no shortage of Looker Studio dashboards (formerly Google Data Studio) useful for GSC analysis, including:<\/p>\n<p>Aleyda Solis\u2019 <a href=\"https:\/\/datastudio.google.com\/u\/0\/reporting\/1Fm7x1vc0vLokRhGf0WqaMd52mw7wjaSI\/page\/6zXD\" target=\"_blank\" rel=\"noopener noreferrer\">free dashboard<\/a>, which uses GSC data to easily identify potential rankings changes in recent days from the Google Core Update.\u00a0Google\u2019s <a href=\"https:\/\/searchengineland.com\/search-traffic-monitoring-dashboard-google-data-studio-383296\" target=\"_blank\" rel=\"noopener noreferrer\">search traffic monitoring dashboard<\/a>, which now pulls Discover and Google News traffic data.Hannah Butler\u2019s <a href=\"https:\/\/hanr.link\/v2-SCES\" target=\"_blank\" rel=\"noopener noreferrer\">Search Console Explorer Studio<\/a>. (And if you want to manipulate GSC data hands-on and find quick insights, you can use Butler\u2019s <a href=\"http:\/\/bit.ly\/SCE-HanR\" target=\"_blank\" rel=\"noopener noreferrer\">Search Console Explorer Sheet<\/a>.)<\/p>\n<p>Dashboards allow SEOs to look at an overview of different trends as opposed to using GSC and doing multiple clicks to get to the data you need.<\/p>\n<p>But if you\u2019re analyzing enterprise sites, you can run into some roadblocks.<\/p>\n<p>Looker Studio and Google Sheets both load slowly, especially when you\u2019re dealing with large sites.\u00a0GSC\u2019s interface has a 1,000-row export limit.GSC has a huge sampling problem. Enterprise SEO teams miss 90% of their GSC keywords, according to <a href=\"https:\/\/similar.ai\/blog\/closing-google-search-console-sampling-gap\/\" target=\"_blank\" rel=\"noopener noreferrer\">Similar.ai<\/a>. And if you know how to extract the data, you can actually get 14x the keywords.\u00a0<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Overcoming_GSCs_sampling_problem\"><\/span>Overcoming GSC\u2019s sampling problem<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Explorer for Search is another tool that you can use for GSC analysis. From Noah Learner and the team at Two Octobers, it is built with data pipelines using GSC\u2019s API which then outputs data to BigQuery (basically bypassing Google Sheets and downloading CSV files), and then visualizes information with Data Studio.<\/p>\n<p>With this, you can have confidence that you\u2019re getting to <strong>almost all<\/strong> the data.\u00a0<\/p>\n<p>There\u2019s still a caveat due to GSC\u2019s sampling problem, especially for large, ecommerce sites with lots of different categories. GSC won\u2019t necessarily show all the data that\u2019s coming in from those directories.<\/p>\n<p>After conducting various tests to get the most data out of the GSC API, the Similar.ai team discovered a way to <a href=\"https:\/\/similar.ai\/blog\/closing-google-search-console-sampling-gap\/\" target=\"_blank\" rel=\"noopener noreferrer\">close the GSC sampling gap<\/a>.<\/p>\n<p>They found that by adding more subdirectories as different profiles within your GSC dashboard, you can extract even more data as Google gives you more information at that lower level.\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>For example, if you\u2019re looking at example.com\/televisions and you add \u201ctelevisions\u201d as a subdirectory in your GSC profile, Google will give you only the keywords and the click information for that subdirectory and down.<\/p>\n<p>And by adding a lot of these different subdirectories, you can extract a lot more information.<\/p>\n<p>That solves the sampling problem, but you can get even more data by using regular expressions.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Getting_more_GSC_data_with_regular_expressions\"><\/span>Getting more GSC data with regular expressions<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Regular expression, or regex, is a powerful tool to understand your data.\u00a0<\/p>\n<p>In April 2021, Google <a href=\"https:\/\/searchengineland.com\/google-search-console-adds-regular-expression-support-and-more-data-filtering-to-performance-reports-347549\" target=\"_blank\" rel=\"noopener\">added regex support to GSC<\/a> \u2013 giving SEOs more ways to slice and dice organic search data.\u00a0<\/p>\n<p>A lot of times, data is not useful unless you can comprehend it. And regex helps to extract actionable insights from GSC\u2019s rich data.<\/p>\n<p>But as powerful as it may be, regex can be difficult to learn.\u00a0<\/p>\n<p>The best place to understand and dive deep into regular expressions is <a href=\"https:\/\/github.com\/google\/re2\/wiki\/Syntax\" target=\"_blank\" rel=\"noopener noreferrer\">Google\u2019s official documentation on GitHub<\/a>. (Google uses RE2 in its products, which is a flavor of regular expression.)<\/p>\n<p>While regex is available in all kinds of different programming languages, you\u2019ll find it almost everywhere even to those who are modifying .htaccess files.<\/p>\n<p>In the next few sections are use cases for leveraging regex for GSC.\u00a0<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Regex_informational_queries\"><\/span>Regex informational queries<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>When looking at actual informational search queries in GSC, you typically want to understand:<\/p>\n<p>How are people actually coming to your site?What questions are they extracting?<\/p>\n<p>Looking at those things from a one-off standpoint, within GSC can be difficult.\u00a0<\/p>\n<p>You\u2019re always searching for the words \u201cwhat,\u201d \u201chow,\u201d \u201cwhy\u201d and then \u201cwhen.\u201d<\/p>\n<p>There are a couple of ways to make extracting informational queries less tedious with regex.<\/p>\n<p>Daniel K. Cheung <a href=\"https:\/\/twitter.com\/danielkcheung\/status\/1524314760346365954\" target=\"_blank\" rel=\"noopener noreferrer\">shared<\/a> a regex string that will show you all queries containing \u201cwhat,\u201d \u201chow,\u201d \u201cwhy\u201d and \u201cwhen\u201d that either got a click or an impression:<\/p>\n<p>&#8220;what|how|why|when&#8221;\u00a0<\/p>\n<p>And this regex string <a href=\"https:\/\/twitter.com\/seo_notebook\/status\/1381792930197831687\" target=\"_blank\" rel=\"noopener noreferrer\">shared<\/a> by Steve Toth takes the previous example up a notch:<\/p>\n<p>^(who|what|where|when|why|how)[&#8221; &#8220;]<\/p>\n<p>You can use this string if you want to capture question-based queries that <strong>start<\/strong> with either \u201cwho,\u201d \u201cwhat,\u201d \u201cwhere,\u201d \u201cwhen,\u201d \u201cwhy\u201d and \u201chow\u201d and then <strong>followed<\/strong> by a space.\u00a0<\/p>\n<p>This is a great list to use when you\u2019re looking for any type of word that would start a question:<\/p>\n<p>are, can, can\u2019t, could, couldn\u2019t, did, didn\u2019t, do, does, doesn\u2019t, how, if, is, isn\u2019t, should, shouldn\u2019t, was, wasn\u2019t, were, weren\u2019t, what, when, where, who, whom, whose, why, will, won\u2019t, would, wouldn\u2019t<\/p>\n<p>Putting all this into regex form would look something like this:\u00a0<\/p>\n<p>^(are|can|can&#8217;t|could|couldn&#8217;t|did|didn&#8217;t|do|does|doesn&#8217;t|how|if|is|isn&#8217;t|should|shouldn&#8217;t|was|wasn&#8217;t|were|weren&#8217;t|what|when|where|who|whom|whose|why|will|won&#8217;t|would|wouldn&#8217;t)s<\/p>\n<p>In this 178-character string:<\/p>\n<p>You have the caret (^) which tells you the query needs to begin with this word:The words are separated with pipes (|) instead of commas.\u00a0All the words are wrapped in parentheses.\u00a0There\u2019s a backslash and the \u201cs\u201d (s) which denotes a space after the word.\u00a0<\/p>\n<p>This is good, but can also get tedious to do.\u00a0\u00a0<\/p>\n<p>Below, Wu simplified the previous list of words to be more regex-friendly and shorter which is ideal for copying and pasting. Maintaining it this way also helps with efficiency.\u00a0<\/p>\n<p>In the first column are the normal words and in the second column, the compressed regex.\u00a0<\/p>\n<p>For instance, the word \u201ccan\u201d uses the compressed version can(\u2018t)?.<\/p>\n<p>What the question mark indicates is that anything within the parentheses is optional. The compressed syntax allows you to cover both the word \u201ccan\u201d and \u201ccan\u2019t.\u201d\u00a0<\/p>\n<p>More interestingly, you can do this with could\/couldn\u2019t, should\/shouldn\u2019t, and would\/wouldn\u2019t where the <em>-ould<\/em> part of the words is the common base, like (c|sh|w)ould(n\u2019t)?. This short string covers all six of those cases.<\/p>\n<p>While simplifying that long list of words turned the string less readable, what\u2019s great is that it fits more into the regex field and allows you to copy-paste easier.<\/p>\n<p>^(are|can(&#8216;t)?|(c|sh|w)ould(n&#8217;t)?|did(n&#8217;t)?|do(es)?(n&#8217;t)?|how|if|is(n&#8217;t)?|was(n&#8217;t)?|were(n&#8217;t)?|wh(at|en|ere|y)who(m|se)?|will|won&#8217;t)s<\/p>\n<p>If you go a step further, you can compress it even more. In this case, Wu reduced the character count from 135 to 113 characters.\u00a0<\/p>\n<p>^(are|can(&#8216;t)?|how|if|wh(at|en|ere|y)|who(m|se)?|will|won&#8217;t|((c|sh|w)ould|did(n&#8217;t)?|do(es)?|was|is|were)(n&#8217;t)?)s<\/p>\n<p>Regular expressions can get really complicated. If you\u2019re getting a regex string from someone else and would like to disambiguate what is doing what, you can use <a href=\"https:\/\/regexper.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Regexper<\/a> to help you visualize it.\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Below you\u2019ll see a comparison of the different regex string versions. It\u2019s easier to maintain the first one, and obviously harder to maintain and read the last one.\u00a0<\/p>\n\n<div class=\"wp-block-image\">\n<\/div>\n<p>But sometimes character count really will matter especially when you have longer regular expressions.<\/p>\n<p>Regex filter limits for GSC is <a href=\"https:\/\/twitter.com\/danielwaisberg\/status\/1402979440183939074\" target=\"_blank\" rel=\"noopener noreferrer\">4,096 characters<\/a>, according to Google Search Advocate Daniel Waisberg.\u00a0<\/p>\n<p>That would seem quite a bit. However, if you have an ecommerce site and have to add domain names, subdomains or longer directories, you\u2019ll most likely hit that limit.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Regex_branded_queries\"><\/span>Regex branded queries<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Another instance where you may start hitting the regex character limit in GSC is when you use it for branded queries.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>When you think about all the different types of misspellings of a brand name that a person could type, you\u2019ll quickly run into that 4,096 character count.\u00a0For instance:<\/p>\n<p>aamaung, damsung, mamsang, sam sung, samaung, samdung, samesung, sameung, samgsung, samgung, samsang, samsaung, samsgu, samshgg, samshng, samsing, samsnug, samssung, samsu, samsuag, samsubg, samsubng, samsug, samsumg, samsumng, samsun g, samsunb, samsund, samsund, samsunh, samsunt \u2026<\/p>\n<p>This is where understanding regex helps. With this string, you can capture the brand name \u201csamsung\u201d along with misspellings:<\/p>\n<p>(s+|a|d|z)[a-zs]{1,4}m?[a-zs]{1,6}(m|u|n|g|t|h|b|v)<\/p>\n<p>A lot of times, people will misspell the middle parts of the word. But in general, they get the format and length right and you can approach your syntax this way.<\/p>\n<p>For brand query misspellings, consider the following:<\/p>\n<p><strong>Main letters<\/strong> that make up the brand query.<strong>Consonants<\/strong>.<strong>Letters surrounding hard consonants<\/strong>.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>In red are the hard consonants that people typically don\u2019t miss when they\u2019re typing in a brand name. These are the main letters that make up that particular brand.\u00a0For \u201csamsung\u201d, the \u201cs\u201d in the beginning, the \u201dm\u201d in the middle, and then \u201cn\u201d and \u201cg\u201d at the end.<\/p>\n<p>The blue letters surrounding those main consonants on the keyboard are the ones people typically mistype. In the example, around \u201cs\u201d, you see the \u201ca\u201d, \u201cd\u201d and \u201cz\u201d. (While the layout is different for international keyboards, the concept is still the same.)\u00a0<\/p>\n<p>The regex string above captures all the possible variants of \u201csamsung.\u201d <\/p>\n<p>The other major trick here is in [a-zs]{1,4}. <\/p>\n<p>In regex form, this basically says, \u201cI want to match any letter \u201ca\u201d to \u201cz\u201d, or a space, one to four times.\u201d\u00a0<\/p>\n<p>This captures all those weird misspellings that can happen in the middle of a brand query \u2013 where a person can potentially hit the same key multiple times or accidentally press space.<\/p>\n<p>Additionally, the brand name is a certain length (\u201csamsung\u201d has seven characters). People likely won\u2019t end up typing 20\u201350 characters.\u00a0<\/p>\n<p>So in this regular expression, we\u2019re guessing that between \u201cs\u201d and \u201cm\u201d in \u201csamsung,\u201d someone\u2019s going to mistype 1\u20134 characters. And then from \u201cm\u201d to \u201cg\u201d at the end, they\u2019ll mistype 1\u20136 characters, with spaces included.\u00a0<\/p>\n<p>Adding all this allows you to capture the many variations of a branded query comprehensively.<\/p>\n<p>The other thing to note is that the brand name could appear in different parts of the query.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>So we need to make sure that the brand name itself, is captured. It should either be:<\/p>\n<p>At the start of the query.In the middle of the query (thus surrounded by spaces).Or at the end of the query.<\/p>\n<p>The regular expression for this is as follows:<\/p>\n<p>(^|s)(s+|a|d|z)[a-zs]{1,4}m?[a-zs]{1,6}(m|u|n|g|t|h|b|v)(s|$)<\/p>\n<p>This captures all queries where the brand name \u201csamsung\u201d is either at the start, middle or end.<\/p>\n<p>Start of string = ^\u00a0Surrounded by spaces = sEnd of string = $<\/p>\n<p>JC Chouinard\u2019s post, <a href=\"https:\/\/www.jcchouinard.com\/regex-in-google-search-console\/\" target=\"_blank\" rel=\"noopener noreferrer\">Regular Expressions (RegEx) in Google Search Console<\/a>, dives even deeper into regex examples.\u00a0<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Regex_and_the_GSC_API_in_action\"><\/span>Regex and the GSC API in action<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Regular expressions came in useful for Wu and his team when they worked with a client that encountered traffic drops following a core update.<\/p>\n<p>After looking at the ecommerce site\u2019s different issues, they discovered that the problem resided in some product detail pages.\u00a0<\/p>\n<p>They needed to segment pagetypes for analysis in GSC. But this was a complex task because of the different URL structures for U.S. and international products.\u00a0\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>The site\u2019s international product URLs included language and country codes, whereas U.S. product URLs did not.\u00a0<\/p>\n<p>Even using regex syntax was tricky because letters and dashes exist in the product slug, categories and subcategories. Additionally, they needed to filter out the international product URLs to capture only U.S. pages.<\/p>\n<p>To get all U.S. product landing + detail pages (<strong>not <\/strong>i18n pages), they came up with the following regex strings:<\/p>\n<p><strong>Include:<\/strong> \/([^\/]+\/){1,2}p?\u00a0<\/p>\n<p><strong>Exclude:<\/strong> \/[a-zA-Z]{2}|[a-zA-Z]{2}-[a-zA-Z]{2}\/\u00a0<\/p>\n<p>Here\u2019s a breakdown:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>The team wanted to match the category, the subcategory and all the products so they included:<\/p>\n<p>Any character that\u2019s not a slash = [^\/]+1 or 2 directories = \/){1,2}Sometimes followed by a product slug = p?<\/p>\n<p>A caret (^) typically means the start of the string. But when it\u2019s inside brackets (as in [^\/]), it indicates a negation (i.e., \u201cnot anything within this box\u201d).\u00a0<\/p>\n<p>So this string \/([^\/]+\/){1,2}p? means \u201cI want any number of characters that is not a slash, leading up to a slash (which denotes the directory), and sometimes followed by the letter \u2018p\u2019 (the prefix for product slugs).\u201d\u00a0<\/p>\n<p>At the same time, the team didn\u2019t want to match the country and language combination which also contained letters and dashes, so they excluded:<\/p>\n<p>Any 2 letter directory = [a-zA-Z]{2}2 letter + 2 letter lang-country combo = [a-zA-Z]{2}-[a-zA-Z]{2}<\/p>\n<p>Creating a regular expression to match all the language and country codes on their own would be tedious because of all the possible combinations, so they were unable to approach this the way did for informational queries (where every single type of combination was excluded).\u00a0<\/p>\n<p>But even after creating these regex strings, they had a problem.\u00a0<\/p>\n<p>In Google Search Console, there\u2019s only one field to paste a regex string. You\u2019ll have to choose either <em>Matches regex <\/em><strong>or <\/strong><em>Doesn\u2019t match regex <\/em>\u2013 you can\u2019t use both at the same time.\u00a0\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>This is where the GSC API came in handy as it allows joining regex strings.<\/p>\n<p>In the <a href=\"https:\/\/developers.google.com\/webmaster-tools\/v1\/searchanalytics\/query\" target=\"_blank\" rel=\"noopener noreferrer\">Google Search Console API documentation<\/a>, there\u2019s a <em>Try it now <\/em>link. <em>\u00a0<\/em><\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Once clicked, it will open up a console that allows you to select a site and make your API request through the web view.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>But to better manage API queries, Wu recommends using Postman on the desktop or Paw (which is native to Mac).<\/p>\n<p>Postman allows you to create queries and save them for later. And if you have access to other sites, you don\u2019t have to create a new query each time. You just simply change out the site name with a variable and then make multiple requests.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Paw, on the other hand, is much easier to look through and utilize.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>To access the API, you\u2019ll need to get your API keys. (Here\u2019s a helpful <a href=\"https:\/\/www.jcchouinard.com\/how-to-get-google-search-console-api-keys\/\" target=\"_blank\" rel=\"noopener noreferrer\">tutorial<\/a> from Chouinard.)\u00a0<\/p>\n<p>Once you get this info, you\u2019ll have your client ID and client secrets, which you\u2019ll add to your OAuth 2.0 authentication within either Postman or Paw.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>From there, you\u2019ll be able to sign in with your normal account.\u00a0\u00a0<\/p>\n<p>Wu mainly made GSC API requests using the regex strings in Paw. The query is entered in the middle of the interface.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>The response from Google is similar to that of the GSC API web view. The data can then be exported for processing.<\/p>\n<p>Since the data is in JSON, the information can be messy and hard to read.\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>For this, you can use a free and open-source command-line JSON processor called <a href=\"https:\/\/stedolan.github.io\/jq\/\" target=\"_blank\" rel=\"noopener noreferrer\">JQ<\/a> to pretty-print the information.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>The data is not that useful until you get it into a spreadsheet. Pipe in the file you\u2019ve exported from Paw to JQ. Open it and then iterate over each row \u2013 saving each element so you can output them to a CSV.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Here, you\u2019ll need to convert clicks and impressions which are floats (a number that has a decimal place). Both need to be converted into strings compatible with a CSV.\u00a0<\/p>\n<p>JQ will then output the following much-simpler format.\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Next, you\u2019ll use <a href=\"https:\/\/github.com\/TomWright\/dasel\" target=\"_blank\" rel=\"noopener noreferrer\">Dasel<\/a> to take this format and then make it into a CSV.\u00a0<\/p>\n<p>And here\u2019s the end result.\u00a0<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>What\u2019s amazing for Wu\u2019s team is that they were able to use the Google Search Console API and regular expressions to:<\/p>\n<p>Filter out all the international queries and look at just the U.S. where they were having the main issues.Identify the days the site was having issues.\u00a0<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Watch_Getting_the_most_out_of_the_Google_Search_Console_API\"><\/span>Watch: Getting the most out of the Google Search Console API<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Below is the complete video of Wu\u2019s SMX Advanced presentation.<\/p>\n\n<div><\/div>\n<p>The post <a href=\"https:\/\/searchengineland.com\/google-search-console-api-regex-389034\" target=\"_blank\" rel=\"noopener\">How to get the most out of the Google Search Console API using regex<\/a> appeared first on <a href=\"https:\/\/searchengineland.com\/\" target=\"_blank\" rel=\"noopener\">Search Engine Land<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Google Search Console is an amazing tool that provides invaluable search data by real users directly from Google. While the charts and tables are friendly to work with, a large part of the data is not accessible from the UI.\u00a0 The only way to get to this hidden data is to use the API and extract all that valuable search data that is available to you \u2013 if you know how. This is possible with regular expressions. Here\u2019s how you can maximize the Google Search Console [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"h5ap_radio_sources":[],"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-6705","post","type-post","status-publish","format-standard","hentry","category-best-seo-blogs"],"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":false,"author_link":"https:\/\/craighordlow.com\/author\/"},"_links":{"self":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/6705","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/comments?post=6705"}],"version-history":[{"count":0,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/6705\/revisions"}],"wp:attachment":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/media?parent=6705"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/categories?post=6705"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/tags?post=6705"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}