{"id":9007,"date":"2023-04-12T14:17:04","date_gmt":"2023-04-12T14:17:04","guid":{"rendered":"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/"},"modified":"2023-04-12T14:17:04","modified_gmt":"2023-04-12T14:17:04","slug":"5-python-scripts-for-automating-seo-tasks","status":"publish","type":"post","link":"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/","title":{"rendered":"5 Python scripts for automating SEO tasks"},"content":{"rendered":"<p>Python is a powerful programming language that has gained popularity in the SEO industry over the past few years.\u00a0<\/p>\n<p>With its relatively simple syntax, efficient performance and abundance of libraries and frameworks, Python has revolutionized how many SEOs approach their work.\u00a0<\/p>\n<p>Python offers a versatile toolset that can help make the optimization process faster, more accurate and more effective.\u00a0<\/p>\n<p>This article explores five Python scripts to help boost your SEO efforts.<\/p>\n<p><a href=\"https:\/\/searchengineland.com\/#h-script-1-automate-a-redirect-map\" target=\"_blank\" rel=\"noopener\">Automate a redirect map<\/a>.<\/p>\n<p><a href=\"https:\/\/searchengineland.com\/#h-script-2-write-meta-descriptions-in-bulk\" target=\"_blank\" rel=\"noopener\">Write meta descriptions in bulk<\/a>.<\/p>\n<p><a href=\"https:\/\/searchengineland.com\/#h-script-3-analyze-keywords-with-n-grams\" target=\"_blank\" rel=\"noopener\">Analyze keywords with N-grams<\/a>.<\/p>\n<p><a href=\"https:\/\/searchengineland.com\/#h-script-4-group-keywords-into-topic-clusters\" target=\"_blank\" rel=\"noopener\">Group keywords into topic clusters<\/a>.<\/p>\n<p><a href=\"https:\/\/searchengineland.com\/#h-script-5-match-keyword-list-to-a-list-of-predefined-topics\" target=\"_blank\" rel=\"noopener\">Match keyword list to a list of predefined topics<\/a>.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_81 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#The_easiest_way_to_get_started_with_Python\" >The easiest way to get started with Python<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Enable_file_uploads\" >Enable file uploads<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Upload_source_data\" >Upload source data<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Script_1_Automate_a_redirect_map\" >Script 1: Automate a redirect map<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#How_this_script_works\" >How this script works<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Get_the_script\" >Get the script<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Script_2_Write_meta_descriptions_in_bulk\" >Script 2: Write meta descriptions in bulk<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#How_the_script_works\" >How the script works<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Get_the_script-2\" >Get the script<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Script_3_Analyze_keywords_with_N-grams\" >Script 3: Analyze keywords with N-grams<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#How_this_script_works-2\" >How this script works<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Get_this_script\" >Get this script<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Script_4_Group_keywords_into_topic_clusters\" >Script 4: Group keywords into topic clusters<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#How_this_script_works-3\" >How this script works<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Get_this_script-2\" >Get this script<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Script_5_Match_keyword_list_to_a_list_of_predefined_topics\" >Script 5: Match keyword list to a list of predefined topics<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#How_this_script_works-4\" >How this script works<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Get_this_script-3\" >Get this script<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/04\/12\/5-python-scripts-for-automating-seo-tasks\/#Working_with_Python_for_SEO\" >Working with Python for SEO<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"The_easiest_way_to_get_started_with_Python\"><\/span>The easiest way to get started with Python<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>If you\u2019re looking to dip your toes in Python programming, Google Colab is worth considering.\u00a0<\/p>\n<p>It\u2019s a free, web-based platform that provides a convenient playground for writing and running Python code without needing a complex local setup.\u00a0<\/p>\n<p>Essentially, it allows you to access Jupyter Notebooks within your browser and provides a host of pre-installed libraries for data science and machine learning.\u00a0<\/p>\n<p>Plus, it\u2019s built on top of Google Drive, so you can easily save and share your work with others. <\/p>\n<p>To get started, follow these steps:<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Enable_file_uploads\"><\/span>Enable file uploads<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Once you open Google Colab, you\u2019ll first need to enable the ability to create a temporary file repository. It\u2019s as simple as clicking the folder icon.\u00a0<\/p>\n<p>This lets you upload temporary files and then download any results files.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"Upload_source_data\"><\/span>Upload source data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Many of our Python scripts require a source file to work. To upload a file, simply click the upload button. <\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Once you finish the setup, you can start testing the following Python scripts.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Script_1_Automate_a_redirect_map\"><\/span>Script 1: Automate a redirect map<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Creating redirect maps for large sites can be incredibly time-consuming. Finding ways to automate the process can help us save time and focus on other tasks.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"How_this_script_works\"><\/span>How this script works<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>This script focuses on analyzing the web content to find closely matching articles.\u00a0<\/p>\n<p>First, it imports two TXT files of URLs: one is for the redirected website (source_urls.txt), and the other for the site absorbing the redirected website (target_urls.txt).<\/p>\n<p>Then, we use the Python library Beautiful Soup to create a web scraper to get the main body content on the page. This script ignores header and footer content.<\/p>\n<p>After it\u2019s crawled the content on all pages, it uses the Python library Polyfuzz to match content between URLs with a similarity percentage.<\/p>\n<p>Finally, it prints the results in a CSV file, including the similarity percentage.\u00a0<\/p>\n<p>From here, you can manually review any URLs with a low similarity percentage to find the next closest match.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Get_the_script\"><\/span>Get the script<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>#import libraries<br \/>\nfrom bs4 import BeautifulSoup, SoupStrainer<br \/>\nfrom polyfuzz import PolyFuzz<br \/>\nimport concurrent.futures<br \/>\nimport csv<br \/>\nimport pandas as pd<br \/>\nimport requests<\/p>\n<p>#import urls<br \/>\nwith open(&#8220;source_urls.txt&#8221;, &#8220;r&#8221;) as file:<br \/>\n    url_list_a = [line.strip() for line in file]<\/p>\n<p>with open(&#8220;target_urls.txt&#8221;, &#8220;r&#8221;) as file:<br \/>\n    url_list_b = [line.strip() for line in file]<\/p>\n<p>#create a content scraper via bs4<br \/>\ndef get_content(url_argument):<br \/>\n    page_source = requests.get(url_argument).text<br \/>\n    strainer = SoupStrainer(&#8216;p&#8217;)<br \/>\n    soup = BeautifulSoup(page_source, &#8216;lxml&#8217;, parse_only=strainer)<br \/>\n    paragraph_list = [element.text for element in soup.find_all(strainer)]<br \/>\n    content = &#8221; &#8220;.join(paragraph_list)<br \/>\n    return content<\/p>\n<p>#scrape the urls for content<br \/>\nwith concurrent.futures.ThreadPoolExecutor() as executor:<br \/>\n    content_list_a = list(executor.map(get_content, url_list_a))<br \/>\n    content_list_b = list(executor.map(get_content, url_list_b))<\/p>\n<p>content_dictionary = dict(zip(url_list_b, content_list_b))<\/p>\n<p>#get content similarities via polyfuzz<br \/>\nmodel = PolyFuzz(&#8220;TF-IDF&#8221;)<br \/>\nmodel.match(content_list_a, content_list_b)<br \/>\ndata = model.get_matches()<\/p>\n<p>#map similarity data back to urls<br \/>\ndef get_key(argument):<br \/>\n    for key, value in content_dictionary.items():<br \/>\n        if argument == value:<br \/>\n            return key<br \/>\n    return key<\/p>\n<p>with concurrent.futures.ThreadPoolExecutor() as executor:<br \/>\n    result = list(executor.map(get_key, data[&#8220;To&#8221;]))<\/p>\n<p>#create a dataframe for the final results<br \/>\nto_zip = list(zip(url_list_a, result, data[&#8220;Similarity&#8221;]))<br \/>\ndf = pd.DataFrame(to_zip)<br \/>\ndf.columns = [&#8220;From URL&#8221;, &#8220;To URL&#8221;, &#8220;% Identical&#8221;]<\/p>\n<p>#export to a spreadsheet<br \/>\nwith open(&#8220;redirect_map.csv&#8221;, &#8220;w&#8221;, newline=&#8221;&#8221;) as file:<br \/>\n    columns = [&#8220;From URL&#8221;, &#8220;To URL&#8221;, &#8220;% Identical&#8221;]<br \/>\n    writer = csv.writer(file)<br \/>\n    writer.writerow(columns)<br \/>\n    for row in to_zip:<br \/>\n        writer.writerow(row)<a href=\"https:\/\/searchengineland.com\/\" target=\"_blank\" rel=\"noopener\"><\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Script_2_Write_meta_descriptions_in_bulk\"><\/span>Script 2: Write meta descriptions in bulk<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>While meta descriptions are not a direct ranking factor, they help us improve our organic click-through rates. Leaving meta descriptions blank increases the chances that Google will create its own.<\/p>\n<p>If your SEO audit shows a large number of URLs missing a meta description, it may be difficult to make time to write all of those by hand, especially for ecommerce websites.\u00a0<\/p>\n<p>This script is aimed to help you save time by automating that process for you.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"How_the_script_works\"><\/span>How the script works<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>First, the script imports a list of URLs from a TXT file (urls.txt).<\/p>\n<p>Then, it parses all of the content on the URLs.<\/p>\n<p>Once the content is parsed, it creates meta descriptions aiming to be below 155 characters.\u00a0<\/p>\n<p>It exports the results into a CSV file.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Get_the_script-2\"><\/span>Get the script<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>!pip install sumy<br \/>\nfrom sumy.parsers.html import HtmlParser<br \/>\nfrom sumy.nlp.tokenizers import Tokenizer<br \/>\nfrom sumy.nlp.stemmers import Stemmer<br \/>\nfrom sumy.utils import get_stop_words<br \/>\nfrom sumy.summarizers.lsa import LsaSummarizer<br \/>\nimport csv<\/p>\n<p>#1) imports a list of URLs from a txt file<br \/>\nwith open(&#8216;urls.txt&#8217;) as f:<br \/>\n    urls = [line.strip() for line in f]<\/p>\n<p>results = []<\/p>\n<p># 2) analyzes the content on each URL<br \/>\nfor url in urls:<br \/>\n    parser = HtmlParser.from_url(url, Tokenizer(&#8220;english&#8221;))<br \/>\n    stemmer = Stemmer(&#8220;english&#8221;)<br \/>\n    summarizer = LsaSummarizer(stemmer)<br \/>\n    summarizer.stop_words = get_stop_words(&#8220;english&#8221;)<br \/>\n    description = summarizer(parser.document, 3)<br \/>\n    description = &#8221; &#8220;.join([sentence._text for sentence in description])<br \/>\n    if len(description) &gt; 155:<br \/>\n        description = description[:152] + &#8216;&#8230;&#8217;<br \/>\n    results.append({<br \/>\n        &#8216;url&#8217;: url,<br \/>\n        &#8216;description&#8217;: description<br \/>\n    })<\/p>\n<p># 4) exports the results to a csv file<br \/>\nwith open(&#8216;results.csv&#8217;, &#8216;w&#8217;, newline=&#8221;) as f:<br \/>\n    writer = csv.DictWriter(f, fieldnames=[&#8216;url&#8217;,&#8217;description&#8217;])<br \/>\n    writer.writeheader()<br \/>\n    writer.writerows(results)<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Script_3_Analyze_keywords_with_N-grams\"><\/span>Script 3: Analyze keywords with N-grams<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>N-grams are not a new concept but are still useful for SEO. They can help us understand themes across large sets of keyword data.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"How_this_script_works-2\"><\/span>How this script works<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>This script outputs results in a TXT file that breaks out the keywords into unigrams, bigrams, and trigrams.\u00a0<\/p>\n<p>First, it imports a TXT file of all your keywords (keyword.txt).<\/p>\n<p>Then it uses a Python library called Counter to analyze and extract the N-grams.<\/p>\n<p>Then it exports the results in a new TXT file.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Get_this_script\"><\/span>Get this script<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>#Import necessary libraries<br \/>\nimport re<br \/>\nfrom collections import Counter<\/p>\n<p>#Open the text file and read its contents into a list of words<br \/>\nwith open(&#8216;keywords.txt&#8217;, &#8216;r&#8217;) as f:<br \/>\n    words = f.read().split()<\/p>\n<p>#Use a regular expression to remove any non-alphabetic characters from the words<br \/>\nwords = [re.sub(r'[^a-zA-Z]&#8217;, &#8221;, word) for word in words]<\/p>\n<p>#Initialize empty dictionaries for storing the unigrams, bigrams, and trigrams<br \/>\nunigrams = {}<br \/>\nbigrams = {}<br \/>\ntrigrams = {}<\/p>\n<p>#Iterate through the list of words and count the number of occurrences of each unigram, bigram, and trigram<br \/>\nfor i in range(len(words)):<br \/>\n    # Unigrams<br \/>\n    if words[i] in unigrams:<br \/>\n        unigrams[words[i]] += 1<br \/>\n    else:<br \/>\n        unigrams[words[i]] = 1<\/p>\n<p>    # Bigrams<br \/>\n    if i &lt; len(words)-1:<br \/>\n        bigram = words[i] + &#8216; &#8216; + words[i+1]<br \/>\n        if bigram in bigrams:<br \/>\n            bigrams[bigram] += 1<br \/>\n        else:<br \/>\n            bigrams[bigram] = 1<\/p>\n<p>    # Trigrams<br \/>\n    if i &lt; len(words)-2:<br \/>\n        trigram = words[i] + &#8216; &#8216; + words[i+1] + &#8216; &#8216; + words[i+2]<br \/>\n        if trigram in trigrams:<br \/>\n            trigrams[trigram] += 1<br \/>\n        else:<br \/>\n            trigrams[trigram] = 1<\/p>\n<p># Sort the dictionaries by the number of occurrences<br \/>\nsorted_unigrams = sorted(unigrams.items(), key=lambda x: x[1], reverse=True)<br \/>\nsorted_bigrams = sorted(bigrams.items(), key=lambda x: x[1], reverse=True)<br \/>\nsorted_trigrams = sorted(trigrams.items(), key=lambda x: x[1], reverse=True)<\/p>\n<p># Write the results to a text file<br \/>\nwith open(&#8216;results.txt&#8217;, &#8216;w&#8217;) as f:<br \/>\n    f.write(&#8220;Most common unigrams:n&#8221;)<br \/>\n    for unigram, count in sorted_unigrams[:10]:<br \/>\n        f.write(unigram + &#8220;: &#8221; + str(count) + &#8220;n&#8221;)<br \/>\n    f.write(&#8220;nMost common bigrams:n&#8221;)<br \/>\n    for bigram, count in sorted_bigrams[:10]:<br \/>\n        f.write(bigram + &#8220;: &#8221; + str(count) + &#8220;n&#8221;)<br \/>\n    f.write(&#8220;nMost common trigrams:n&#8221;)<br \/>\n    for trigram, count in sorted_trigrams[:10]:<br \/>\n        f.write(trigram + &#8220;: &#8221; + str(count) + &#8220;n&#8221;)<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Script_4_Group_keywords_into_topic_clusters\"><\/span>Script 4: Group keywords into topic clusters<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>With new SEO projects, keyword research is always in the early stages. Sometimes we deal with thousands of keywords in a dataset, making grouping challenging.\u00a0<\/p>\n<p>Python allows us to automatically cluster keywords into similar groups to identify trend trends and complete our keyword mapping.\u00a0<\/p>\n<h3><span class=\"ez-toc-section\" id=\"How_this_script_works-3\"><\/span>How this script works<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>This script first imports a TXT file of keywords (keywords.txt).<\/p>\n<p>Then the script analyzes the keywords using TfidfVectorizer and AffinityPropagation.<\/p>\n<p>Then it assigns a numeric value to each topic cluster.<\/p>\n<p>The results are then exported into a csv file.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Get_this_script-2\"><\/span>Get this script<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>import csv<br \/>\nimport numpy as np<br \/>\nfrom sklearn.cluster import AffinityPropagation<br \/>\nfrom sklearn.feature_extraction.text import TfidfVectorizer<\/p>\n<p># Read keywords from text file<br \/>\nwith open(&#8220;keywords.txt&#8221;, &#8220;r&#8221;) as f:<br \/>\n    keywords = f.read().splitlines()<\/p>\n<p># Create a Tf-idf representation of the keywords<br \/>\nvectorizer = TfidfVectorizer()<br \/>\nX = vectorizer.fit_transform(keywords)<\/p>\n<p># Perform Affinity Propagation clustering<br \/>\naf = AffinityPropagation().fit(X)<br \/>\ncluster_centers_indices = af.cluster_centers_indices_<br \/>\nlabels = af.labels_<\/p>\n<p># Get the number of clusters found<br \/>\nn_clusters = len(cluster_centers_indices)<\/p>\n<p># Write the clusters to a csv file<br \/>\nwith open(&#8220;clusters.csv&#8221;, &#8220;w&#8221;, newline=&#8221;&#8221;) as f:<br \/>\n    writer = csv.writer(f)<br \/>\n    writer.writerow([&#8220;Cluster&#8221;, &#8220;Keyword&#8221;])<br \/>\n    for i in range(n_clusters):<br \/>\n        cluster_keywords = [keywords[j] for j in range(len(labels)) if labels[j] == i]<br \/>\n        if cluster_keywords:<br \/>\n            for keyword in cluster_keywords:<br \/>\n                writer.writerow([i, keyword])<br \/>\n        else:<br \/>\n            writer.writerow([i, &#8220;&#8221;])<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Script_5_Match_keyword_list_to_a_list_of_predefined_topics\"><\/span>Script 5: Match keyword list to a list of predefined topics<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>This is similar to the previous script, except this allows you to match a list of keywords to a predefined set of topics.\u00a0<\/p>\n<p>This is great for large sets of keywords because it processes them in batches of 1,000 to prevent system crashes.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"How_this_script_works-4\"><\/span>How this script works<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>This script imports a keyword list (keywords.txt) and a topics list (topics.txt).<\/p>\n<p>Then it analyzes the topics and keyword lists and matches them to the closest match. If it doesn\u2019t find a match, it categorizes it as other.\u00a0<\/p>\n<p>The results are then exported into a CSV file.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Get_this_script-3\"><\/span>Get this script<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>import pandas as pd<br \/>\nimport spacy<br \/>\nfrom spacy.lang.en.stop_words import STOP_WORDS<\/p>\n<p># Load the Spacy English language model<br \/>\nnlp = spacy.load(&#8220;en_core_web_sm&#8221;)<\/p>\n<p># Define the batch size for keyword analysis<br \/>\nBATCH_SIZE = 1000<\/p>\n<p># Load the keywords and topics files as Pandas dataframes<br \/>\nkeywords_df = pd.read_csv(&#8220;keywords.txt&#8221;, header=None, names=[&#8220;keyword&#8221;])<br \/>\ntopics_df = pd.read_csv(&#8220;topics.txt&#8221;, header=None, names=[&#8220;topic&#8221;])<\/p>\n<p># Define a function to categorize a keyword based on the closest related topic<br \/>\ndef categorize_keyword(keyword):<br \/>\n    # Tokenize the keyword<br \/>\n    tokens = nlp(keyword.lower())<br \/>\n    # Remove stop words and punctuation<br \/>\n    tokens = [token.text for token in tokens if not token.is_stop and not token.is_punct]<br \/>\n    # Find the topic that has the most token overlaps with the keyword<br \/>\n    max_overlap = 0<br \/>\n    best_topic = &#8220;Other&#8221;<br \/>\n    for topic in topics_df[&#8220;topic&#8221;]:<br \/>\n        topic_tokens = nlp(topic.lower())<br \/>\n        topic_tokens = [token.text for token in topic_tokens if not token.is_stop and not token.is_punct]<br \/>\n        overlap = len(set(tokens).intersection(set(topic_tokens)))<br \/>\n        if overlap &gt; max_overlap:<br \/>\n            max_overlap = overlap<br \/>\n            best_topic = topic<br \/>\n    return best_topic<\/p>\n<p># Define a function to process a batch of keywords and return the results as a dataframe<br \/>\ndef process_keyword_batch(keyword_batch):<br \/>\n    results = []<br \/>\n    for keyword in keyword_batch:<br \/>\n        category = categorize_keyword(keyword)<br \/>\n        results.append({&#8220;keyword&#8221;: keyword, &#8220;category&#8221;: category})<br \/>\n    return pd.DataFrame(results)<\/p>\n<p># Initialize an empty dataframe to hold the results<br \/>\nresults_df = pd.DataFrame(columns=[&#8220;keyword&#8221;, &#8220;category&#8221;])<\/p>\n<p># Process the keywords in batches<br \/>\nfor i in range(0, len(keywords_df), BATCH_SIZE):<br \/>\n    keyword_batch = keywords_df.iloc[i:i+BATCH_SIZE][&#8220;keyword&#8221;].tolist()<br \/>\n    batch_results_df = process_keyword_batch(keyword_batch)<br \/>\n    results_df = pd.concat([results_df, batch_results_df])<\/p>\n<p># Export the results to a CSV file<br \/>\nresults_df.to_csv(&#8220;results.csv&#8221;, index=False)<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Working_with_Python_for_SEO\"><\/span>Working with Python for SEO<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Python is an incredibly powerful and versatile tool for SEO professionals.\u00a0<\/p>\n<p>Whether you\u2019re a beginner or a seasoned practitioner, the free scripts I\u2019ve shared in this article offer a great starting point for exploring the possibilities of Python in SEO.\u00a0<\/p>\n<p>With its intuitive syntax and vast array of libraries, Python can help you automate tedious tasks, analyze complex data, and gain new insights into your website\u2019s performance. So why not give it a try? <\/p>\n<p>Good luck, and happy coding!<\/p>\n<p>The post <a href=\"https:\/\/searchengineland.com\/python-scripts-automating-seo-tasks-395527\" target=\"_blank\" rel=\"noopener\">5 Python scripts for automating SEO tasks<\/a> appeared first on <a href=\"https:\/\/searchengineland.com\/\" target=\"_blank\" rel=\"noopener\">Search Engine Land<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Python is a powerful programming language that has gained popularity in the SEO industry over the past few years.\u00a0 With its relatively simple syntax, efficient performance and abundance of libraries and frameworks, Python has revolutionized how many SEOs approach their work.\u00a0 Python offers a versatile toolset that can help make the optimization process faster, more accurate and more effective.\u00a0 This article explores five Python scripts to help boost your SEO efforts. Automate a redirect map. Write meta descriptions in bulk. Analyze keywords with N-grams. Group keywords [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"h5ap_radio_sources":[],"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-9007","post","type-post","status-publish","format-standard","hentry","category-best-seo-blogs"],"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":false,"author_link":"https:\/\/craighordlow.com\/author\/"},"_links":{"self":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/9007","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/comments?post=9007"}],"version-history":[{"count":0,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/9007\/revisions"}],"wp:attachment":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/media?parent=9007"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/categories?post=9007"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/tags?post=9007"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}