{"id":9781,"date":"2023-06-06T14:46:01","date_gmt":"2023-06-06T14:46:01","guid":{"rendered":"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/"},"modified":"2023-06-06T14:46:01","modified_gmt":"2023-06-06T14:46:01","slug":"how-to-use-google-entities-and-gpt-4-to-create-article-outlines","status":"publish","type":"post","link":"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/","title":{"rendered":"How to use Google entities and GPT-4 to create article outlines"},"content":{"rendered":"<p>In this article, you\u2019ll learn how to use some scraping and Google\u2019s Knowledge Graph to do automated prompt engineering that generates an outline and summary for an article that, if well written, will contain many key ingredients to rank well.<\/p>\n<p>At the root of things, we\u2019re telling GPT-4 to produce an article outline based on a keyword and the top entities they have found on a well-ranking page of your choice.\u00a0<\/p>\n<p>The entities are ordered by their salience score. <\/p>\n<p>\u201cWhy salience score?\u201d you might ask.\u00a0<\/p>\n<p>Google describes salience in their <a href=\"https:\/\/cloud.google.com\/natural-language\/docs\/reference\/rest\/v1\/Entity\" target=\"_blank\" rel=\"noopener noreferrer\">API docs<\/a> as:<\/p>\n<p>\u201cThe salience score for an entity provides information about the importance or centrality of that entity to the entire document text. Scores closer to 0 are less salient, while scores closer to 1.0 are highly salient.\u201d<\/p>\n<p>Seems a pretty good metric to use to influence which entities should exist in a piece of content you might want to write, doesn\u2019t it?<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_81 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Getting_started\" >Getting started<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Using_Google_entities_and_GPT-4_to_create_article_outlines\" >Using Google entities and GPT-4 to create article outlines<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_1_Tell_me_what_you_want\" >Step 1: Tell me what you want<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_2_Installing_the_required_libraries\" >Step 2: Installing the required libraries<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_3_Authentication\" >Step 3: Authentication<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_4_Create_the_functions\" >Step 4: Create the functions<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_5_Scrape_the_URL_and_print_the_entities\" >Step 5: Scrape the URL and print the entities<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_6_Analyze_the_entities\" >Step 6: Analyze the entities<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/craighordlow.com\/index.php\/2023\/06\/06\/how-to-use-google-entities-and-gpt-4-to-create-article-outlines\/#Step_7_Generate_the_article_outline\" >Step 7: Generate the article outline<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Getting_started\"><\/span>Getting started<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There are two ways you can go about this:<\/p>\n<p>Spend about 5 minutes (maybe 10 if you need to set up your computer) and run the scripts from your machine, or\u2026<\/p>\n<p><a href=\"https:\/\/colab.research.google.com\/drive\/15MUAWBN8oSdJPHBNHFpdhV3OSMVR2GpM?usp=sharing\" target=\"_blank\" rel=\"noopener noreferrer\">Jump to the Colab<\/a> I created and start playing around right away.<\/p>\n<p>I\u2019m partial to the first, but I\u2019ve also jumped to a Colab or two in my day. <\/p>\n<p>Assuming you\u2019re still here and want to get this set up on your own machine but don\u2019t yet have Python installed or an IDE (Integrated Development Environment), I will direct you first to a quick read on <a href=\"https:\/\/wandb.ai\/onlineinference\/beginner-ml\/reports\/Beginner-ML-Installing-Acaconda-Jupyter-Notebook---Vmlldzo0MDc4NDkw\" target=\"_blank\" rel=\"noopener noreferrer\">setting your machine up to use Jupyter Notebook<\/a>. It shouldn\u2019t take more than about 5 minutes.<\/p>\n<p>Now, it\u2019s time to get going!<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Using_Google_entities_and_GPT-4_to_create_article_outlines\"><\/span>Using Google entities and GPT-4 to create article outlines<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>To make this easy to follow along, I\u2019m going to format the directions as follows:<\/p>\n<p><strong>Step<\/strong>: A brief description of the step we\u2019re on.<\/p>\n<p><strong>Code<\/strong>: The code to complete that step.<\/p>\n<p><strong>Explanation<\/strong>: A short explanation of what the code is doing.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_1_Tell_me_what_you_want\"><\/span>Step 1: Tell me what you want<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Before we dive into creating the outlines, we need to define what we want.\u00a0<\/p>\n<p>query = input (&#8220;What do you want to rank for :&#8221;)<br \/>\nprint(query)<br \/>\nurl = input(&#8220;What URL should I be inspired by : &#8220;)<br \/>\nprint(url)<\/p>\n<p>When run, this block will prompt the user (probably you) to enter the query you\u2019d like the article to rank for\/be about, as well as give you a place to put in the URL of an article you\u2019d like your piece to be inspired by.\u00a0<\/p>\n<p>I\u2019d suggest an article that ranks well, is in a format that will work for your site, and that you think is well-deserving of the rankings by the article\u2019s value alone and not just the strength of the site.<\/p>\n<p>When run, it will look like:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"Step_2_Installing_the_required_libraries\"><\/span>Step 2: Installing the required libraries<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Next, we must install all the libraries we will use to make the magic happen.<\/p>\n<p>!pip install google-cloud-language beautifulsoup4 openai<br \/>\n!pip install wandb &#8211;upgrade<br \/>\n!pip install &#8211;force-reinstall -Iv protobuf==3.20.00<\/p>\n<p>import requests<br \/>\nimport json<br \/>\nfrom bs4 import BeautifulSoup<br \/>\nfrom google.cloud import language_v1<br \/>\nfrom google.oauth2 import service_account<br \/>\nimport os<br \/>\nimport openai<br \/>\nimport pandas as pd<br \/>\nimport wandb<\/p>\n<p>We\u2019re installing the following libraries:<\/p>\n<p><strong>Requests<\/strong>: This library allows making HTTP requests to retrieve content from websites or web APIs.<\/p>\n<p><strong>JSON<\/strong>: It provides functions to work with JSON data, including parsing JSON strings into Python objects and serializing Python objects into JSON strings.<\/p>\n<p><strong>BeautifulSoup<\/strong>: This library is used for web scraping purposes. It helps in parsing and navigating HTML or XML documents and extracting relevant information from them.<\/p>\n<p><strong>Google.cloud.language_v1<\/strong>: It is a library from Google Cloud that provides natural language processing capabilities. It allows for the performing various tasks like sentiment analysis, <strong>entity recognition<\/strong>, and syntax analysis on text data.<\/p>\n<p><strong>Google.oauth2.service_account<\/strong>: This library is part of the Google OAuth2 Python package. It provides support for authenticating with Google APIs using a service account, which is a way to grant limited access to the resources of a Google Cloud project.<\/p>\n<p><strong>OS<\/strong>: This library provides a way to interact with the operating system. It allows accessing various functionalities like file operations, environment variables, and process management.<\/p>\n<p><strong>OpenAI<\/strong>: This library is the OpenAI Python package. It provides an interface to interact with OpenAI\u2019s language models, including GPT-4 (and 3). It allows developers to generate text, perform text completions, and more.<\/p>\n<p><strong>Pandas<\/strong>: It is a powerful library for data manipulation and analysis. It provides data structures and functions to efficiently handle and analyze structured data, such as tables or CSV files.<\/p>\n<p><strong>WandB<\/strong>: This library stands for \u201cWeights &amp; Biases\u201d and is a tool for experiment tracking and visualization. It helps log and visualize the metrics, hyperparameters, and other important aspects of machine learning experiments.<\/p>\n<p>When run, it looks like this:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p><!-- START INLINE FORM --><\/p>\n<div class=\"nl-inline-form border py-2 px-1 my-2\">\n<div class=\"row align-items-center\">\n<div class=\"col-12 col-lg-3 col-xl-4 pe-md-0 pb-2 pb-lg-0\">\n<p class=\"inline-form-text text-center mb-0\">Get the daily newsletter search marketers rely on.<\/p>\n<\/div>\n<div class=\"col-12 col-lg-7 col-xl-6  pe-lg-0\">\n<div class=\"form-nl-inline\">\n<div class=\"row g-0\">\n<div class=\"form-group col-12 col-lg-8 mb-0\"><\/div>\n<div class=\"col-12 col-lg-4 col-xl-4 mb-0\">\n<div class=\"d-none\">\n<p>Processing\u2026Please wait.<\/p>\n<\/div>\n<p>\t\t\t\t\t\t\t<button class=\"btn btn-dark inlineButton w-100 rounded-0 font-weight-bold\">SUBSCRIBE<\/button>\n\t\t\t\t\t\t<\/p><\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"col-12 col-lg-2 col-xl-2 ps-1\">\n<p class=\"text-center mb-0\"><a class=\"nl-terms\" href=\"https:\/\/searchengineland.com\/termsofuse\" target=\"_blank\" rel=\"noopener noreferrer\">See terms.<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><!-- END INLINE FORM --><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_3_Authentication\"><\/span>Step 3: Authentication<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>I\u2019m going to have to sidetrack us for a moment to head off and get our authentication in place. We will need an OpenAI API key and Google Knowledge Graph Search credentials.<\/p>\n<p>This will only take a few minutes.<\/p>\n<p><strong>Getting your OpenAI API<\/strong><\/p>\n<p>At present, you likely need to<a href=\"https:\/\/openai.com\/waitlist\/gpt-4-api\" target=\"_blank\" rel=\"noopener noreferrer\"> join the<\/a> waitlist. I&#8217;m lucky to have access to the API early, and so I am writing this to help you get set up as soon as you get it.<\/p>\n<p>The signup images are from GPT-3 and will be updated for GPT-4 once the flow is available to all.<\/p>\n<p>Before you can use GPT-4, you&#8217;ll need an API key to access it.<\/p>\n<p>To get one, simply head over to <a href=\"https:\/\/openai.com\/product\" target=\"_blank\" rel=\"noopener noreferrer\">OpenAI&#8217;s product page<\/a>, and click <em>Get started<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Choose your signup method (I chose Google) and run through the verification process. You&#8217;ll need access to a phone that can receive texts for this step.<\/p>\n<p>Once that&#8217;s complete, you&#8217;ll create an API key. This is so OpenAI can connect your scripts to your account. <\/p>\n<p>They must know who&#8217;s doing what and determine if and how much they should charge you for what you&#8217;re doing.<\/p>\n<p><strong>OpenAI pricing<\/strong><\/p>\n<p>Upon signing up, you get a $5 credit which will get you surprisingly far if you&#8217;re just experimenting.<\/p>\n<p>As of this writing, the pricing past that is:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p><strong>Creating your OpenAI key<\/strong><\/p>\n<p>To create your key, click on your profile in the top right and choose <em>View API keys<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>&#8230;and then you&#8217;ll create your key.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Once you close the lightbox, you can&#8217;t view your key and will have to recreate it, so for this project, simply copy it to a Notepad doc to use shortly.<\/p>\n<p><strong>Note:<\/strong> Don&#8217;t save your key (a Notepad doc on your desktop is not highly secure). Once you&#8217;ve used it momentarily, close the Notepad doc without saving it.<\/p>\n<p><strong>Getting your Google Cloud authentication<\/strong><\/p>\n<p>First, you\u2019ll need to log in to your Google account. (You&#8217;re on an SEO site, so I assume you have one. )<\/p>\n<p>Once you\u2019ve done that, you can review the <a href=\"https:\/\/developers.google.com\/knowledge-graph\" target=\"_blank\" rel=\"noopener noreferrer\">Knowledge Graph API info<\/a> if you feel so inclined or <a href=\"https:\/\/console.cloud.google.com\/apis\/dashboard?_ga=2.269169907.32958462.1685586945-205021193.1685586339&amp;_gac=1.229650670.1685586339.Cj0KCQjw4NujBhC5ARIsAF4Iv6eKzSpGvyn0py4FuArYWHGqsfDbnB24Pl71RVsVX-4EWbz6lbKtLAAaAmU1EALw_wcB\" target=\"_blank\" rel=\"noopener noreferrer\">jump right to the API Console<\/a> and get going.<\/p>\n<p>Once you\u2019re at the console:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Name it something like \u201cDave\u2019s Awesome Articles.\u201d You know\u2026 easy to remember.<\/p>\n<p>Next, you\u2019ll enable the API by clicking <em>Enable APIs and services<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Find the Knowledge Graph Search API, and enable it.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>You\u2019ll then be taken back to the main API page, where you can create credentials:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>And we\u2019ll be creating a service account.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Simply create a service account:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Fill in the required information:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>(You\u2019ll need to give it a name and grant it owner privileges.)<\/p>\n<p>Now we have our service account. All that\u2019s left is to create our key.\u00a0<\/p>\n<p>Click the three dots under <em>Actions<\/em> and click <em>Manage keys<\/em>.<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>Click <em>Add key<\/em> then <em>Create new key<\/em>:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>The key type will be JSON.<\/p>\n<p>Immediately, you\u2019ll see it download to your default download location.<\/p>\n<p>This key will give access to your APIs, so keep it safe, just like your OpenAI API.<\/p>\n<p>Alright\u2026 and we\u2019re back. Ready to continue with our script?<\/p>\n<p>Now that we have them, we need to define our API key and path to the downloaded file. The code to do this is:<\/p>\n<p>os.environ[&#8216;GOOGLE_APPLICATION_CREDENTIALS&#8217;] = &#8216;\/PATH-TO-FILE\/FILENAME.JSON&#8217;<br \/>\n%env OPENAI_API_KEY=YOUR_OPENAI_API_KEY<br \/>\nopenai.api_key = os.environ.get(&#8220;OPENAI_API_KEY&#8221;)<\/p>\n<p>You will replace YOUR_OPENAI_API_KEY with your own key.<\/p>\n<p>You will also replace \/PATH-TO-FILE\/FILENAME.JSON with the path to the service account key you just downloaded, including the file name.<\/p>\n<p>Run the cell and you\u2019re ready to move on.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_4_Create_the_functions\"><\/span>Step 4: Create the functions<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Next, we\u2019ll create the functions to:<\/p>\n<p>Scrape the webpage we entered above.<\/p>\n<p>Analyze the content and extract the entities.<\/p>\n<p>Generate an article using GPT-4.<\/p>\n<p>#The function to scrape the web page<br \/>\ndef scrape_url(url):<br \/>\n    response = requests.get(url)<br \/>\n    soup = BeautifulSoup(response.content, &#8220;html.parser&#8221;)<br \/>\n    paragraphs = soup.find_all(&#8220;p&#8221;)<br \/>\n    text = &#8221; &#8220;.join([p.get_text() for p in paragraphs])<br \/>\n    return text<\/p>\n<p>#The function to pull and analyze the entities on the page using Google&#8217;s Knowledge Graph API<br \/>\ndef analyze_content(content):<br \/>\n    client = language_v1.LanguageServiceClient()<br \/>\n    response = client.analyze_entities(<br \/>\n        request={&#8220;document&#8221;: language_v1.Document(content=content, type_=language_v1.Document.Type.PLAIN_TEXT), &#8220;encoding_type&#8221;: language_v1.EncodingType.UTF8}<br \/>\n    )<br \/>\n    top_entities = sorted(response.entities, key=lambda x: x.salience, reverse=True)[:10]<br \/>\n    for entity in top_entities:<br \/>\n        print(entity.name)<br \/>\n    return top_entities<\/p>\n<p>#The function to generate the content<br \/>\ndef generate_article(content):<br \/>\n    openai.api_key = os.environ[&#8220;OPENAI_API_KEY&#8221;]<br \/>\n    response = openai.ChatCompletion.create(<br \/>\n        messages = [{&#8220;role&#8221;: &#8220;system&#8221;, &#8220;content&#8221;: &#8220;You are a highly skilled writer, and you want to produce articles that will appeal to users and rank well.&#8221;},<br \/>\n                  {&#8220;role&#8221;: &#8220;user&#8221;, &#8220;content&#8221;: content}],<br \/>\n        model=&#8221;gpt-4&#8243;,<br \/>\n        max_tokens=1500, #The maximum with GPT-3 is 4096 including the prompt<br \/>\n        n=1, #How many results to produce per prompt<br \/>\n        #best_of=1 #When n&gt;1 completions can be run server-side and the &#8220;best&#8221; used<br \/>\n        stop=None,<br \/>\n        temperature=0.8 #A number between 0 and 2, where higher numbers add randomness<br \/>\n)<br \/>\n    return response.choices[0].message.content.strip()<\/p>\n<p>This is pretty much exactly what the comments describe. We\u2019re creating three functions for the purposes outlined above.<\/p>\n<p>Keen eyes will notice:<\/p>\n<p>messages = [{&#8220;role&#8221;: &#8220;system&#8221;, &#8220;content&#8221;: &#8220;You are a highly skilled writer, and you want to produce articles that will appeal to users and rank well.&#8221;},<\/p>\n<p>You can edit the content (You are a highly skilled writer, and you want to produce articles that will appeal to users and rank well.) and describe the role you want ChatGPT to take. You can also add tone (e.g., \u201cYou are a friendly writer \u2026\u201d).<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_5_Scrape_the_URL_and_print_the_entities\"><\/span>Step 5: Scrape the URL and print the entities<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Now we\u2019re getting our hands dirty. It\u2019s time to:<\/p>\n<p>Scrape the URL we entered above.<\/p>\n<p>Pull all the content that lives within paragraph tags.<\/p>\n<p>Run it through Google Knowledge Graph API.<\/p>\n<p>Output the entities for a quick preview.\u00a0<\/p>\n<p>Basically, you want to see anything at this stage. If you see nothing, check a different site.<\/p>\n<p>content = scrape_url(url)<br \/>\nentities = analyze_content(content)<\/p>\n<p>You can see that line one calls the function that scrapes the URL we first entered. The second line analyzes the content to extract the entities and key metrics.\u00a0<\/p>\n<p>Part of the analyze_content function also prints a list of the entities found for quick reference and verification.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_6_Analyze_the_entities\"><\/span>Step 6: Analyze the entities<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>When I first started playing around with the script, I started with 20 entities and quickly discovered that\u2019s usually too many. But is the default (10) right?<\/p>\n<p>To find out, we\u2019ll write the data to W&amp;B Tables for easy assessment. It\u2019ll keep the data indefinitely for future evaluation.\u00a0<\/p>\n<p>First, you&#8217;ll need to take about 30 seconds to sign up. (Don&#8217;t worry, it\u2019s free for this type of thing!) You can do so at https:\/\/wandb.ai\/site.<\/p>\n<p>Once you\u2019ve done that, the code to do this is:<\/p>\n<p>run = wandb.init(project=&#8221;Article Summary With Entities&#8221;)<br \/>\ncolumns=[&#8220;Name&#8221;, &#8220;Salience&#8221;]<br \/>\nent_table = wandb.Table(columns=columns)<\/p>\n<p>for entity in entities:<br \/>\n    ent_table.add_data(entity.name, entity.salience)<\/p>\n<p>run.log({&#8220;Entity Table&#8221;: ent_table})<br \/>\nwandb.finish()<\/p>\n<p>When run, the output looks like this:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>And when you click the link to view your run, you\u2019ll find:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>You can see a drop in salience score. Remember that this score calculates how important that term is to the page, not the query.<\/p>\n<p>When reviewing this data, you can choose to adjust the number of entities based on salience, or just when you see irrelevant terms pop up.<\/p>\n<p>To adjust the number of entities, you\u2019d head to the functions cell and edit:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>You\u2019ll then need to run the cell again and the one you ran to scrape and analyze the content to use the new entity count.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Step_7_Generate_the_article_outline\"><\/span>Step 7: Generate the article outline<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>The moment you&#8217;ve all been waiting for, it\u2019s time to generate the article outline.<\/p>\n<p>This is done in two parts. First, we need to generate the prompt by adding the cell:<\/p>\n<p>entity_names = [entity.name for entity in entities]<br \/>\ngpt_prompt = f&#8221;Create an outline for an article about {query} that includes the following entities: {&#8216;, &#8216;.join(entity_names)}.&#8221;<br \/>\nprint(gpt_prompt)<\/p>\n<p>This essentially creates a prompt to generate an article:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>And then, all that\u2019s left is to generate the article outline using the following:<\/p>\n<p>generated_article = generate_article(gpt_prompt)<br \/>\nprint(generated_article)<\/p>\n<p>Which will produce something like:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>And if you\u2019d also like to get a summary written up, you can add:<\/p>\n<p>gpt_prompt2 = f&#8221;Write an article summary about {query} for an article with an outline of: {generated_article}.&#8221;<br \/>\ngenerated_article = generate_article(gpt_prompt2)<br \/>\nprint(generated_article)<\/p>\n<p>Which will produce something like:<\/p>\n<div class=\"wp-block-image\">\n<\/div>\n<p>The post <a href=\"https:\/\/searchengineland.com\/google-entities-gpt-4-create-article-outlines-427882\" target=\"_blank\" rel=\"noopener\">How to use Google entities and GPT-4 to create article outlines<\/a> appeared first on <a href=\"https:\/\/searchengineland.com\/\" target=\"_blank\" rel=\"noopener\">Search Engine Land<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>In this article, you\u2019ll learn how to use some scraping and Google\u2019s Knowledge Graph to do automated prompt engineering that generates an outline and summary for an article that, if well written, will contain many key ingredients to rank well. At the root of things, we\u2019re telling GPT-4 to produce an article outline based on a keyword and the top entities they have found on a well-ranking page of your choice.\u00a0 The entities are ordered by their salience score. \u201cWhy salience score?\u201d you might ask.\u00a0 Google [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"h5ap_radio_sources":[],"cybocfi_hide_featured_image":"","footnotes":""},"categories":[1],"tags":[],"class_list":["post-9781","post","type-post","status-publish","format-standard","hentry","category-best-seo-blogs"],"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":false,"author_link":"https:\/\/craighordlow.com\/author\/"},"_links":{"self":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/9781","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/comments?post=9781"}],"version-history":[{"count":0,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/posts\/9781\/revisions"}],"wp:attachment":[{"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/media?parent=9781"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/categories?post=9781"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/craighordlow.com\/index.php\/wp-json\/wp\/v2\/tags?post=9781"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}