reza-rostampisheh-mE4Q1WHerkc-unsplash.jpg

Guide

Building a Google Trends Report using Python

Khuyen Tran

Khuyen Tran

When you search for something on Google, have you ever wanted to dive deeper into the statistics and trends of the keyword? This guide shows you how to use Python to create and publish an interactive report.

Motivation

When you search for something on Google, have you ever wanted to dive deeper into the statistics of a certain keyword?

For example,

  • What are the most popular related queries? 'Food near me', 'Chinese food', 'fast food', 'dog food'?
  • How much has the interest on your query changed over time?
  • Which countries or states is the keyword most popular in?

Gaining the answers to these questions will give you real, data-driven insights into who cares about your topic and how it's changing over time. It can also be helpful for brainstorming marketing campaigns or new products.

If you are comfortable writing Python, you have everything in your toolkit to analyse and visualise the data already.

But how do you get this dataset? You might think that finding the answer could be a manual process of working with Google's APIs. Luckily, there is a tool that allows you to answer all the questions above in a couple of lines of codes without writing your own scraping data and creating complex functions! It is called pytrends.

pytrends is an unofficial API for Google Trends. It allows a simple interface for automating the downloading of data from Google Trends.

In this article, I will show you how to use pytrends to analyze keywords and generate a shareable report for your analyses like this:

The code for this repository can be found here. Use this Binder link if you would like to quickly run the code on the cloud.

Getting started

There are four Python libraries which I will use for this article:

  • pytrends: to get the data with Google API and analyze the data
  • Plotly for high-level visualization
  • Folium to plot the data on a map
  • Datapane to build your reports and share your results

First, make sure to create a free account and sign in Datapane to get your token to create the report

!pip install -U pytrend datapane
!datapane login --server=https://datapane.com/ --token=<token>

Next, we will choose the location of the data to be the US and time zone to be US CST (tz=360). Since I am interested in food, I will choose the keyword to be food. Feel free to change the keyword to anything that you are interested in!

  1. import pandas as pd
  2. from pytrends.request import TrendReq
  3. import altair as alt
  4. import plotly.express as px
  5. pytrends = TrendReq(hl='en-US', tz=360)
  6. keyword = 'food'
  7. pytrends.build_payload([keyword], cat=0, timeframe='today 5-y', geo='', gprop='')

Related Queries

What are the related queries to the keyword 'food'? More importantly, which ones among those queries are the most popular ones? We can find that out with pytrends.related_queries()

  1. top_queries = pytrends.related_queries()[keyword]['top']
  2. fig = px.bar(top_queries,
  3. x='query',
  4. y='value')
  5. fig.show()

Neat! The top 5 most common keywords related to food are:

  • food near me
  • Chinese food
  • fast food
  • dog food
  • food delivery

It makes sense that these queries are the most common, as many people will search for food near me to check out the nearest restaurants to their location. Many Americans enjoy Chinese food, and of course, fast food as well.

Many Americans also have dogs as pets and like the convenience of food delivery.

Interest Over Time

How has the interest in the food has changed over the last 5 years? Let's investigate this by looking at the search volume.

  1. r_o_t = pytrends.interest_over_time().reset_index()
  2. r_o_t.rename(columns={keyword : 'search_volume'}, inplace=True)
  3. r_o_t_plot = alt.Chart(r_o_t).encode(x='date', y='search_volume').mark_area(line=True).interactive().properties(width=700)
  4. r_o_t_plot

The search volume has gone up and down over the years, but overall, the volume has gone up!

Geographical Interest

Which countries are most interested in food? This could be easily found with pytrends.interest_by_region

  1. regions = pytrends.interest_by_region(resolution='COUNTRY', inc_geo_code=True)
  2. regions.rename(columns={keyword : 'search_volume'}, inplace=True)

This allows us to observe the top 20 countries that have the highest search volume.

  1. bar = px.bar(regions.reset_index().sort_values(by='search_volume', ascending=False)[:20],
  2. x='geoName',
  3. y='search_volume')
  4. bar.show()

The top 5 countries with the highest search volume on food are:

  • Singapore
  • United States
  • New Zealand
  • Australia
  • Canada

These are all the most developed English-speaking countries in the world. Maybe that has something to do with the number of search volume?

Let's take a different look at this data by using choropleth map. We will use Folium, a Python visualization library, to create an interactive map.

Choropleth can be easily created by binding the data between Pandas DataFrames/Series and Geo/TopoJSON geometries like below

  1. import pandas as pd
  2. import folium
  3. url = 'https://raw.githubusercontent.com/python-visualization/folium/master/examples/data/world-countries.json'
  4. m = folium.Map()
  5. folium.Choropleth(
  6. geo_data=url,
  7. name='choropleth',
  8. data=regions.reset_index(),
  9. columns=['geoName', 'search_volume'],
  10. key_on='properties.name',
  11. fill_opacity=0.7,
  12. line_opacity=0.2,
  13. legend_name='Search volume'
  14. ).add_to(m)
  15. folium.LayerControl().add_to(m)
  16. m

Cool!

Create a Report and Share it with your Friends!

Now we have created interesting plots on Google Trend, it is time to create an interactive report and share it with your company or community. We will use Datapane, a Python tool to share Python analyses, to create and publish the report to the web.

  1. import datapane as dp
  2. from datetime import date
  3. your_name = 'Khuyen'
  4. r = [
  5. f"""
  6. # Search volume analysis for {keyword}
  7. _Analysis built by {your_name} on {date.today()}_
  8. ## Related queries
  9. """,
  10. dp.Plot(fig),
  11. '## Top 20 Countries',
  12. dp.Plot(bar),
  13. "## Geographical interest",
  14. dp.Plot(m),
  15. "## Change over time",
  16. dp.Plot(r_o_t_plot),
  17. "## Dataset",
  18. dp.Table(r_o_t)
  19. ]
  20. dp.Report(*r).publish(name=f"google_trend")

Now a link with all the plots in this article is generated and ready to share or embed.

Conclusion

Congratulations! You have learned how to use pytrends to get insights into the Google trends your keywords. The best way to learn a new concept is to create something with it. Change and run the code here to create your own Google trend report on your favourite keyword.

Need to share Python analyses?

Datapane is an API and framework which makes it easy for people analysing data in Python to publish interactive reports and deploy their analyses.

© 2020 Datapane. All rights reserved. Terms of Service

0.7.4 (v0.7.4-3-gf71aa8d8)