Live Weather using BeautifulSoup

Let’s start with a basic command-line app to get weather using a simple google search. I will use the BeautifulSoup and Requests library in python for this project. The idea is simple, perform a google search using the user input to modify the search string and then parse out the result in a presentable format.

---

Importing Libraries

Let’s import some libraries which we will be using for our project.

import requests
from bs4 import BeautifulSoup

---

Setting User-Agent

This lets the server and network peers identify the application, operating system, vendor, and/or version of the requesting user agent. It’s a good practice to add this as it avoids your query from being detected as a bot.

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

---

Generating request URL

To generate a request URL, simply search within google and use that as your base URL. In my case, I tried searching for Toronto weather and used that as a base URL. We can then modify that URL to make it a little dynamic by accepting input from our users.

Few things to note here, a user can input the city name as well as the country with a “,” in between. If we look at the search URL, all the spaces are replaced by “+” and “,” by “%2C”. Also, the user should not specify “weather” in the name so we need to add that to our string as well.

So let’s try to incorporate that into our code as well.

# https://www.google.com/search?client=firefox-b-d&q=toronto+weather    -- base URL

city = input('Enter city name: ')

# Replacing ',' -> '%2C' and removing any spaces
search_string = city.replace(',', '%2C').replace(' ', '').strip()

# Adding '+weather' to our string
search_string = search_string + '+weather'

# Creating the request URL
url = f'https://www.google.com/search?client=firefox-b-d&q={search_string}'

---

Requesting results using our custom URL

We can simply use the “requests.get” method from the requests library and get the result of our custom URL. If the request is successful we should get “200” as a status code.

res = requests.get(url=url, headers=headers)

# Checking if response was successful
print(res.status_code)

Output: 
200

---

Let’s parse our data into a digestible format

Now that we have a response from our URL, let’s parse our data into a digestible format so that we can extract the data. The process is very simple, we just need to use the BeautifulSoup library for this. There are different parsers available in BeautifulSoup, we will be using the “html.parser” for this project. Also, make sure to convert the result into a text by using the “.text” method.

soup = BeautifulSoup(res.text, 'html.parser')

---

Let’s extract our data

Now we have the entire website which is generated using that search URL. We need to extract the data which we require. There are multiple ways to extract the data, and this is where things may get a little complicated. The easiest way is to inspect the website and try to find unique IDs for the things you want to extract. A little knowledge about HTML tags makes extracting data a little easier. I am going to assume that you have that knowledge and won’t dive deep into it here.

city_name = soup.select('#wob_loc')[0].getText().strip()    # City
local_time = soup.select('#wob_dts')[0].getText().strip()   # Local Time
weather = soup.select('#wob_dc')[0].getText().strip()   # Weather Description
temp_c = soup.select('#wob_tm')[0].getText().strip() + '°C' # Weather in Celcius
temp_f = soup.select('#wob_ttm')[0].getText().strip() + '°F'    # Weather in Farenheit
precipitaion = soup.select('#wob_pp')[0].getText().strip()  # Precipitation
humidity = soup.select('#wob_hm')[0].getText().strip()  # Humidity
wind_km = soup.select('#wob_ws')[0].getText().strip()   # Wind Speed in km/h
wind_m = soup.select('#wob_tws')[0].getText().strip()   # Wind Speed in mph

---

Let’s output our data in a presentable format

So now we have all the information we require. Let’s customize our output so that it looks a little official 😉

Intended Output:

Searching on Google…
City Name
Local Time
Weather Description: Weather °C | Weather °F
Precipitation
Humidity
Wind Speed (in km/h) | Wind Speed (in mph)

print(f"""
Searching on Google...\n
{city_name}
{local_time}
{weather}: {temp_c} | {temp_f}
Precipitation: {precipitaion}
Humidity: {humidity}
Wind Speed: {wind_km} | {wind_m}
""")

---

Let’s combine all the pieces into a project

# Importing Libraries
import requests
from bs4 import BeautifulSoup

# Function for getting the weather
def get_weather() -> str:
    """ Search the input from the user in google and provide with the weather """

    # Setting the header
    headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}

    # Get the city name
    city = input('Enter city name: ')

    # Generating the url
    search_string = city.replace(',', '%2C').replace(' ', '').strip() + '+weather'
    url = f'https://www.google.com/search?client=firefox-b-d&q={search_string}'

    # Requesting the result
    res = requests.get(url=url, headers=headers)

    if res.status_code == 200:
        try:
            soup = BeautifulSoup(res.text, 'html.parser')
            city_name = soup.select('#wob_loc')[0].getText().strip()    # City
            local_time = soup.select('#wob_dts')[0].getText().strip()   # Local Time
            weather = soup.select('#wob_dc')[0].getText().strip()   # Weather Description
            temp_c = soup.select('#wob_tm')[0].getText().strip() + '°C' # Weather in Celcius
            temp_f = soup.select('#wob_ttm')[0].getText().strip() + '°F'    # Weather in Farenheit
            precipitaion = soup.select('#wob_pp')[0].getText().strip()  # Precipitation
            humidity = soup.select('#wob_hm')[0].getText().strip()  # Humidity
            wind_km = soup.select('#wob_ws')[0].getText().strip()   # Wind Speed in km/h
            wind_m = soup.select('#wob_tws')[0].getText().strip()   # Wind Speed in mph

            print(f"""
                Searching on Google...\n
                {city_name}
                {local_time}
                {weather}: {temp_c} | {temp_f}
                Precipitation: {precipitaion}
                Humidity: {humidity}
                Wind Speed: {wind_km} | {wind_m}
                """)
        except:
            print(f"Sorry can't find weather for the '{city}' city.")
    else:
        print(f"Sorry can't find weather for the provided '{city}' city.")

if __name__ == '__main__':
    get_weather()

---

Final Output:

>>> get_weather()     
Enter city name: toronto

                Searching on Google...

                Toronto, ON
                Saturday 11:00 a.m.
                Sunny: 25°C | 77°F
                Precipitation: 8%
                Humidity: 48%
                Wind Speed: 13 km/h | 8 mph

---

You can find the above code on my Github Repository here.