Get JSON Data from URL in Python

How to Get JSON Data from URL in Python ?

As a Data Scientist and AI programmer, you do most of the works on the JSON data. You usually fetch the JSON data from a particular URL and visualizes it. But traversing into a JSON data is always a challenging task for beginners. In this intuition, you will know how to get JSON data from URL in python. You will learn the following things.

How to read JSON data from the URL?
Convert JSON string to a dictionary.
Traverse in the JSON response

What is JSON?

JSON stands for JavaScript Object Notation and is widely used as a data interchange format. In fact, If you build a particular JSON restful API, then you can integrate it anywhere like Mobile Apps and Web Application e.t.c.

How to read JSON data from the URL?

Reading the JSON data from the URL require urllib request package. The package urllib is a python module with inbuilt methods for the opening and retrieving XML, HTML, JSON e.t.c.
Lets define the method getResponse(url) for retrieving the HTML or JSON from a particular URL. Inside the parameter, we are passing the URL of the JSON response. The following is the full code for the getResponse() function.

def getResponse(url):
    operUrl = urllib.request.urlopen(url)
    if(operUrl.getcode()==200):
       data = operUrl.read()
    else:
       print("Error receiving data", operUrl.getcode())
    return data

First of all, you will open the URL using the urllib.request.urlopen() method. If the address is correct you will get 200 response otherwise 404 response message. Check for it using the getcode() method. If it is 200 then read entire HTML string or JSON as a string else print the error message with the code.

How to convert JSON String to Dictionary?

The above function will return the entire HTML or JSON as a string. To use it as an object in Python you have to first convert it into a dictionary. Python has a package json that handles this process. Let’s import JSON and add some lines of code in the above method. json.loads() method parse the entire JSON string and returns the JSON object. It completes the function for getting JSON response from the URL.

def getResponse(url):
    operUrl = urllib.request.urlopen(url)
    if(operUrl.getcode()==200):
        data = operUrl.read()
        jsonData = json.loads(data)
    else:
        print("Error receiving data", operUrl.getcode())
    return jsonData

You can see the after reading JSON response I am loading all the JSON data in a jsonData variable and returning it.

How to traverse inside the JSON response?

The first format the JSON data such that you can easily read and find to traverse inside the JSON. Go to the URL https://jsoneditoronline.org/ and paste the JSON response. You will know the idea of the JSON. Like in the case of our example. The URL gives a response on the Indian state name and its state id.

Format the JSON data

state name and id response

Steps

First, select the key you want to get values from the key.

Like in our example I select the first states key for traversing.

Traverse to all the JSON response using for loop.

Select the value you want to get using the key name. For example, I want to get the Indian State name then i will use jsonData[“state][“state_name”]. It means first go to the state key and then the state_name.  In the same way for state id jsonData[“state”][“state_id”].

Let’s add some more lines of code in the main function.

def main():

    urlData = "http://vocab.nic.in/rest.php/states/json"
    jsonData = getResponse(urlData)
    # print the state id and state name corresponding
    for i in jsonData["states"]:
        print(f'State Name:  {i["state"]["state_name"]} , State ID : {i["state"]["state_id"]}')

Congrats, you have successfully received the JSON response from the URL in python interpreter.  The output will be like this.

Full Code

import urllib.request
import json

def getResponse(url):
    operUrl = urllib.request.urlopen(url)
    if(operUrl.getcode()==200):
        data = operUrl.read()
        jsonData = json.loads(data)
    else:
        print("Error receiving data", operUrl.getcode())
    return jsonData

def main():

    urlData = "http://vocab.nic.in/rest.php/states/json"
    jsonData = getResponse(urlData)
    # print the state id and state name corresponding
    for i in jsonData["states"]:
        print(f'State Name:  {i["state"]["state_name"]} , State ID : {i["state"]["state_id"]}')

if __name__ == '__main__':
    main()

 

Join our list

Subscribe to our mailing list and get interesting stuff and updates to your email inbox.

Thank you for signup. A Confirmation Email has been sent to your Email Address.

Something went wrong.

 
Thank you For sharing.We appreciate your support. Don't Forget to LIKE and FOLLOW our SITE to keep UPDATED with Data Science Learner