Data Extraction from APIs with Python – Currency Exchange

There are several popular platforms that give developers access to their “web services”, aka “APIs” (Application Programming Interface). So using APIs is the official way for data extraction and doing other stuff allowed by such applications. You can even benefit from some APIs to build other applications. REST APIs usually generate output in JSON or XML format because most of programming languages can handle these formats easily. In fact, JSON (JavaScript Object Notation) is very similar to data types in programming languages; for example, it is very similar to Python dictionaries. If a REST API allows you to get the data you want to retrieve, then you do not need regular web scraping.

Some APIs require authentication (API Key or Client ID and Client Secret, similar to a username and password, so to speak) to control their usage, and some do not. We will explain this later in multiple APIs. For the purpose of clarifying the basics, we will start with a very simple currency rate conversion API that does not require any authentication.

In this tutorial, you will learn how to use Python to extract data from ExchangeRatesAPI.io which is -according to its official website- “a free service for current and historical foreign exchange rates published by the European Central Bank.”

API URL

Just as most REST APIs that return JSON, ExchangeRatesAPI gives you a URL with a “query string” with one “parameter” or more. Here is an example URL for finding the rates of USD and GBP in EUR: https://api.exchangeratesapi.io/latest?symbols=USD,GBP

In this URL, you have a query question mark “?” followed by one parameter which is “symbols=” and then the value you need.

If you click this URL, you will get this result:

 

As you can see in this example, in Python terms, it is like a dictionary that includes a smaller dictionary. This can get complicated in other APIs like having also lists and multiple dictionaries – again in Python terms.

 

Note: To preview the JSON output in a readable format, you can use Firefox. By default, Firefox has a built-in JSON viewer that shows JSON in a nice format once you open the URL. In the Firefox JSON viewer, you can also click the tab called “Raw Data” and then the sub-tab “Pretty Print” to see JSON in a view similar to how it is displayed above. Chrome can have the same with extensions. For more complicated tasks, you can download Postman free app, but for now the Firefox built-in JSON is just enough. So keep it simple for now and let’s continue.

Reading and Parsing the API Output with Python

 

1- To handle the API output, you need to import two Python libraries:

requests (or urllib2 or the like) to connect to the URL.

json to parse the JSON output and extract the data you need.

 

 

2- Connect to the URL as if you are opening it in browser – figuratively 😉

 

If you print(response) you will get <Response [200]> which means you are successfully connected.

 

3- Read the output:

Note: At this point, you can try to find out the type of “data” using type(data) and you will get str in Python 3 which means “string” or unicode in Python 2 which means a Unicode string. Even in some APIs, if you print(data), you will see how it is enclosed with quotes.

 

4- Parse JSON – convert the string to JSON:

 

Note: Be careful, it is “loads” not “load”.

Note: You can now try to find out the type of “parsed” using type(parsed) and you will get dict which means “dictionary”; so it was converted from a String to a Dictionary.

 

You can even have a “pretty print” to make it more readable for you. This step is not required though.

The output will look like this:

 

So now, you can deal with it as a regular dictionary.

 

5- Extract data:

 

If you want to extract “date”, just type:

 

Now you can extract the rate of GBP by accessing the main dictionary “parsed”, then sub-dictionary “rates” and finally the key “GBP” to get its value. Be careful, do not forget quotes.

 

You can extract the rate of USD in the same way:

Then try to print them:

 

So here is your whole code:

 

Get Rates of Other Currencies

 

You can use this URL to get rate of all the available currencies while the base is USA. Feel free to open it in your browser to see how it looks like https://api.exchangeratesapi.io/latest?base=USD

 

As you can see, it is a dictionary including the rates dictionary. So let’s extract the rates dictionary:

 

Try to print it if you like to see how it looks like:

 

Here is how it looks like:

 

So now to get the currencies and their rates, you need to iterate the keys and values of this dictionary using items() like this:

 

Note: In Python 3, you should use items() while in Python 2, you should use iteritems()

 

You can do it in another way:

 

 

Our full code will be as follows:

 

Note also that if you are using a recent version of the “requests” library, instead of response.text you can simply use the json() method like this:

 

Changing API URL Parameters

 

Actually, you can add the parameters in the URL you add to your code. Still, in some cases it helps to do this programmatically, which can be easily done using standard Python concatenation.

 

For example, instead of having USD as the only base, you can have a list of “base” currencies to iterate them like this:

 

Simple Currency Converter

 

Now, let’s write a code that will ask the user to enter three values: “Convert from”, “Convert to”, and “Amount”, and it will give them the conversion rate for the currencies and amount they entered:

 

• First, import the libraries, Requests and JSON.

• Then, use the input() method to take the values from the user, and pass these values to the 3 required variables.

Note 1: input() in Python 3 is raw_input() in Python 2.

Note 2: input() gets a string; that is why you have to change the data type of the variable “amount” to float to be able multiply it later by the rate.

• Concatenate the API URL with the “base” you get from the user.

Then, as usual, send your request, convert the output to a JSON object, and extract the rates into a dictionary.

• Finally, loop over the keys and values of the rates dictionary using  items() if you are using Python 3 or  iteritems() if you are using Python 2.

• Add a condition that if the currency in the dictionary key equals the currency that the user entered, then get the rate from the value of the same key and multiply it by the amount that the user entered to get the conversion rate.

• Of course, you can fine-tune this code to handle some issues like of the user entered the currency in a lowercase instead of the uppercase, or if the user entered a wrong currency code.

Now run your code and see how it works. For example, enter “USD” for “Convert from: “, “GBP” for “Convert to: ” and “30” for “Amount: “, and voila… you get the conversion rate.

 

API with Access Key

 

Now, let’s try another exchange rate service that has the same idea but requires an access key – it is Fixer.

1- Go to fixer.io – as you can see, it defines itself as a “Foreign exchange
rates and currency conversion JSON API”.

2- Click “sign up free”, then select the free plan to the left by clicking the grey button “Get Free API Key“, and fill in the form to sign up.

3- Now, on Fixer dashboard, you can see a label called “Your API Access Key” followed by a combination of numbers and strings which is the Access Key you need for your API URL to work.

4- If you move to the “3-Step Quickstart Guide”, you can see all the URLs you can use which consists of several parts:

• the main URL, which is: http://data.fixer.io/api/

• an additional part of the URL, for example “latest”, or you can find other options under the “Endpoint URLs” tabs; here you have a date like “2013-03-16” to see old conversion rates.

• finally, you have a query question mark “?”, followed by parameters, and the main parameter is your “access_key”.

So this is an example URL, and if you click it, you will see that you have a JSON output including the exchange rates:

http://data.fixer.io/api/latest?access_key=9d1f065a6a9816f9dd7667b0c9b26bab

If you want to determine the currency symbols, you can add another parameter after an ampersand mark “&” so the URL will be:

http://data.fixer.io/api/latest?access_key=9d1f065a6a9816f9dd7667b0c9b26bab&symbols=USD,GBP

This URL will show you this output:

 

So let’s try the code we previously tried, but now for Fixer with an Access Key:

 

When run the code, you should get:

… which means it works perfectly.

 

So that is it for this initial API tutorial. Now you are ready to apply what you have learnt to any other API for data extraction. We are going to publish more API tutorials soon.

 

You can learn more about APIs and Python by joining this course for free: 30 Days of Python | Unlock Your Python Potential by Justin Mitchel.