Extracting Facebook Posts & Comments with BeautifulSoup & Requests

Facebook is the biggest social network of our times, containing a lot of valuable data that can be useful in so many cases. Imagine being able to extract this data and use it as your project’s dataset.

In this tutorial, you are going to use Python to extract data from any Facebook profile or page. The data that you will be extracting from a predefined amount of posts is:

  • Post URL
  • Post text
  • Post media URL

You will be extracting comments from posts as well and from each comment:

  • Profile name
  • Profile URL
  • Comment text

Continue reading “Extracting Facebook Posts & Comments with BeautifulSoup & Requests”

Scraping Tweets and Performing Sentiment Analysis

Sentiment Analysis is a special case of text classification where users’ opinions or sentiments regarding a product are classified into predefined categories such as positive, negative, neutral etc.  Public sentiments can then be used for corporate decision making regarding a product which is being liked or disliked by the public.

Both rule-based and statistical techniques have been developed for sentimental analysis.  With the advancements in Machine Learning and natural language processing techniques, Sentiment Analysis techniques have improved a lot.

In this tutorial, you will see how Sentiment Analysis can be performed on live Twitter data. The tutorial is divided into two major sections: Scraping Tweets from Twitter and Performing Sentiment Analysis.

Continue reading “Scraping Tweets and Performing Sentiment Analysis”

Google Places API: Extracting Location Data & Reviews

Google places API allows developers to access a wealth of information from Google’s database for over 100 million places including location data, contact information, user ratings and reviews and more.

In this tutorial, you will learn how to create a reusable class to read and extract location related information from Google Places API. This tutorial will help you if you want to extract business’s name, address, phone number, website, and reviews.

Continue reading “Google Places API: Extracting Location Data & Reviews”

Selenium: Web Scraping Booking.com Accommodations

Booking.com is a travel fare aggregator website and travel metasearch engine for lodging reservations. This websites has more than 29,094,365 listings in 230 countries and territories worldwide.

Websites like Booking.com contains a lot of data that can be scraped and processes that can be automatized.

In this Selenium tutorial, will learn how to automate an accommodation search and to scrape the results using Python with Selenium.

Continue reading “Selenium: Web Scraping Booking.com Accommodations”

Data Extraction from APIs with Python – Currency Exchange

There are several popular platforms that give developers access to their “web services”, aka “APIs” (Application Programming Interface). So using APIs is the official way for data extraction and doing other stuff allowed by such applications. You can even benefit from some APIs to build other applications. REST APIs usually generate output in JSON or XML format because most of programming languages can handle these formats easily. In fact, JSON (JavaScript Object Notation) is very similar to data types in programming languages; for example, it is very similar to Python dictionaries. If a REST API allows you to get the data you want to retrieve, then you do not need regular web scraping.

Some APIs require authentication (API Key or Client ID and Client Secret, similar to a username and password, so to speak) to control their usage, and some do not. We will explain this later in multiple APIs. For the purpose of clarifying the basics, we will start with a very simple currency rate conversion API that does not require any authentication.

In this tutorial, you will learn how to use Python to extract data from ExchangeRatesAPI.io which is -according to its official website- “a free service for current and historical foreign exchange rates published by the European Central Bank.” Continue reading “Data Extraction from APIs with Python – Currency Exchange”