Today, we are going to learn about Scrapy; what Scrapy is overall, Scrapy versus other Python-based web scraping tools, why you should use it and when it makes sense to use some other tools, pros and cons of Scrapy, and that would be it. So let’s begin! Continue reading “Python Web Scraping Libraries: BeautifulSoup vs. Scrapy vs. Selenium”
Some APIs require authentication (API Key or Client ID and Client Secret, similar to a username and password, so to speak) to control their usage, and some do not. We will explain this later in multiple APIs. For the purpose of clarifying the basics, we will start with a very simple currency rate conversion API that does not require any authentication.
In this tutorial, you will learn how to use Python to extract data from ExchangeRatesAPI.io which is -according to its official website- “a free service for current and historical foreign exchange rates published by the European Central Bank.” Continue reading “Data Extraction from APIs with Python – Currency Exchange”
If you come here searching for a discount coupon for our best-selling Scrapy course, you finally got it. Click here to get your Scrapy course coupon. Enjoy your hassle-free learning experience and earn more income through applying for well-paid Scrapy jobs. Continue reading “Scrapy: Powerful Web Scraping & Crawling with Python – Course”
In this Scrapy tutorial, you will learn how to write a Craigslist crawler to scrape Craigslist‘s “Architecture & Engineering” jobs in New York and store the data to a CSV file.
This tutorial is one lecture of our comprehensive Scrapy online course on Udemy, Scrapy: Powerful Web Scraping & Crawling with Python Continue reading “Scrapy Tutorial: Web Scraping Craigslist”
There are several Web Scraping best practices you have to follow. Among them are critical questions you have to ask yourself beforehand. Continue reading “3 Questions to Ask Before Web Scraping”
So let’s assume we want to scrape the titles of jobs available in Boston from Craigslist. For now, we will work on one page only. Continue reading “Beautiful Soup Tutorial #3: Web Scraping Craigslist (One Page)”
After installing the required libraries: BeautifulSoup, Requests, and LXML, let’s learn how to extract URLs. Continue reading “Beautiful Soup Tutorial #2: Extracting URLs”
To start Web Scraping tutorials, the first thing to do is to install the 3 libraries: BeautifulSoup, Requests, and LXML. We will use PIP. Note that sudo might be required if you are on Linux or Mac. Continue reading “Beautiful Soup Tutorial #1: Install BeautifulSoup, Requests & LXML”