In this tutorial, you will learn how to use Scrapy to log into websites that require entering a username and password before showing specific pages.
Continue reading “Logging in with Scrapy FormRequest”
11 Best Python Online Courses on Udemy
With online courses, you can study anywhere, at the time suitable for you, get full lifetime access, and a Certificate of Completion. It is a wonderful experience that enables you to be taught by international skillful instructors whom you might not get the chance to meet otherwise. Nowadays, e-learning is a great advantage of technology that enables you to easily improve your skills and enjoy the learning experience.
In this article, we are trying to recommend the 11 best Python courses currently available on Udemy, the prominent online courses platform.
Python Web Scraping Libraries: BeautifulSoup vs. Scrapy vs. Selenium
Today, we are going to learn about Scrapy; what Scrapy is overall, Scrapy versus other Python-based web scraping tools, why you should use it and when it makes sense to use some other tools, pros and cons of Scrapy, and that would be it. So let’s begin! Continue reading “Python Web Scraping Libraries: BeautifulSoup vs. Scrapy vs. Selenium”
Data Extraction from APIs with Python – Currency Exchange
There are several popular platforms that give developers access to their “web services”, aka “APIs” (Application Programming Interface). So using APIs is the official way for data extraction and doing other stuff allowed by such applications. You can even benefit from some APIs to build other applications. REST APIs usually generate output in JSON or XML format because most of programming languages can handle these formats easily. In fact, JSON (JavaScript Object Notation) is very similar to data types in programming languages; for example, it is very similar to Python dictionaries. If a REST API allows you to get the data you want to retrieve, then you do not need regular web scraping.
Some APIs require authentication (API Key or Client ID and Client Secret, similar to a username and password, so to speak) to control their usage, and some do not. We will explain this later in multiple APIs. For the purpose of clarifying the basics, we will start with a very simple currency rate conversion API that does not require any authentication.
In this tutorial, you will learn how to use Python to extract data from ExchangeRatesAPI.io which is -according to its official website- “a free service for current and historical foreign exchange rates published by the European Central Bank.” Continue reading “Data Extraction from APIs with Python – Currency Exchange”
Scrapy: Powerful Web Scraping & Crawling with Python – Course
If you come here searching for a discount coupon for our best-selling Scrapy course, you finally got it. Click here to get your Scrapy course coupon. Enjoy your hassle-free learning experience and earn more income through applying for well-paid Scrapy jobs. Continue reading “Scrapy: Powerful Web Scraping & Crawling with Python – Course”
Scrapy Tutorial: Web Scraping Craigslist
In this Scrapy tutorial, you will learn how to write a Craigslist crawler to scrape Craigslist‘s “Architecture & Engineering” jobs in New York and store the data to a CSV file.
This tutorial is one lecture of our comprehensive Scrapy online course on Udemy, Scrapy: Powerful Web Scraping & Crawling with Python Continue reading “Scrapy Tutorial: Web Scraping Craigslist”
3 Questions to Ask Before Web Scraping
There are several Web Scraping best practices you have to follow. Among them are critical questions you have to ask yourself beforehand. Continue reading “3 Questions to Ask Before Web Scraping”
Beautiful Soup Tutorial #3: Web Scraping Craigslist (One Page)
So let’s assume we want to scrape the titles of jobs available in Boston from Craigslist. For now, we will work on one page only. Continue reading “Beautiful Soup Tutorial #3: Web Scraping Craigslist (One Page)”
Beautiful Soup Tutorial #2: Extracting URLs
After installing the required libraries: BeautifulSoup, Requests, and LXML, let’s learn how to extract URLs. Continue reading “Beautiful Soup Tutorial #2: Extracting URLs”
Beautiful Soup Tutorial #1: Install BeautifulSoup, Requests & LXML
To start Web Scraping tutorials, the first thing to do is to install the 3 libraries: BeautifulSoup, Requests, and LXML. We will use PIP. Note that sudo might be required if you are on Linux or Mac. Continue reading “Beautiful Soup Tutorial #1: Install BeautifulSoup, Requests & LXML”