Beautiful Soup Tutorial #3: Web Scraping Craigslist (One Page)

So let’s assume we want to scrape the titles of jobs available in Boston from Craigslist. For now, we will work on one page only.

We will simply search the website, and get the URL:  https://boston.craigslist.org/search/sof

 

If you have not already, please revise the previous BeautifulSoup Tutorials:

  1. Beautiful Soup Tutorial #1: Install BeautifulSoup, Requests & LXML
  2. Beautiful Soup Tutorial #2: Extracting URLs

 

In the current tutorial, the only new part is:

 

Now, it will find all <a> tags whose class name is ‘result-title’.

How can you know the class name? You can find it by opening the URL in your browser, moving the cursor on a job title, right-clicking, and selecting “Inspect“. You can see now the HTML code like this:

 

You can try the same for addresses. The following code will extract all <span> tags whose class name is ‘result-hood’.

 

Copy the code below, run it on your machine, and let me know if you have questions.

 

✅ ✅ ✅  If you want to learn more about web scraping, you can join this online video course:

Web Scraping with Python: BeautifulSoup, Requests & Selenium   👈

👆

Rating: 5.0/5. From 2 votes.
Please wait...

Leave a Reply