www.crummy.com Open in urlscan Pro
66.160.141.133  Public Scan

Form analysis
1 forms found in the DOM

GET /search/

<form method="get" action="/search/">
  <input type="text" name="q" maxlength="255" value="">
</form>

Text Content

[ Download | Documentation | Hall of Fame | For enterprise | Source | Changelog
| Discussion group | Zine ]


BEAUTIFUL SOUP

You didn't write that awful page. You're just trying to get some data out of it.
Beautiful Soup is here to help. Since 2004, it's been saving programmers hours
or days of work on quick-turnaround screen scraping projects.

Beautiful Soup is a Python library designed for quick turnaround projects like
screen-scraping. Three features make it powerful:

 1. Beautiful Soup provides a few simple methods and Pythonic idioms for
    navigating, searching, and modifying a parse tree: a toolkit for dissecting
    a document and extracting what you need. It doesn't take much code to write
    an application
 2. Beautiful Soup automatically converts incoming documents to Unicode and
    outgoing documents to UTF-8. You don't have to think about encodings, unless
    the document doesn't specify an encoding and Beautiful Soup can't detect
    one. Then you just have to specify the original encoding.
 3. Beautiful Soup sits on top of popular Python parsers like lxml and html5lib,
    allowing you to try out different parsing strategies or trade speed for
    flexibility.

Beautiful Soup parses anything you give it, and does the tree traversal stuff
for you. You can tell it "Find all the links", or "Find all the links of class
externalLink", or "Find all the links whose urls match "foo.com", or "Find the
table heading that's got bold text, then give me that text."

Valuable data that was once locked up in poorly-designed websites is now within
your reach. Projects that would have taken hours take only minutes with
Beautiful Soup.

Interested? Read more.


GETTING AND GIVING SUPPORT

Beautiful Soup for enterprise available via Tidelift

If you have questions, send them to the discussion group. If you find a bug,
file it on Launchpad. If it's a security vulnerability, report it confidentially
through Tidelift.

If you use Beautiful Soup as part of your work, please consider a Tidelift
subscription. This will support many of the free software projects your
organization depends on, not just Beautiful Soup.

If Beautiful Soup is useful to you on a personal level, you might like to read
Tool Safety, a short zine I wrote about what I learned about software
development from working on Beautiful Soup. Thanks!


DOWNLOAD BEAUTIFUL SOUP

The current release is Beautiful Soup 4.9.3 (October 3, 2020). You can install
Beautiful Soup 4 with pip install beautifulsoup4.

In Debian and Ubuntu, Beautiful Soup is available as the python-bs4 package (for
Python 2) or the python3-bs4 package (for Python 3). In Fedora it's available as
the python-beautifulsoup4 package.

Beautiful Soup is licensed under the MIT license, so you can also download the
tarball, drop the bs4/ directory into almost any Python application (or into
your library path) and start using it immediately. (If you want to do this under
Python 3, you will need to manually convert the code using 2to3.)

Beautiful Soup 4 works on both Python 2 (2.7+) and Python 3. Support for Python
2 will be discontinued on or after December 31, 2020—one year after the Python 2
sunsetting date.


BEAUTIFUL SOUP 3

Beautiful Soup 3 was the official release line of Beautiful Soup from May 2006
to March 2012. It does not support Python 3 and it will be discontinued on or
after December 31, 2020—one year after the Python 2 sunsetting date. If you have
any active projects using Beautiful Soup 3, you should migrate to Beautiful Soup
4 as part of your Python 3 conversion.

Here's the Beautiful Soup 3 documentation.

The current and hopefully final release of Beautiful Soup 3 is 3.2.2 (October 5,
2019). It's the BeautifulSoup package on pip. It's also available as
python-beautifulsoup in Debian and Ubuntu, and as python-BeautifulSoup in
Fedora.

Once Beautiful Soup 3 is discontinued, these package names will be available for
use by a more recent version of Beautiful Soup.

Beautiful Soup 3, like Beautiful Soup 4, is supported through Tidelift.


HALL OF FAME

Over the years, Beautiful Soup has been used in hundreds of different projects.
There's no way I can list them all, but I want to highlight a few high-profile
projects. Beautiful Soup isn't what makes these projects interesting, but it did
make their completion easier:

 * "Movable Type", a work of digital art on display in the lobby of the New York
   Times building, uses Beautiful Soup to scrape news feeds.
 * Jiabao Lin's DXY-COVID-19-Crawler uses Beautiful Soup to scrape a Chinese
   medical site for information about COVID-19, making it easier for researchers
   to track the spread of the virus. (Source: "How open source software is
   fighting COVID-19")
 * Reddit uses Beautiful Soup to parse a page that's been linked to and find a
   representative image.
 * Alexander Harrowell uses Beautiful Soup to track the business activities of
   an arms merchant.
 * The developers of Python itself used Beautiful Soup to migrate the Python bug
   tracker from Sourceforge to Roundup.
 * The Lawrence Journal-World uses Beautiful Soup to gather statewide election
   results.
 * The NOAA's Forecast Applications Branch uses Beautiful Soup in TopoGrabber, a
   script for downloading "high resolution USGS datasets."

If you've used Beautiful Soup in a project you'd like me to know about, please
do send email to me or the discussion group.


DEVELOPMENT

Development happens at Launchpad. You can get the source code or file bugs.

--------------------------------------------------------------------------------

This document (source) is part of Crummy, the webspace of Leonard Richardson
(contact information). It was last modified on Saturday, October 03 2020,
15:42:16 Nowhere Standard Time and last built on Friday, December 03 2021,
11:00:01 Nowhere Standard Time.



Crummy is © 1996-2021 Leonard Richardson. Unless otherwise noted, all text
licensed under a Creative Commons License.



Document tree:

http://www.crummy.com/software/BeautifulSoup/ Site Search: