这篇文章主要介绍怎么使用Python爬虫库BeautifulSoup遍历文档树并对标签进行操作，文中介绍的非常详细，具有一定的参考价值，感兴趣的小伙伴们一定要看完！
Python是一种跨平台的、具有解释性、编译性、互动性和面向对象的脚本语言，其最初的设计是用于编写自动化脚本，随着版本的不断更新和新功能的添加，常用于用于开发独立的项目和大型项目。
使用Python爬虫库BeautifulSoup对文档树进行遍历并对标签进行操作的实例
html_doc = """ <html><head><title>The Dormouse's story</title></head> <p class="title"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link1">Elsie</a>, <a href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ from bs4 import BeautifulSoup soup = BeautifulSoup(html_doc,'lxml')
一、子节点
一个Tag可能包含多个字符串或者其他Tag，这些都是这个Tag的子节点.BeautifulSoup提供了许多操作和遍历子结点的属性。
1.通过Tag的名字来获得Tag
print(soup.head) print(soup.title)
<head><title>The Dormouse's story</title></head> <title>The Dormouse's story</title>
通过名字的方法只能获得第一个Tag，如果要获得所有的某种Tag可以使用find_all方法
soup.find_all('a')[<a class="sister" href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link1">Elsie</a>, <a class="sister" href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link2">Lacie</a>, <a class="sister" href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link3">Tillie</a>]
2.contents属性：将Tag的子节点通过列表的方式返回
head_tag = soup.head head_tag.contents
[<title>The Dormouse's story</title>]
title_tag = head_tag.contents[0] title_tag
<title>The Dormouse's story</title>
title_tag.contents
["The Dormouse's story"]
3.children：通过该属性对子节点进行循环
for child in title_tag.children: print(child)
The Dormouse's story
4.descendants： 不论是contents还是children都是返回直接子节点，而descendants对所有tag的子孙节点进行递归循环
for child in head_tag.children: print(child)
<title>The Dormouse's story</title>
for child in head_tag.descendants: print(child)
<title>The Dormouse's story</title> The Dormouse's story
5.string 如果tag只有一个NavigableString类型的子节点，那么tag可以使用.string得到该子节点
title_tag.string
"The Dormouse's story"
如果一个tag只有一个子节点，那么使用.string可以获得其唯一子结点的NavigableString.
head_tag.string
"The Dormouse's story"
如果tag有多个子节点，tag无法确定.string对应的是那个子结点的内容，故返回None
print(soup.html.string)
None
6.strings和stripped_strings
如果tag包含多个字符串，可以使用.strings循环获取
for string in soup.strings: print(string)
The Dormouse's story The Dormouse's story Once upon a time there were three little sisters; and their names were Elsie , Lacie and Tillie ; and they lived at the bottom of a well. ...
.string输出的内容包含了许多空格和空行，使用strpped_strings去除这些空白内容
for string in soup.stripped_strings: print(string)
The Dormouse's story The Dormouse's story Once upon a time there were three little sisters; and their names were Elsie , Lacie and Tillie ; and they lived at the bottom of a well. ...
二、父节点
1.parent：获得某个元素的父节点
title_tag = soup.title title_tag.parent
<head><title>The Dormouse's story</title></head>
字符串也有父节点
title_tag.string.parent
<title>The Dormouse's story</title>
2.parents：递归的获得所有父辈节点
link = soup.a for parent in link.parents: if parent is None: print(parent) else: print(parent.name)
p body html [document]
三、兄弟结点
sibling_soup = BeautifulSoup("<a><b>text1</b><c>text2</c></b></a>",'lxml')
print(sibling_soup.prettify())<html> <body> <a> <b> text1 </b> <c> text2 </c> </a> </body> </html>
1.next_sibling和previous_sibling
sibling_soup.b.next_sibling
<c>text2</c>
sibling_soup.c.previous_sibling
<b>text1</b>
在实际文档中.next_sibling和previous_sibling通常是字符串或者空白符
soup.find_all('a')[<a class="sister" href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link1">Elsie</a>, <a class="sister" href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link2">Lacie</a>, <a class="sister" href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link3">Tillie</a>]
soup.a.next_sibling # 第一个<a></a>的next_sibling是,\n
',\n'
soup.a.next_sibling.next_sibling
<a class="sister" href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link2">Lacie</a>
2.next_siblings和previous_siblings
for sibling in soup.a.next_siblings: print(repr(sibling))
',\n' <a class="sister" href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link2">Lacie</a> ' and\n' <a class="sister" href="http://example.com/tillie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link3">Tillie</a> ';\nand they lived at the bottom of a well.'
for sibling in soup.find(id="link3").previous_siblings: print(repr(sibling))
' and\n' <a class="sister" href="http://example.com/lacie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link2">Lacie</a> ',\n' <a class="sister" href="http://example.com/elsie" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" id="link1">Elsie</a> 'Once upon a time there were three little sisters; and their names were\n'
四、回退与前进
1.next_element和previous_element
指向下一个或者前一个被解析的对象(字符串或tag)，即深度优先遍历的后序节点和前序节点
last_a_tag = soup.find("a", id="link3")
print(last_a_tag.next_sibling)
print(last_a_tag.next_element); and they lived at the bottom of a well. Tillie
last_a_tag.previous_element
' and\n'
2.next_elements和previous_elements
通过.next_elements和previous_elements可以向前或向后访问文档的解析内容，就好像文档正在被解析一样
for element in last_a_tag.next_elements: print(repr(element))
'Tillie' ';\nand they lived at the bottom of a well.' '\n' <p class="story">...</p> '...' '\n'
以上是“怎么使用Python爬虫库BeautifulSoup遍历文档树并对标签进行操作”这篇文章的所有内容，感谢各位的阅读！希望分享的内容对大家有帮助，更多相关知识，欢迎关注亿速云行业资讯频道！
免责声明：本站发布的内容（图片、视频和文字）以原创、转载和分享为主，文章观点不代表本网站立场，如果涉及侵权请联系站长邮箱：is@yisu.com进行举报，并提供相关证据，一经查实，将立刻删除涉嫌侵权内容。
