这篇文章主要介绍了Python爬虫如何采集微博视频数据，具有一定借鉴价值，感兴趣的朋友可以参考下，希望大家阅读完这篇文章之后大有收获，下面让小编带着大家一起了解一下。
requests
pprint
版 本：python 3.8
-编辑器：pycharm 2021.2
作用：批量获取互联网数据(文本, 图片, 音频, 视频)
本质：一次次的请求与响应
1. 导入所需模块
计算
安全
数据库
网络和加速
企业服务
import requests
import pprint2. 找到目标网址
打开开发者工具，选中Fetch/XHR，选中数据所在的标签，找到目标所在url
https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor
3. 发送网络请求
headers = {
'cookie': '',
'referer': 'https://weibo.com/tv/channel/4379160563414111/editor',
'user-agent': '',
}
data = {
'data': '{"Component_Channel_Editor":{"cid":"4379160563414111","count":9}}'
}
url = 'https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor'
json_data = requests.post(url=url, headers=headers, data=data).json()4. 获取数据
json_data_2 = requests.post(url=url_1, headers=headers, data=data_1).json()5. 筛选数据
dict_urls = json_data_2['data']['Component_Play_Playinfo']['urls']
video_url = "https:" + dict_urls[list(dict_urls.keys())[0]]
print(title + "\t" + video_url)6. 保存数据
video_data = requests.get(video_url).content
with open(f'video\\{title}.mp4', mode='wb') as f:
f.write(video_data)
print(title, "爬取成功................")
完整代码
import requests
import pprint
headers = {
'cookie': '添加自己的',
'referer': 'https://weibo.com/tv/channel/4379160563414111/editor',
'user-agent': '',
}
data = {
'data': '{"Component_Channel_Editor":{"cid":"4379160563414111","count":9}}'
}
url = 'https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor'
json_data = requests.post(url=url, headers=headers, data=data).json()
print(json_data)
ccs_list = json_data['data']['Component_Channel_Editor']['list']
next_cursor = json_data['data']['Component_Channel_Editor']['next_cursor']
for ccs in ccs_list:
oid = ccs['oid']
title = ccs['title']
data_1 = {
'data': '{"Component_Play_Playinfo":{"oid":"' + oid + '"}}'
}
url_1 = 'https://weibo.com/tv/api/component?page=/tv/show/' + oid
json_data_2 = requests.post(url=url_1, headers=headers, data=data_1).json()
dict_urls = json_data_2['data']['Component_Play_Playinfo']['urls']
video_url = "https:" + dict_urls[list(dict_urls.keys())[0]]
print(title + "\t" + video_url)
video_data = requests.get(video_url).content
with open(f'video\\{title}.mp4', mode='wb') as f:
f.write(video_data)
print(title, "爬取成功................")感谢你能够认真阅读完这篇文章，希望小编分享的“Python爬虫如何采集微博视频数据”这篇文章对大家有帮助，同时也希望大家多多支持亿速云，关注亿速云行业资讯频道，更多相关知识等着你来学习!
亿速云「云服务器」，即开即用、新一代英特尔至强铂金CPU、三副本存储NVMe SSD云盘，价格低至29元/月。点击查看>>
免责声明：本站发布的内容（图片、视频和文字）以原创、转载和分享为主，文章观点不代表本网站立场，如果涉及侵权请联系站长邮箱：is@yisu.com进行举报，并提供相关证据，一经查实，将立刻删除涉嫌侵权内容。
