首页 > 编程语言 > 详细

python爬虫开发与项目实践一书 爬取盗墓笔记 其中的json_dump报错问题

时间:2018-06-21 18:13:25      阅读:217      评论:0      收藏:0      [点我收藏+]
# encoding utf-8    将wb模式改为b模式就不会报错的  环境python36
import requests
import json
from bs4 import BeautifulSoup
user_agent = ‘Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)‘
headers = {‘User-Agent‘: user_agent}
r = requests.get(‘http://seputu.com/‘, headers=headers)
r.encoding = ‘utf-8‘
# print(r.text)
soup = BeautifulSoup(r.text, ‘html.parser‘)
content = []
for mulu in soup.find_all(class_=‘mulu‘):
    h2 = mulu.find("h2")
    # print(h2)
    if not (h2 is None):
        h2_title = h2.string
        list = []
        for a in mulu.find(class_=‘box‘).find_all(‘a‘):
            href = a.get(‘href‘)
            box_title = a.get(‘title‘)
            # href = bytes(href, encoding=‘utf-8‘)
            # box_title = bytes(box_title, encoding=‘utf-8‘)
            print(href, box_title)
            list.append({‘href‘: href, ‘box_title‘: box_title})
        content.append({‘title‘: h2_title, ‘content‘:list})
with open(‘../json_file/mulu.json‘, ‘wb‘) as fp:
    json.dump(content, fp=fp, indent=4)

  

python爬虫开发与项目实践一书 爬取盗墓笔记 其中的json_dump报错问题

原文:https://www.cnblogs.com/Ychao/p/9210341.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!