首页 > 其他 > 详细

抓取贴吧

时间:2018-06-08 19:01:34      阅读:179      评论:0      收藏:0      [点我收藏+]

import os
import urllib.request
from urllib import parse

#https://tieba.baidu.com/f?kw=%E7%BE%8E%E5%A5%B3&pn=50

def writePage(filename,html):
"""
:param filename:
"""
with open(filename, "wb") as f:
f.write(html)

def request(url,begPage,endPage):
for index in range(begPage, endPage + 1):
ua_url = fullurl + "&pn=" + str((index - 1) * 50)
print(ua_url)
request = urllib.request.Request(ua_url, headers=headers)
response = urllib.request.urlopen(request).read()
filename = "第" + str(index) + "页.html"
writePage(filename,response)

if __name__ == "__main__":
url = "https://tieba.baidu.com/f?"
key = input("please input keywords to query >>")
begPage = int(input("请输入开始页数 >> "))
endPage = int(input("请输入结束页数 >> "))
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0"}
kw = parse.urlencode({"kw": key})
fullurl = url + kw
request(fullurl,begPage,endPage)

抓取贴吧

原文:https://www.cnblogs.com/angle90/p/9157065.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!