在爬取失败后使用递归调用再次进行爬取
def download3(url, num_retries=2): print(‘Downloading:‘, url) try: html = urllib2.urlopen(url).read() except urllib2.URLError as e: print(‘Download error:‘, e.reason) html = None if num_retries > 0: if hasattr(e, ‘code‘) and 500 <= e.code < 600: # retry 5XX HTTP errors html = download3(url, num_retries-1) return html
首先使用try-except进行异常处理
hasattr(object, name) # 用于查看对象是否有对应的属性,返回布尔值
原文:https://www.cnblogs.com/lixin2011/p/14077344.html