首页 > 其他 > 详细

加速scrapy

时间:2017-12-13 11:04:28      阅读:272      评论:0      收藏:0      [点我收藏+]
def parse(self, response):
# Get the next index URLs and yield Requests
next_sel = response.xpath(//*[contains(@class,"next")]//@href) for url in next_sel.extract():
yield Request(urlparse.urljoin(response.url, url))

# Iterate through products and create PropertiesItems selectors = response.xpath(
//*[@itemtype="http://schema.org/Product"]) for selector in selectors:
yield self.parse_item(selector, response)

 

加速scrapy

原文:http://www.cnblogs.com/realmonkeykingsun/p/8031447.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!