1，PhantomJS
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities from selenium import webdriver dcap = dict(DesiredCapabilities.PHANTOMJS) dcap[‘phantomjs.page.settings.userAgent‘] = ‘Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.75 Safari/537.36‘ browser = webdriver.PhantomJS(desired_capabilities=dcap) browser.save_screenshot(‘ctrip.png‘)(你可以截图下来看看你是不是正常访问页面数据)
2，Chrome
将上面的PhantomJS替换掉就可以。
Scrapy selenium 设置头部信息headers-------UA
原文：http://www.cnblogs.com/fh-fendou/p/7798476.html