首页 > Web开发 > 详细

根据请求网页状态返回码 批量过滤无效网址

时间:2015-02-25 16:59:51      阅读:474      评论:0      收藏:0      [点我收藏+]
#coding=gbk
 
import os
import httplib2
import socket
 

def GetWebStatus(host):
    try:
        h =httplib2.Http()             
        resp, content = h.request(host)
        resultStatus =  resp.get(‘status‘)        
        if(resultStatus != ‘200‘):
            #print(host+"无效")
            return 0
        else:
            #print(host+"有效")
            return 1
    except Exception:
        return 0
 
 
def ReadHost():

    obn = open(‘d:/white.txt‘, ‘rb‘);
    for line in obn:
        line = line.strip(‘\r\n‘)
        ss = ‘http://www.‘+line
        if (GetWebStatus(ss) == 0):
            fw = open("d:/wuxiao.txt","a+")
            fw.writelines(line)
            fw.writelines(‘\n‘)
            fw.close()   
        else:
            fw = open("d:/out.txt","a+")
            fw.writelines(line)
            fw.writelines(‘\n‘)
            fw.close()        
    obn.close();

 
if __name__ == "__main__":
    ReadHost();

 

根据请求网页状态返回码 批量过滤无效网址

原文:http://www.cnblogs.com/xiaobaichuangtianxia/p/4299736.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!