1.在使用多线程处理任务时也不是线程越多越好,由于在切换线程的时候,需要切换上下文环境,依然会造成cpu的大量开销。为解决这个问题,线程池的概念被提出来了。预先创建好一个较为优化的数量的线程,让过来的任务立刻能够使用,就形成了线程池。在python中,没有内置的较好的线程池模块,需要自己实现或使用第三方模块。下面是一个简单的线程池:
import threading,time,os,queue
class ThreadPool(object):
def __init__(self,maxsize):
self.maxsize = maxsize
self._q = queue.Queue(self.maxsize)
for i in range(self.maxsize):
self._q.put(threading.Thread)
def getThread(self):
return self._q.get()
def addThread(self):
self._q.put(threading.Thread)
def fun(num,p):
print(‘this is thread [%s]‘%num)
time.sleep(1)
p.addThread()
if __name__ == ‘__main__‘:
pool = ThreadPool(2)
for i in range(103):
t = pool.getThread()
a = t(target = fun,args = (i,pool))
a.start()
2.利用线程池和paramiko实现对远程服务器的访问获取到相关信息:(自己写的例子,比较low)
import paramiko,threading
import queue
class ThreadPool(object):
def __init__(self,maxsize):
self.maxsize = maxsize
self._q = queue.Queue(self.maxsize)
for i in range(self.maxsize):
self._q.put(threading.Thread)
def getThread(self):
return self._q.get()
def addThread(self):
self._q.put(threading.Thread)
def ssh_fun(ip,user,password,pool):
try:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(ip, 22, user, password)
stdin, stdout, stderr = ssh.exec_command(‘hostname‘)
info = stdout.read().decode().strip()
print(‘IP:%s hostname:%s‘%(ip,info))
ssh.close()
except Exception:
print(‘sorry I can`t connect this server [%s]‘%ip)
pool.addThread()
if __name__ == ‘__main__‘:
t_list = []
pool = ThreadPool(2)
with open(‘aaa‘,‘r+‘,encoding=‘utf-8‘) as f:
for line in f:
split = line.split()
ip,user,password = split[0],split[1],split[2]
th = pool.getThread()
t = th(target=ssh_fun,args=(ip,user,password,pool))
t.start()
t_list.append(t)
for i in t_list:
i.join()
在这里我为了测试线程池中只有两个线程,并且我这个是读取aaa文件的,这个文件中包含用户名和密码等相关信息,样子如下(其实可以把这些放进数据库中,使用python从数据库中进行读取):
192.168.32.167 root 111111 192.168.32.110 root 111111 192.168.32.120 root 111111 192.168.32.150 root 111111
而最后执行的效果如下:
IP:192.168.32.167 hostname:ns.root
sorry I can`t connect this server [192.168.32.110]
IP:192.168.32.150 hostname:localhost.localdomain
sorry I can`t connect this server [192.168.32.120]
原文:http://www.cnblogs.com/hjc4025/p/6950157.html