python中提供多进程包：multiprocessing，支持子进程，通信，共享内存，执行不同形式的同步，提供了Process、Pipi、Lock等组件
多进程和多线程
多进程和多线程区别：
多线程使用的是CPU的一个核，适合IO密集型
多进程使用的是CPU的多个核，适合运算密集型
1）multiprocessing的方法
cpu_count()：统计cpu总数
active_children()：获取所有子进程
例子：
#!/usr/bin/env python import multiprocessing p = multiprocessing.cpu_count() m = multiprocessing.active_children() print(p) print(m)
运行结果：
8
[]
2）Process进程
创建一个Process对象：p = multiprocessing.Precess(target=worker,args=(2,))
说明：
target = 函数名字
args = 函数需要的的参数，以tuple形式传入
3）Process常用方法
is_alive()：判断进程是否存活
run()：启动进程
start()：启动进程，会自动调用run方法，常用
join(timeout=)：等待进程结束或者知道超时
4）Process常用属性
name：进程名字
pid：进程的pid
例子：
#!/usr/bin/env python
import time
import multiprocessing
def worker(interval):
time.sleep(interval)
print("hello,China")
if __name__ == "__main__":
p = multiprocessing.Process(target=worker,args=(5,))
p.start()
print(p.is_alive())
p.join(timeout=3)
print(p.name)
print(p.pid)
print("This is end")运行结果：
True
Process-1
121764
This is end
hello,China
实例: 多进程
import time
import multiprocessing
def worker(name,interval):
print("{0} start" .format(name))
time.sleep(interval)
print("{0} end" .format(name))
if __name__ == "__main__":
print("main start")
print("The computer has {0} core" .format(multiprocessing.cpu_count()))
p1 = multiprocessing.Process(target=worker,args=("worker",2))
p2 = multiprocessing.Process(target=worker,args=("worker",3))
p3 = multiprocessing.Process(target=worker,args=("worker",4))
p1.start()
p2.start()
p3.start()
for p in multiprocessing.active_children():
print("The pid of {0} is {1}" .format(p.name,p.pid) )
print("main end")运行结果：
main start
The computer has 4 core
The pid of Process-1 is 21112
The pid of Process-3 is 20536
The pid of Process-2 is 2116
main end
worker start
worker start
worker start
worker end
worker end
worker end
二、lock组件
当我们用多进程来读写文件时，如果一个写一个读同时进行时不行的，必须一个写完之后，另一个才可以读。因此需要用到一个锁机制进行控制
实例1：多进程不加锁
import multiprocessing
import time
def add(number,value,lock):
print("init,member = {1}".format(value,number))
for i in xrange(1,6):
number += value
time.sleep(1)
print("add {0},number = {1}".format(value,number))
if __name__ == "__main__":
lock = multiprocessing.Lock()
number = 0
p1 = multiprocessing.Process(target=add,args=(number,1,lock))
p3 = multiprocessing.Process(target=add,args=(number,3,lock))
p1.start()
p3.start()
print("main end")运行结果：
main end
init,member = 0
init,member = 0
add 1,number = 1
add 3,number = 3
add 1,number = 2
add 3,number = 6
add 1,number = 3
add 3,number = 9
add 1,number = 4
add 3,number = 12
add 1,number = 5
add 3,number = 15
实例2：多进程加锁
import multiprocessing
import time
def add(number,value,lock):
lock.acquire()
try:
print("init,member = {1}".format(value,number))
for i in xrange(1,6):
number += value
time.sleep(1)
print("add {0},number = {1}".format(value,number))
except Exception as e:
raise e
finally:
lock.release()
if __name__ == "__main__":
lock = multiprocessing.Lock()
number = 0
p1 = multiprocessing.Process(target=add,args=(number,1,lock))
p3 = multiprocessing.Process(target=add,args=(number,3,lock))
p1.start()
p3.start()
print("main end")运行结果：
main end
init,member = 0
add 1,number = 1
add 1,number = 2
add 1,number = 3
add 1,number = 4
add 1,number = 5
init,member = 0
add 3,number = 3
add 3,number = 6
add 3,number = 9
add 3,number = 12
add 3,number = 15
三、共享内存
两个“同时“读写的文件，其中一个作用的结果对另外一个有影响。multiprocessing提供了Value和Array模块
实例1：多进程内存共享不加锁
import multiprocessing
import time
def add(number,value1):
try:
print("init,member = {1}".format(value1,number.value))
for i in xrange(1,6):
number.value += value1
time.sleep(1)
print("add {0},number = {1}".format(value1,number.value))
except Exception as e:
raise e
if __name__ == "__main__":
lock = multiprocessing.Lock()
number = multiprocessing.Value("i",0)
p1 = multiprocessing.Process(target=add,args=(number,1))
p3 = multiprocessing.Process(target=add,args=(number,3))
p1.start()
p3.start()
print("main end")运行结果：
main end
init,member = 0
init,member = 1
add 1,number = 4
add 3,number = 5
add 1,number = 8
add 3,number = 9
add 1,number = 12
add 3,number = 13
add 1,number = 16
add 3,number = 17
add 1,number = 20
add 3,number = 20
实例2: 多进程共享内存加锁
import multiprocessing
import time
def add(number,value1,lock):
lock.acquire()
try:
print("init,member = {1}".format(value1,number.value))
for i in xrange(1,6):
number.value += value1
time.sleep(1)
print("add {0},number = {1}".format(value1,number.value))
except Exception as e:
raise e
finally:
lock.release()
if __name__ == "__main__":
lock = multiprocessing.Lock()
number = multiprocessing.Value("i",0)
p1 = multiprocessing.Process(target=add,args=(number,1,lock))
p3 = multiprocessing.Process(target=add,args=(number,3,lock))
p1.start()
p3.start()
print("main end")运行结果：
main end
init,member = 0
add 1,number = 1
add 1,number = 2
add 1,number = 3
add 1,number = 4
add 1,number = 5
init,member = 5
add 3,number = 8
add 3,number = 11
add 3,number = 14
add 3,number = 17
add 3,number = 20
原文：http://blog.51cto.com/huangzp/2045990