这个想法是运行一组任务,当它们完成时处理它们,一旦返回所需的结果,就取消剩余的任务。我不介意等待已经开始的任务完成,但应该禁止开始新的任务。
我试过调用pool.shutdown(cancel_futures=True)
,但它似乎不工作在这里:
import time
import multiprocessing
import concurrent.futures
import random
filelist = ['001', '002', '003', '004', '005', '006', '007', '008', '009', '010']
parallel = 2
def worker(name, lock, index):
with lock:
index.value += 1
pc = index.value*10
print(f'starting: {name} ({pc})')
d = random.randrange(1, 5)
time.sleep(d)
return pc == 60
print('INIT')
with multiprocessing.Manager() as manager:
lock = manager.Lock()
index = manager.Value('b', 0)
if parallel > 0:
with concurrent.futures.ProcessPoolExecutor(max_workers=parallel) as pool:
tasks = [pool.submit(worker, filename, lock, index) for filename in filelist]
for task in concurrent.futures.as_completed(tasks):
rc = task.result()
if rc:
print('Found!')
pool.shutdown(cancel_futures=True)
break
else:
for filename in filelist:
rc = worker(filename, lock, index)
if rc:
print('Found!')
break
print('END')
字符串
在串行模式(parallel = 0
)下运行时,我得到了所需的结果:
INIT
starting: 001 (10)
starting: 002 (20)
starting: 003 (30)
starting: 004 (40)
starting: 005 (50)
starting: 006 (60)
Found!
END
型
但当并行启动时,它会一直持续到最后:
INIT
starting: 001 (10)
starting: 002 (20)
starting: 003 (30)
starting: 004 (40)
starting: 005 (50)
starting: 006 (60)
Found!
starting: 007 (70)
starting: 008 (80)
starting: 009 (90)
END
型
那么,有没有办法退出as_completed
循环呢?
1条答案
按热度按时间wn9m85ua1#
是的。你可以用
multiprocessing
模块中的Pool
类来做例子:字符串
输出:
型