我写了一个算法,并试图比较不同版本的性能。我的基准函数使用线程池,但它需要相同的时间或比单核实现更长。我用pypy和python,版本3.11和结果是相同的。
要进行基准测试的方法:
def main(print_results=True):
results = Queue()
start_time = time.time()
words = get_set_from_dict_file("usa.txt")
results.put(f"Total words read: {len(words)}")
results.put(f"Total time taken to read the file: {round((time.time() - start_time) * 1000)} ms")
start_time_2 = time.time()
pairs = getPairs(words)
results.put(f"Number of words that can be built with 3 letter word + letter + 3 letter word: {len(pairs)}")
results.put(f"Total time taken to find the pairs: {round((time.time() - start_time_2) * 1000)} ms")
results.put(f"Time taken: {round((time.time() - start_time) * 1000)}ms")
if print_results:
[print(x) for x in results.queue]
return (time.time() - start_time) * 1000
多线程线程池:
def benchmark(n=1000):
# start number of threads equal to 90% of cores running main() using multiprocessing, continue until n runs complete
core_count = os.cpu_count()
thread_num = floor(core_count * 0.9)
pool = ThreadPool(thread_num)
results = pool.map_async(main, [False] * n)
results = results.get()
pool.close()
avg_time_ms = round(sum(results) / len(results))
# Save best run time and its code as a pickle file in format (time, code)
# Currently hidden code
return avg_time_ms, -1
试验:
if __name__ == "__main__":
print("Do you want to benchmark? (y/n)")
if input().upper() == "Y":
print("Benchmark n times: (int)")
n = input()
n = int(n) if (n.isdigit() and 0 < int(n) <= 1000) else 100
start = time.time()
bench = benchmark(n)
end = time.time()
print("\n----------Multi-Thread Benchmark----------")
print(f"Average time taken: {bench[0]} ms")
print(f"Best time taken yet: {bench[1]} ms")
print(f"Total bench time: {end - start:0.5} s")
start = time.time()
non_t_results = [main(False) for _ in range(n)]
end = time.time()
print("\n----------Single-Thread Benchmark----------")
print(f"Average time taken: {round(sum(non_t_results) / len(non_t_results))} ms")
print(f"Total bench time: {end - start:0.5} s")
else:
main()
每次运行它时,无论池中运行的线程数或线程数如何,池的完成速度都不会更快。
Do you want to benchmark? (y/n)
y
Benchmark n times: (int)
50
----------Multi-Thread Benchmark----------
Average time taken: 276 ms
Best time taken yet: -1 ms
Total bench time: 2.2814 s
----------Single-Thread Benchmark----------
Average time taken: 36 ms
Total bench time: 1.91 s
Process finished with exit code 0
我希望线程池完成得更快。
1条答案
按热度按时间mbskvtky1#
原来我用的是线程而不是进程。多亏了评论者,我才能理解ThreadPool是用于并发处理的,Pool是用于并行处理的。
以下是更改后的基准: