当前环境

vLLM main branch commit c8a7e932

🐛 描述错误

在使用 --pipeline_parallel_size=2 时，如果提示使用的令牌数超过一半，将抛出错误。
vLLM 在加载模型时报告的容量为 1650 个块 / 26.4k 个令牌，而 --max_model_len 被设置为 24000:

INFO 08-01 15:24:16 distributed_gpu_executor.py:56] # GPU blocks: 1650, # CPU blocks: 0

但是当发送任何具有 >13k 个令牌的提示时，它会抛出一个 input prompt is too long 错误：

WARNING 08-01 15:24:58 scheduler.py:706] Input prompt (512 tokens) is too long and exceeds the capacity of block_manager

添加这些打印语句：

diff --git a/vllm/core/block_manager_v1.py b/vllm/core/block_manager_v1.py
index e29eba37..3bf57230 100644
--- a/vllm/core/block_manager_v1.py
+++ b/vllm/core/block_manager_v1.py
@@ -224,6 +224,7 @@ class BlockSpaceManagerV1(BlockSpaceManager):
     ) -> None:
         self.block_size = block_size
         self.num_total_gpu_blocks = num_gpu_blocks
+        logger.info(f"self.num_total_gpu_blocks = {num_gpu_blocks}")
         self.num_total_cpu_blocks = num_cpu_blocks

         if enable_caching and sliding_window is not None:
@@ -286,6 +287,7 @@ class BlockSpaceManagerV1(BlockSpaceManager):
         # Use watermark to avoid frequent cache eviction.
         if (self.num_total_gpu_blocks - num_required_blocks <
                 self.watermark_blocks):
+            logger.info(f"total_gpu_blocks({self.num_total_gpu_blocks}) - num_required_blocks({num_required_blocks}) < watermark_blocks({self.watermark_blocks})")
             return AllocStatus.NEVER
         if num_free_gpu_blocks - num_required_blocks >= self.watermark_blocks:
             return AllocStatus.OK

显示检查是针对 825 个块(13.2k 个令牌)进行的，这是开始时报告容量的一半，这个请求以一个 21.8k 的提示失败：

INFO 08-01 15:36:54 distributed_gpu_executor.py:56] # GPU blocks: 1650, # CPU blocks: 0
[...]
INFO 08-01 15:37:31 model_runner.py:1219] Graph capturing finished in 37 secs.
INFO 08-01 15:37:31 block_manager_v1.py:227] self.num_total_gpu_blocks = 825
INFO 08-01 15:37:31 block_manager_v1.py:227] self.num_total_gpu_blocks = 825
[...]
INFO:     127.0.0.1:38182 - "POST /v1/chat/completions HTTP/1.1" 200 OK
INFO 08-01 15:38:17 block_manager_v1.py:290] total_gpu_blocks(825) - num_required_blocks(1363) < watermark_blocks(8)
WARNING 08-01 15:38:17 scheduler.py:706] Input prompt (512 tokens) is too long and exceeds the capacity of block_manager