当前环境
我使用基于vllm/vllm-openai:0.4.1镜像的docker容器托管llama3-8B-instruct模型。
openai版本：1.6.1(以下示例脚本)

🐛 描述bug

描述

当我们提示模型生成一个包含guided_json参数设置和日期字段的json时，我们得到一个无效的json响应。
这并不依赖于特定的模型。我们在mixtral模型中也得到了相同的错误。

代码

from openai import OpenAI

model_name = "/docker_share/models/llama3_8b_instruct"
num_prompt_tokens = 1000
base_url = "http://model-url"
api_key = "EMPTY"
client = OpenAI(base_url=base_url, api_key=api_key)
def call_vllm(prompt, guided_json=None):
        if guided_json:
            extra_body = {
                 "guided_json": guided_json
                 }
        messages = [{"role": "user", "content": prompt}]
        
        chat_completion = client.chat.completions.create(
            model=model_name,
            messages=messages,
            extra_body=extra_body,
            seed=42,
            max_tokens=num_prompt_tokens,
            temperature=0
        )
        return chat_completion.choices[0].message.content

guided_json = """{ 
    "additionalProperties": false, 
    "properties": { 
        "DocumentDate": {"anyOf": [{"format": "date", "type": "string"}, {"type": "null"}], 
        "description": "Document date", 
        "title": "DocumentDate"} 
    }, 
    "required": ["DocumentDate"], 
    "title": "Document", 
    "type": "object" 
}"""

if __name__ == "__main__":
    res = call_vllm("Extract a date from this string: date of the document - 01.01.1999", guided_json=guided_json)
    print(res)

结果

{ "DocumentDate": 1999-01-01 }

预期结果

{ "DocumentDate": "1999-01-01" }