在执行query_engine.image_query(image_path, query_str)函数时，出现了一个KeyError: 'context_str'错误。我认为问题出在llama_index/core/query_engine/multi_modal.py文件中的SimpleMultiModalQueryEngine类的_get_response_with_images()函数中，没有提供content_str。
版本：0.10.50
重现步骤：
from llama_index.core.indices.multi_modal.base import (
MultiModalVectorStoreIndex,
)
from llama_index.vector_stores.qdrant import QdrantVectorStore
from llama_index.core import SimpleDirectoryReader, StorageContext
from llama_index.embeddings.clip import ClipEmbedding
import qdrant_client
client = qdrant_client.QdrantClient(path="qdrant_mm_wiki")
text_store = QdrantVectorStore(client=client, collection_name="text_collection")
image_store = QdrantVectorStore(client=client, collection_name="image_collection")
storage_context = StorageContext.from_defaults(vector_store=text_store, image_store=image_store)
from llama_index.core import Settings
Settings.embed_model = ClipEmbedding()
image_embed_model = ClipEmbedding()
documents = SimpleDirectoryReader("./mixed_wiki",recursive=True).load_data()
from llama_index.core.node_parser import SentenceSplitter
Settings.text_splitter = SentenceSplitter(chunk_size=60, chunk_overlap=5)
index = MultiModalVectorStoreIndex.from_documents(
documents,
storage_context=storage_context,
transformations=[SentenceSplitter(chunk_size=60, chunk_overlap=5)],
image_embed_model=image_embed_model,
)
from llama_index.multi_modal_llms.ollama import OllamaMultiModal
mm_model = OllamaMultiModal(model="llava:13b")
import numpy as np
from llama_index.core.prompts import PromptTemplate
from llama_index.core.query_engine import SimpleMultiModalQueryEngine
qa_tmpl_str = (
"Given the images provided, "
"answer the query.\n"
"Query: {query_str}\n"
"Answer: "
)
qa_tmpl = PromptTemplate(qa_tmpl_str)
query_str = 'What is the main object in the picture?'
## query and response
query_engine = index.as_query_engine(llm=mm_model, text_qa_template=qa_tmpl)
response = query_engine.image_query('./mixed_wiki/1.jpg', query_str)相关日志/跟踪信息：无响应
1条答案
按热度按时间py49o6xq1#
嘿，@StarVolcano!我在这里帮助你解决仓库中的任何错误、问题或贡献。在你等待人工维护者时，请告诉我如何帮助你。
要修复
KeyError: 'context_str'函数中的_get_response_with_images(),你需要确保context_str键包含在_image_qa_template的格式字符串中。当前_get_response_with_images()的实现仅在格式字符串中包含query_str,这很可能是导致KeyError的原因。以下是
_get_response_with_images()函数的修正版本：这确保了
context_str包含在格式字符串中，防止了KeyError。此外，请确保image_qa_template已正确设置，以便接受context_str和query_str作为参数 1(https://github.com/run-llama/llama_index/blob/main/llama-index-legacy/llama_index/legacy/query_engine/multi_modal.py)2(https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/query_engine/multi_modal.py)。