如何使用BAAI/bge-reranker-v2-minicpm-layerwise rerank模型调用xinferenceapi

nlejzf6q 于 6个月前发布在其他

关注(0)|答案(2)|浏览(116)

GRADE_MODEL_NAME = ApolloClient.get_value(key="GRADE_MODEL_NAME", default_val="",
                                               namespace=SERVICE_NAMESPACE)
    XINFERENCE_URL = ApolloClient.get_value(key="XINFERENCE_URL", default_val="",
                                            namespace=SERVICE_NAMESPACE)
    client = Client(XINFERENCE_URL)
    __reranker:RESTfulRerankModelHandle = client.get_model(GRADE_MODEL_NAME)
    query = "介绍一下xxxxx"
    docs: List[Document] = []
    docs.append(Document(page_content="xxx是xxxx",
                         metadata={"title": "xxxx工作报告"}))
  

    rerank_list = __reranker.rerank(documents=[item.page_content
                                               for index, item in enumerate(docs)
                                               if index==1
                                               ],
                                    query=query,
                                    top_n=1)
    print(rerank_list.get("results"))

1.如何使用rerank设置指定层的rerank
2.按照现有的方法进行rerank报错，错误信息如下

File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/client/restful/restful_client.py", line 181, in rerank
    raise RuntimeError(
RuntimeError: Failed to rerank documents, detail: [address=0.0.0.0:43511, pid=90365] 'float' object is not subscriptable

请帮忙协助解决一下？谢谢

inference

来源：https://github.com/xorbitsai/inference/issues/1819

2条答案

按热度按时间

qncylg1j1#

服务端报错详情，应该是这个：relevance_score=float(similarity_scores[arg]),

Traceback (most recent call last):
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/api/restful_api.py", line 1165, in rerank
    scores = await model.rerank(
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xoscar/backends/context.py", line 227, in send
    return self._process_result_message(result)
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
    raise message.as_instanceof_cause()
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xoscar/backends/pool.py", line 659, in send
    result = await self._run_coro(message.message_id, coro)
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
    return await coro
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xoscar/api.py", line 384, in __on_receive__
    return await super().__on_receive__(message)  # type: ignore
  File "xoscar/core.pyx", line 558, in __on_receive__
    raise ex
  File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.__on_receive__
    async with self._lock:
  File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.__on_receive__
    with debug_async_timeout('actor_lock_timeout',
  File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive__
    result = await result
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/core/utils.py", line 45, in wrapped
    ret = await func(*args, **kwargs)
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/core/model.py", line 87, in wrapped_func
    ret = await fn(self, *args, **kwargs)
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/core/model.py", line 551, in rerank
    return await self._call_wrapper(
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/core/model.py", line 111, in _async_wrapper
    return await fn(*args, **kwargs)
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/core/model.py", line 377, in _call_wrapper
    ret = await asyncio.to_thread(fn, *args, **kwargs)
  File "/usr/local/python3/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/usr/local/python3/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/model/rerank/core.py", line 234, in rerank
    docs = [
  File "/usr/local/Galileo/galileo/lib/python3.10/site-packages/xinference/model/rerank/core.py", line 237, in <listcomp>
    relevance_score=float(similarity_scores[arg]),
TypeError: [address=0.0.0.0:43511, pid=90365] 'float' object is not subscriptable

赞(0）回复(0）举报 6个月前

92vpleto2#

这个问题已经过时，因为它已经开放了7天，没有任何活动。

赞(0）回复(0）举报 6个月前