使用Langchain的响应流是Typescript

iecba09b  于 2023-06-30  发布在  TypeScript
关注(0)|答案(1)|浏览(203)

我试着写一段代码,从Pinecone中找到topK匹配向量,并根据topK结果返回所问问题的摘要。
为此,我使用LangChain和OpenAI(在typescript中创建嵌入)。
这就是代码:

import { PineconeClient } from "@pinecone-database/pinecone";
import { VectorDBQAChain } from "langchain/chains";
import { PineconeStore } from "langchain/vectorstores/pinecone";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { OpenAI } from "langchain/llms/openai";


const embeddings = new OpenAIEmbeddings({
    openAIApiKey: process.env.OPENAI_API_KEY,
});

const pinecone = new PineconeClient();

const model = new OpenAI({
    openAIApiKey: process.env.OPENAI_API_KEY,
});


async function main(namespaceID: string) {

    await pinecone.init({
        apiKey: process.env.PINECONE_API_KEY,
        environment: "us-west4-gcp-free",
    });

    const pineconeIndex = pinecone.Index("index-test");

    const vectorStore = await PineconeStore.fromExistingIndex(
        embeddings,
        { pineconeIndex }
    );

    vectorStore.namespace = namespaceID

    const chain = VectorDBQAChain.fromLLM(model, vectorStore, {
        k: 5,
        returnSourceDocuments: true,
        verbose: true,
    });
    const response = await chain.call({ query: "What is the status of Project X" });
    console.log(response);


}

main("TEST_NAMESPACE")

这段代码运行良好。现在,我希望能够将响应从chain.call()流式传输到我的控制台。
对此有什么解决方案或变通办法吗?

ctrmrzij

ctrmrzij1#

以下是您提供的代码的最后一部分的修改版本,它启用了响应流:

let streamedResponse = "";
const streamingModel = new ChatOpenAI({
    streaming: true,
    callbacks: [{
        handleLLMNewToken(token) {
            streamedResponse += token;
        },
    }],
    openAIApiKey: process.env.OPENAI_API_KEY
});
const chain = VectorDBQAChain.fromLLM(streamingModel, vectorStore, {
    k: 5,
    returnSourceDocuments: true,
    verbose: true,
});
const response = await chain.call({
    query: "What is the status of Project X"
});
console.log({ streamedResponse });

您需要添加一个新的导入:

import { ChatOpenAI } from 'langchain/chat_models/openai'

LangChain关于Streaming的文档

相关问题