如何在NodeJS脚本中访问Chrome Devtools网络日志数据?

ffscu2ro  于 2023-04-29  发布在  Node.js
关注(0)|答案(2)|浏览(204)

我目前正在尝试构建一个需要从网站(OpenSea市场)读取数据的项目。当我转到页面时,它向API发送POST请求(在图1中),我想在我的NodeJS/TypeScript项目中访问它返回的数据(在图2中)。
有没有API可以做到这一点,或者甚至不可能?
Pic 1 (我正在尝试访问graphql/ request)

图2 (这是我试图进入代码的数据)

emeijp43

emeijp431#

您可以劫持XMLHttpRequestfetch函数。非传统的,但它会工作
基于我放置的注解,客户端可以使用一个小函数来完成这些工作

function listen(fn){
  //this line simply adds functions to call in an array based on setup below
  if(listen.prototype.arguments){listen.prototype.arguments.push(fn)}
  
  //below is setup(functions hijacked so services that use them report to this too)
  listen.prototype.arguments=[] //array of functions to call
  let original1=XMLHttpRequest.prototype.send, original2=fetch
  function replace1(){
    let toReturn=original1.bind(this)(...arguments)
    this.addEventListener("load",res=>fn(this.responseText))
    return toReturn
  }
  function replace2(){
    let toReturn=original2.bind(this)(...arguments)
    toReturn.then(res=>res.text().then(fn))
    return toReturn
  }
  Object.defineProperty(XMLHttpRequest.prototype,"send",{value:replace1})
  Object.defineProperty(window,"fetch",{value:replace2})
}


//example usage
let socket=some_Web_Socket_Your_Backend_To_Handle_This_Data
//perhaps this is not the most elaborate example but I hope the point is understood
listen(socket.send.bind(socket))
wvyml7n5

wvyml7n52#

  • 请查看此服务,开始构建自己的REST API并在项目中使用它。Leetcode Public REST API
  • 下面的代码可能会给予你深入了解如何提取数据。假设这是GraphQL查询。
export const recentSubmissionsQuery = `
    query recentAcSubmissions($username: String!, $limit: Int!) {
        recentAcSubmissionList(username: $username, limit: $limit) {
            id
            title
            titleSlug
            timestamp
        }
    }
`;

您可以在NodeJS/ExpressJS中创建一个端点,如下所示

router.get("/:username/submissions", async (req, res) => {
  const query = recentSubmissionsQuery;
  const { username } = req.params;
  let { limit } = req.query;
  if (!limit) limit = 3;

  const data = await fetchGraphQLData(query, { username, limit });
  res.json(data);
});

fetchGraphQLData看起来类似于这样:

import fetch from "node-fetch";

const URL = BASE_URL;

const fetchGraphQLData = async (query, variables) => {
  const options = {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      query,
      variables,
    }),
  };

  try {
    const response = await fetch(URL, options);
    const data = await response.json();
    return data;
  } catch (error) {
    console.error(error);
  }
};

export default fetchGraphQLData;
  • 使用**node-fetchaxios**库来执行HTTP请求,否则部署中会遇到问题。

相关问题