NodeJS 在传递数组的forEach()方法中嵌套的参数时查询DynamoDB表

jaql4c8m  于 2022-12-12  发布在  Node.js
关注(0)|答案(2)|浏览(141)

我使用DocumentClient扫描DynamoDB表中的所有项-在Lambda函数中。然后循环遍历每个项并提取所需的有效负载。我将在新的查询中使用有效负载中的该项作为ExpressionAttributeValues的参数。所有内容都可以独立运行。问题是当嵌套在数组forEach中时使用异步函数queryItems()方法。函数queryItems出现解析错误。在循环外调用该函数时,我可以查询表,但如何单独查询每一项?我不确定如何处理此问题。

'use strict';

const aws = require('aws-sdk');
const docClient = new aws.DynamoDB.DocumentClient();

var paramsAll = {
  TableName: 'MyTable',
  Select: "ALL_ATTRIBUTES"
};

exports.handler = async (event, context) => {
  try {
    let arr = [];
    let sequence = '';
    //scan all items in table
    docClient.scan(paramsAll, function(err, data) {
      if (err) {
        //handle error
      }
      else {
        //Loop through each item in the table:
        let items = (data.Items);
        items.forEach(function(Item) {
          let p = (Item.payload);
          //Extract sequence from the payload
          sequence = (p.seq);
          arr.push(sequence);
          //perform other function with this array (not listed for brevity)
        });
        //Here is where I'm having the issue:
        arr.forEach(function(Item) {
          //Pass these items as a paramater within queryItems function but getting Parsing Error: unexpected token queryItems
          const results = await queryItems(Item);
          //do something with the results...
        })
      }
    });
  }
  catch (err) {
    return { error: err };
  }
};

async function queryItems(p) {
  try {
    var params = {
      TableName: 'MyTable',
      KeyConditionExpression: '#seq = :value',
      ExpressionAttributeValues: { ':value': p },
      ExpressionAttributeNames: { '#seq': 'seq' }
    };
    const data = await docClient.query(params).promise();
    return data;
  }
  catch (err) {
    return err;
  }
}
js5cn81o

js5cn81o1#

我确实遇到过类似的问题。我认为发生的只是一个Javascript语法问题,在提供给forEach的同步函数中await ing queryItems将产生一个错误。(尽管,在运行代码时,我确实得到了特定的错误"SyntaxError: await is only valid in async functions and the top level bodies of modules",所以可能还有其他事情发生。)
我认为DynamoDB查询没有什么问题,但是hoangdv的建议非常正确。具体来说,我还建议对scan使用promise样式,虽然for...loop肯定可以工作,但使用Promise.allmap完成所有查询会快得多。下面是我如何修改代码的:

'use strict';

const aws = require('aws-sdk');
const docClient = new aws.DynamoDB.DocumentClient();

// avoid var unless you specifically require it's hoisting behavior.
const paramsAll = {
  TableName: 'MyTable',
  Select: "ALL_ATTRIBUTES" // most likely not needed, I'd review this section of the docs: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html#DDB-Scan-request-Select
};

exports.handler = async (event, context) => {
  try {
    // unless you need to assign a new array to this variable, it is better practice to use const instead.
    const arr = [];
    // let sequence = ''; // see line 24 for why I commented this out.

    // scan all items in table.
    // Destructure Items out of the response.
    // You may also need to continue scanning with the LastEvaluatedKey depending on the size of your table, and/or your use case.
    // You'd continue scanning in a while loop, for example.
    const { Items, LastEvaluatedKey } = await docClient.scan(paramsAll).promise();

    // push the sequence to the arr. 
    // There is most likely a reason you omitted for brevity to have sequence defined above,
    // but since this example doesn't need it above, I've omitted it entirely
    Items.forEach(Item => {
      const p = Item.payload;
      arr.push(p.seq);
    });

    // use a for loop or map here instead. forEach will return undefined, which cannot be await'ed.
    // instead, map will return a new array of Promises (since the callback is async).
    // Then, you can use Promise.all to await until each Promise in the array is resolved.
    // Keep in mind, depending on how many items you are iterating through, you may run into DynamoDB's ThrottlingException.
    //    You would have to batch the queries (in other words, split the arr into pieces, and iterate over each piece), which would have to be done before using map. Then, sleep for a few milliseconds before starting on the next piece.
    //    I doubt the queries will be quick enough to cause this when using a for loop, though.
    await Promise.all(arr.map(async Item => {
      const results = await queryItems(Item);
      // do something with the results...
    }));
  }
  catch (err) {
    // Again, not sure what the use case is, but just FYI this is not a valid return value if this lambda function is intended for use with using API Gateway.
    // See here :) https://docs.aws.amazon.com/lambda/latest/dg/services-apigateway.html#apigateway-types-transforms
    return { error: err };
  }
};

// Presumably, MyTable has a partitionKey of seq, otherwise this KeyConditionExpression is invalid.
async function queryItems(p) {
  try {
    var params = {
      TableName: 'MyTable',
      KeyConditionExpression: '#seq = :value',
      ExpressionAttributeValues: { ':value': p },
      ExpressionAttributeNames: { '#seq': 'seq' }
    };
    const data = await docClient.query(params).promise();
    return data;
  }
  catch (err) {
    return err;
  }
}
pgccezyw

pgccezyw2#

问题是如何在for循环中等待,最好在循环中使用Promise.all()map来等待:

await Promise.all(arr.map(async Item => {
      const results = await queryItems(Item);
      // do something with the results...
    }));

不过,我似乎并不太理解你的逻辑。
1.您扫描了一个名为MyTable的表,但没有分页,这意味着您最多只能获得1MB的数据。
1.有了结果,你就可以去掉seq的值,然后再次读取MyTable中的每一项,这次使用Queryseq作为键。

相关问题