cassandra node.js更新计数器时驱动程序内存泄漏

wpx232ag 于 2021-06-13 发布在 Cassandra

关注(0)|答案(2)|浏览(399)

正在尝试迁移计数器表。从源表中读取每一行并将结果写入新表。当写入数百万行时，js的问题是堆内存不足。正在使用同步代码来确保在尝试写入另一行之前先写入该行。固定观察堆的问题似乎是在使用时堆会上升 cassandra-driver 执行 UPDATE 查询。

const query = `UPDATE keyspace.table_name
  SET counter1 = counter1 + ${x1}, counter2 = counter2 + ${x2}
  WHERE partition_key = ? AND clustering_key = ?`;

await cassandraDestination.execute(query, params, { prepare: true });

在这种情况下如何避免内存泄漏？

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1317672cfc5d]
Security context: 0x192500f1d971 <JSObject>
    1: borrowNextConnection [0x2ea411f96091] [/home/asen.chekov/projects/stats/node_modules/cassandra-driver/lib/request-handler.js:~64] [pc=0x131767db1395](this=0x2ea411f95da1 <JSFunction RequestHandler (sfi = 0x2e8feb1ef231)>,0x21125933f379 <Object map = 0x1ddd59acbdb1>,0x21125933f439 <Object map = 0x1ddd59a82521>,0x110a33490991 <ProfileManager map...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20191016.122316.23174.001.json
Node.js report completed
 1: 0x953b10 node::Abort() [node]
 2: 0x9547f4 node::OnFatalError(char const*, char const*) [node]
 3: 0xb32bee v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb32e24 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xf32452  [node]
 6: 0xf32558 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [node]
 7: 0xf3ec78 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xf3f78b v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xf424c1 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
10: 0xf0c6f4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
11: 0x11c2b3e v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
12: 0x1317672cfc5d 
Aborted (core dumped)```

cassandra JavaScript node.js cql cassandra-driver

来源：https://stackoverflow.com/questions/58409561/cassandra-node-js-driver-memory-leak-when-update-counter

2条答案

按热度按时间

dtcbnfnu1#

请注意，签名的方法 execute() 方法是：

execute(query: string, params: Array, options: object)

另外，您不应该在prepare语句中硬编码参数。
正确用法是：

const query = `UPDATE keyspace.table_name
  SET counter = counter + 1, counter1 = counter1 + 1
  WHERE partition_key = ? AND clustering_key = ?`;

await cassandraDestination.execute(query, [ key1, key2 ], { prepare: true });

赞(0）回复(0）举报 2021-06-14

uplii1fm2#

这解决了问题。
正确的查询是：

const query = `UPDATE keyspace.table_name
  SET counter1 = counter1 + ?, counter2 = counter2 + ?
  WHERE partition_key = ? AND clustering_key = ?`;

await cassandraDestination
  .execute(query, [ counter1, counter2, key1, key2 ], {
    prepare: true,
  });

赞(0）回复(0）举报 2021-06-13

我来回答

cassandra node.js更新计数器时驱动程序内存泄漏

2条答案

相关问题

热门标签

最新问答