向kafka发送压缩(snappy)消息时出现不满意的链接错误

l7mqbcuq  于 2021-06-07  发布在  Kafka
关注(0)|答案(1)|浏览(518)

在我的javaweb应用程序中,我向kafka发送消息。
我想在发送前压缩邮件,因此我要设置我的生产者属性:
props.put(“compression.codec”,“2”);
据我所知,“2”代表snappy,但当发送信息时,我得到:

  1. java.lang.UnsatisfiedLinkError: org.xerial.snappy.SnappyNative.maxCompressedLength(I)I
  2. at org.xerial.snappy.SnappyNative.maxCompressedLength(Native Method)
  3. at org.xerial.snappy.Snappy.maxCompressedLength(Snappy.java:316)
  4. at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:79)
  5. at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:66)
  6. at kafka.message.SnappyCompression.<init>(CompressionUtils.scala:61)
  7. at kafka.message.CompressionFactory$.apply(CompressionUtils.scala:82)
  8. at kafka.message.CompressionUtils$.compress(CompressionUtils.scala:109)
  9. at kafka.message.MessageSet$.createByteBuffer(MessageSet.scala:71)
  10. at kafka.message.ByteBufferMessageSet.<init>(ByteBufferMessageSet.scala:44)
  11. at kafka.producer.async.DefaultEventHandler$$anonfun$3.apply(DefaultEventHandler.scala:94)
  12. at kafka.producer.async.DefaultEventHandler$$anonfun$3.apply(DefaultEventHandler.scala:82)
  13. at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
  14. at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:233)
  15. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:95)
  16. at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:95)
  17. at scala.collection.Iterator$class.foreach(Iterator.scala:772)
  18. at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157)
  19. at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190)
  20. at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45)
  21. at scala.collection.mutable.HashMap.foreach(HashMap.scala:95)
  22. at scala.collection.TraversableLike$class.map(TraversableLike.scala:233)
  23. at scala.collection.mutable.HashMap.map(HashMap.scala:45)
  24. at kafka.producer.async.DefaultEventHandler.serialize(DefaultEventHandler.scala:82)
  25. at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:44)
  26. at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:116)
  27. at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:95)
  28. at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:71)
  29. at scala.collection.immutable.Stream.foreach(Stream.scala:526)
  30. at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:70)
  31. at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:41)

为了解决这个问题,我尝试将snappy依赖项添加到我的pom中:

  1. <dependency>
  2. <groupId>org.xerial.snappy</groupId>
  3. <artifactId>snappy-java</artifactId>
  4. <version>${snappy-version}</version>
  5. <scope>provided</scope>
  6. </dependency>

并将jar添加到/lib/ext下的jetty服务器,但仍然会出现此错误。
如果在“compression.codec”属性中设置“0”而不是“2”,则不会像预期的那样出现异常。
我应该怎么做才能使用快速压缩?
这是我的snappy版本(我应该使用另一个吗?):1.1.0.1
我正在Jetty8.1.9上部署我的应用程序,它运行在Ubuntu12.10上。

kkih6yb8

kkih6yb81#

  1. <dependency>
  2. <groupId>org.xerial.snappy</groupId>
  3. <artifactId>snappy-java</artifactId>
  4. <version>1.1.1.3</version>
  5. </dependency>

我有同样的问题,上面的代码解决了我的问题。jar包含所有操作系统的本机库。以下是我的开发环境:

  1. JDK version: 1.7.0_76
  2. Kafka version: 2.10-0.8.2.1
  3. Zookeeper version: 3.4.6

相关问题