apache storm-初始化错误(worker)

col17t5w  于 2021-06-21  发布在  Storm
关注(0)|答案(2)|浏览(475)

风暴-1.0.1
我的风暴.yaml：

storm.zookeper.servers:
  - "zookeper"

storm.zookeeper.port: 2181

# nimbus.host: "nimbus"

nimbus.seeds: ["nimbus"]
storm.local.dir: "/var/log/storm"
supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703
worker.childopts: "-Xmx768m"
nimbus.childopts: "-Xmx512m"
supervisor.childopts: "-Xmx256m"

storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100

在我的主管日志中我看到：

2016-08-26 19:22:28.023 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:28.523 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:28.883 o.a.s.d.supervisor [INFO] Worker Process b4a402ba-458d-4c4e-b167-143bd9134200 exited with code: 13
2016-08-26 19:22:28.956 o.a.s.d.supervisor [INFO] Worker Process c9718f4b-131f-4cfe-bce7-e3e448790a4e exited with code: 13
2016-08-26 19:22:28.966 o.a.s.d.supervisor [INFO] Worker Process d30be8b6-6ed0-496e-9d23-6879df1f1bdd exited with code: 13
2016-08-26 19:22:29.023 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:29.040 o.a.s.d.supervisor [INFO] Worker Process e80983c3-1ec6-4b25-9ef1-f29eb42194ae exited with code: 13
2016-08-26 19:22:29.524 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:30.024 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:30.524 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:31.025 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:31.525 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:32.025 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started
2016-08-26 19:22:32.525 o.a.s.d.supervisor [INFO] e80983c3-1ec6-4b25-9ef1-f29eb42194ae still hasn't started

工作日志包含以下错误：
http://nimbus：8000/log？file=wordcount2-8-1472263925%2f6701%2fworker.log

2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.version=4.6.5-300.fc24.x86_64
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.name=dmitry
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/dmitry
2016-08-26 19:28:39.016 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.dir=/var/log/storm/workers/3abac0cb-5981-4a87-9ffb-ed2c8053108b
2016-08-26 19:28:39.017 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost:2181 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@79316f3a
2016-08-26 19:28:39.067 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-08-26 19:28:39.163 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2016-08-26 19:28:39.176 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x156c953110f00d8, negotiated timeout = 20000
2016-08-26 19:28:39.180 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-08-26 19:28:39.181 o.a.s.zookeeper [INFO] Zookeeper state update: :connected:none
2016-08-26 19:28:39.191 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] backgroundOperationsLoop exiting
2016-08-26 19:28:39.198 o.a.s.s.o.a.z.ZooKeeper [INFO] Session: 0x156c953110f00d8 closed
2016-08-26 19:28:39.199 o.a.s.s.o.a.z.ClientCnxn [INFO] EventThread shut down
2016-08-26 19:28:39.201 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-08-26 19:28:39.201 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3dd4a6fa
2016-08-26 19:28:39.212 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-08-26 19:28:39.213 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2016-08-26 19:28:39.220 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x156c953110f00d9, negotiated timeout = 20000
2016-08-26 19:28:39.220 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-08-26 19:28:39.250 o.a.s.s.a.AuthUtils [INFO] Got AutoCreds []
2016-08-26 19:28:39.254 o.a.s.d.worker [INFO] Reading Assignments.
2016-08-26 19:28:39.310 o.a.s.m.TransportFactory [INFO] Storm peer transport plugin:backtype.storm.messaging.netty.Context
2016-08-26 19:28:39.311 o.a.s.d.worker [ERROR] Error on initialization of server mk-worker
java.lang.RuntimeException: Fail to construct messaging plugin from plugin backtype.storm.messaging.netty.Context
    at org.apache.storm.messaging.TransportFactory.makeContext(TransportFactory.java:53) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.worker$worker_data.invoke(worker.clj:266) ~[storm-core-1.0.1.jar:1.0.1]
    at org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto__$reify__8452.run(worker.clj:611) ~[storm-core-1.0.1.jar:1.0.1]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101]
    at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_101]
    at org.apache.storm.daemon.worker$fn__8450$exec_fn__2461__auto____8451.invoke(worker.clj:609) ~[storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:178) ~[clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
    at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]
Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.netty.Context
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_101]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_101]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_101]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_101]
    at java.lang.Class.forName0(Native Method) ~[?:1.8.0_101]
    at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_101]
    at org.apache.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38) ~[storm-core-1.0.1.jar:1.0.1]
    ... 14 more
2016-08-26 19:28:39.319 o.a.s.util [ERROR] Halting process: ("Error on initialization")
java.lang.RuntimeException: ("Error on initialization")
    at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$fn__8450$mk_worker__8545.doInvoke(worker.clj:583) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.RestFn.invoke(RestFn.java:512) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker$_main.invoke(worker.clj:771) [storm-core-1.0.1.jar:1.0.1]
    at clojure.lang.AFn.applyToHelper(AFn.java:165) [clojure-1.7.0.jar:?]
    at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.7.0.jar:?]
    at org.apache.storm.daemon.worker.main(Unknown Source) [storm-core-1.0.1.jar:1.0.1]

nimbus正在运行

(petrel) [dmitry:~/Projects/experiments/storm/petrel/chapter3/example1]$ netstat -an | grep 6627
69:tcp6       0      0 :::6627                 :::*                    LISTEN     
75:tcp6       0      0 192.168.14.164:48582    192.168.14.164:6627     ESTABLISHED
77:tcp6       0      0 192.168.14.164:49154    192.168.14.164:6627     ESTABLISHED
78:tcp6       0      0 192.168.14.164:48548    192.168.14.164:6627     ESTABLISHED
80:tcp6       0      0 192.168.14.164:6627     192.168.14.164:49154    ESTABLISHED
81:tcp6       0      0 192.168.14.164:48578    192.168.14.164:6627     ESTABLISHED
82:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48064    ESTABLISHED
83:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48484    ESTABLISHED
85:tcp6       0      0 192.168.14.164:48560    192.168.14.164:6627     ESTABLISHED
86:tcp6       0      0 192.168.14.164:49130    192.168.14.164:6627     ESTABLISHED
87:tcp6       0      0 192.168.14.164:6627     192.168.14.164:49380    ESTABLISHED
89:tcp6       0      0 192.168.14.164:6627     192.168.14.164:49148    ESTABLISHED
90:tcp6       0      0 192.168.14.164:48058    192.168.14.164:6627     ESTABLISHED
91:tcp6       0      0 192.168.14.164:49424    192.168.14.164:6627     TIME_WAIT  
92:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48034    ESTABLISHED
93:tcp6       0      0 192.168.14.164:6627     192.168.14.164:48498    ESTABLISHED
Javapythonnettyapache-storm

2条答案

按热度按时间
ocebsuys

ocebsuys1#

问题是我的javau home env var指向了一个错误的文件夹。
之前：
java\u home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-1.b14.fc24.x86\u 64/jre/
之后：
java\u home=/usr/lib/jvm/java-1.8.0-oracle-1.8.0.92.x86\u 64/
我得到的建议是通过运行storm示例中的示例java拓扑来检查storm是否正确安装。
我遵循了这个官方教程。

cd ~/Development/storm/examples/storm-starter
mvn package

给我这个：

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 49.281 s
[INFO] Finished at: 2016-08-27T19:16:54-07:00
[INFO] Final Memory: 21M/391M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project storm-starter: Could not resolve dependencies for project org.apache.storm:storm-starter:jar:1.0.1: Could not find artifact jdk.tools:jdk.tools:jar:1.7 at specified path /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.101-1.b14.fc24.x86_64/jre/../lib/tools.jar -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

(petrel) [dmitry:/storm-starter]$ dnf list installed | grep java

55:abrt-java-connector.x86_64           1.1.0-8.fc24                @koji-override-0
749:java-1.8.0-openjdk.x86_64           1:1.8.0.101-1.b14.fc24      @updates        
750:java-1.8.0-openjdk-headless.x86_64  1:1.8.0.101-1.b14.fc24      @updates        
751:java-1.8.0-oracle.x86_64            1.8.0.92-1.fc24.R           @russianfedora-nonfree
752:java-1.8.0-oracle-devel.x86_64      1.8.0.92-1.fc24.R           @russianfedora-nonfree
753:java-1.8.0-oracle-headless.x86_64   1.8.0.92-1.fc24.R           @russianfedora-nonfree
754:javapackages-tools.noarch           4.6.0-14.fc24               @koji-override-0
1846:python3-javapackages.noarch        4.6.0-14.fc24               @koji-override-0
2431:tzdata-java.noarch                 2016f-1.fc24                @updates

当我将java_home切换到/usr/lib/jvm/java-1.8.0-oracle-1.8.0.92.x86_64/时，我就能够编译并将示例tology提交到远程集群中（它仍然是storm的本地安装，zookeeper/nimbus/supervisor/ui/logviewer正在本地运行）
还请记住，雅虎的一篇文章中，他们推荐“netty”作为传输，并提供以下值：


# storm.messaging.transport: "backtype.storm.messaging.netty.Context"

# storm.messaging.netty.server_worker_threads: 1

# storm.messaging.netty.client_worker_threads: 1

# storm.messaging.netty.buffer_size: 5242880

# storm.messaging.netty.max_retries: 100

# storm.messaging.netty.max_wait_ms: 1000

# storm.messaging.netty.min_wait_ms: 100

这会对系统产生负面影响，因为Storm1.0+的类结构发生了变化。
类在1.0.1中从backtype.storm重新打包到org.apache.storm。

赞(0）分享  回复(0）举报  2021-06-21
jm81lzqq

jm81lzqq2#

因为错误指示类 backtype.storm.messaging.netty.Context 找不到。
原因：java.lang.classnotfoundexception:backtype.storm.messaging.netty.context
在Storm中 1.0.0 这是一个重大的重构重命名许多包。我猜你想用 org.apache.storm.messaging.netty.Context 现在。

storm.messaging.transport: "org.apache.storm.messaging.netty.Context"

看到了吗https://storm.apache.org/releases/1.0.0/index.html

赞(0）分享  回复(0）举报  2021-06-21
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com