我已经建立了一个storm开发集群,它利用了几个资源,比如storm实时处理食谱,通过p在osx上设置storm。泰勒·戈茨,当然还有内森·马兹的所有内容。目前,我已经使用puppet和vagrant在一台机器上建立了一个集群,其中有nimbus、zookeeper和两个管理器的vm。本地机器本身就是主机。我似乎可以提交storm starter的拓扑示例https://github.com/nathanmarz/storm-starter 让他们运行,但我有一对夫妇的问题,要么运行各种工具,通过风暴或验证集群的实际工作!
问题:每当我从storm的bin运行任何storm工具(如storm ui或storm logviewer)时,控制台都会立即抛出以下内容并通常挂起: Running: java -server -Dstorm.options= -Dstorm.home=/Users/jhogan/Development/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /Users/jhogan/Development/storm/lib/asm-4.0.jar:/Users/jhogan/Development/storm/lib/carbonite-1.3.2.jar:/Users/jhogan/Development/storm/lib/clj-stacktrace-0.2.4.jar:/Users/jhogan/Development/storm/lib/clj-time-0.4.1.jar:/Users/jhogan/Development/storm/lib/clojure-1.4.0.jar:/Users/jhogan/Development/storm/lib/clout-1.0.1.jar:/Users/jhogan/Development/storm/lib/commons-codec-1.4.jar:/Users/jhogan/Development/storm/lib/commons-exec-1.1.jar:/Users/jhogan/Development/storm/lib/commons-fileupload-1.2.1.jar:/Users/jhogan/Development/storm/lib/commons-io-1.4.jar:/Users/jhogan/Development/storm/lib/commons-lang-2.5.jar:/Users/jhogan/Development/storm/lib/commons-logging-1.1.1.jar:/Users/jhogan/Development/storm/lib/compojure-1.1.3.jar:/Users/jhogan/Development/storm/lib/core.incubator-0.1.0.jar:/Users/jhogan/Development/storm/lib/curator-client-1.0.1.jar:/Users/jhogan/Development/storm/lib/curator-framework-1.0.1.jar:/Users/jhogan/Development/storm/lib/disruptor-2.10.1.jar:/Users/jhogan/Development/storm/lib/guava-13.0.jar:/Users/jhogan/Development/storm/lib/hiccup-0.3.6.jar:/Users/jhogan/Development/storm/lib/httpclient-4.1.1.jar:/Users/jhogan/Development/storm/lib/httpcore-4.1.jar:/Users/jhogan/Development/storm/lib/jetty-6.1.26.jar:/Users/jhogan/Development/storm/lib/jetty-util-6.1.26.jar:/Users/jhogan/Development/storm/lib/jgrapht-core-0.9.0.jar:/Users/jhogan/Development/storm/lib/jline-2.11.jar:/Users/jhogan/Development/storm/lib/joda-time-2.0.jar:/Users/jhogan/Development/storm/lib/json-simple-1.1.jar:/Users/jhogan/Development/storm/lib/junit-3.8.1.jar:/Users/jhogan/Development/storm/lib/kryo-2.17.jar:/Users/jhogan/Development/storm/lib/log4j-over-slf4j-1.6.6.jar:/Users/jhogan/Development/storm/lib/logback-classic-1.0.6.jar:/Users/jhogan/Development/storm/lib/logback-core-1.0.6.jar:/Users/jhogan/Development/storm/lib/math.numeric-tower-0.0.1.jar:/Users/jhogan/Development/storm/lib/meat-locker-0.3.1.jar:/Users/jhogan/Development/storm/lib/minlog-1.2.jar:/Users/jhogan/Development/storm/lib/netty-3.6.3.Final.jar:/Users/jhogan/Development/storm/lib/objenesis-1.2.jar:/Users/jhogan/Development/storm/lib/reflectasm-1.07-shaded.jar:/Users/jhogan/Development/storm/lib/ring-core-1.1.5.jar:/Users/jhogan/Development/storm/lib/ring-devel-0.3.11.jar:/Users/jhogan/Development/storm/lib/ring-jetty-adapter-0.3.11.jar:/Users/jhogan/Development/storm/lib/ring-servlet-0.3.11.jar:/Users/jhogan/Development/storm/lib/servlet-api-2.5-20081211.jar:/Users/jhogan/Development/storm/lib/servlet-api-2.5.jar:/Users/jhogan/Development/storm/lib/slf4j-api-1.6.5.jar:/Users/jhogan/Development/storm/lib/snakeyaml-1.11.jar:/Users/jhogan/Development/storm/lib/storm-core-0.9.1-incubating.jar:/Users/jhogan/Development/storm/lib/tools.cli-0.2.2.jar:/Users/jhogan/Development/storm/lib/tools.logging-0.2.3.jar:/Users/jhogan/Development/storm/lib/tools.macro-0.1.0.jar:/Users/jhogan/Development/storm/lib/zookeeper-3.3.3.jar:/Users/jhogan/Development/storm:/Users/jhogan/Development/storm/conf -Xmx768m -Dlogfile.name=ui.log -Dlogback.configurationFile=/Users/jhogan/Development/storm/logback/cluster.xml backtype.storm.ui.core
在这种情况下,我想启动storm ui。用户界面登录storm(从主机): 2014-03-20 12:26:24 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog 2014-03-20 12:26:24 o.m.log [INFO] jetty-6.1.26 2014-03-20 12:26:24 o.m.log [INFO] Started SocketConnector@0.0.0.0:8080
…不太合理,因为我假设我需要连接到nimbus机器生成的ui:
流浪汉档案: boxes = [ { :name => :nimbus, :ip => '10.200.1.100', :cpus =>2, :memory => 512, :instance => 'm1.small' }, { :name => :supervisor1, :ip => '10.200.1.101', :cpus =>4, :memory => 1024, :instance => 'm1.medium' }, { :name => :supervisor2, :ip => '10.200.1.102', :cpus =>4, :memory => 1024, :instance => 'm1.medium' }, { :name => :zookeeper1, :ip => '10.200.1.201', :cpus =>1, :memory => 1024, :instance => 'm1.small' }, ]
在这两种情况下,我得到一个404错误的页面。
运行“风暴列表”:
`1941 [main] INFO backtype.storm.thrift - Connecting to Nimbus at 10.200.1.100:6627
Exception in thread "main" org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
at backtype.storm.thrift$nimbus_client_and_conn.invoke(thrift.clj:71)
at backtype.storm.command.list$_main.invoke(list.clj:22)
at clojure.lang.AFn.applyToHelper(AFn.java:159)
at clojure.lang.AFn.applyTo(AFn.java:151)
at backtype.storm.command.list.main(Unknown Source)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
... 6 more`
每台机器似乎都设置和运行正确,我可以使用vagrantssh将ssh连接到其中任何一台机器上。我可以提交拓扑:
`storm jar storm-starter-0.0.1-SNAPSHOT-jar-with-dependencies.jar storm.starter.WordCountTopology`
让它们看起来运行,但我不知道如何检查系统是否真的正常运行。在哪里检查拓扑进度的日志?为什么我不能按照文档中的规定连接到10.200.1.100:8080上的storm ui?
我是这个系统的新手,所以如果我没有发布正确的数据,我很抱歉。让我知道什么部分的傀儡,流浪汉,或风暴,我需要张贴,以协助这个决议。
谢谢!
1条答案
按热度按时间3mpgtkmj1#
我看你可能有港口问题。确保nimbus、zookeeper和主管都能够通过适当的端口相互交谈。
至于日志,您可以检查每个主管上的工作日志,以查看实际发生的情况。