4.2.16版本:
位点不更新了,channel显示工作中: 没有开监控,无其他日志,重启后正常同步
2018-11-29 12:06:47.441 [pipelineId = 68,taskName = SelectTask] ERROR c.a.o.s.c.core.impl.DefaultCommunicationClientImpl - call[192.168.1.156:1099] , retry[1]
com.alibaba.dubbo.rpc.RpcException: Invoke remote method timeout. method: acceptEvent, provider: dubbo://192.168.1.156:1099/endpoint?acceptEvent.timeout=50000&client=netty&codec=dubbo&connections=30&iothreads=4&lazy=true&payload=8388608&serialization=java&threads=50, cause: Waiting server-side response timeout by scan timer. start time: 2018-11-29 12:05:57.419, end time: 2018-11-29 12:06:47.440, client elapsed: 0 ms, server elapsed: 50021 ms, timeout: 50000 ms, request: Request [id=55189962, version=2.0.0, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=acceptEvent, parameterTypes=[class com.alibaba.otter.shared.communication.core.model.Event], arguments=[FindChannelEvent[channelId=69,pipelineId=,type=findChannel]], attachments={path=endpoint, version=0.0.0}]], channel: /192.168.1.188:57365 -> /192.168.1.156:1099
at com.alibaba.dubbo.rpc.protocol.dubbo.DubboInvoker.doInvoke(DubboInvoker.java:99) ~[dubbo-2.5.3.jar:2.5.3]
at com.alibaba.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:144) ~[dubbo-2.5.3.jar:2.5.3]
at com.alibaba.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:52) ~[dubbo-2.5.3.jar:2.5.3]
at com.alibaba.dubbo.common.bytecode.proxy0.acceptEvent(proxy0.java) ~[na:2.5.3]
at com.alibaba.otter.shared.communication.core.impl.dubbo.DubboCommunicationConnection.call(DubboCommunicationConnection.java:45) ~[shared.communication-4.2.16.jar:na]
at com.alibaba.otter.shared.communication.core.impl.DefaultCommunicationClientImpl.call(DefaultCommunicationClientImpl.java:96) ~[shared.communication-4.2.16.jar:na]
at com.alibaba.otter.node.common.communication.NodeCommmunicationClient.callManager(NodeCommmunicationClient.java:74) [node.common-4.2.16.jar:na]
at com.alibaba.otter.node.common.config.impl.ConfigClientServiceImpl$3.apply(ConfigClientServiceImpl.java:171) [node.common-4.2.16.jar:na]
at com.alibaba.otter.node.common.config.impl.ConfigClientServiceImpl$3.apply(ConfigClientServiceImpl.java:165) [node.common-4.2.16.jar:na]
at com.alibaba.otter.shared.common.utils.cache.RefreshMemoryMirror.get(RefreshMemoryMirror.java:65) [shared.common-4.2.16.jar:na]
at com.alibaba.otter.node.common.config.impl.ConfigClientServiceImpl.findChannelByPipelineId(ConfigClientServiceImpl.java:81) [node.common-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.config.ArbitrateConfigUtils.getChannel(ArbitrateConfigUtils.java:80) [shared.arbitrate-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.setl.helper.StagePathUtils.getChannelId(StagePathUtils.java:178) [shared.arbitrate-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.setl.helper.StagePathUtils.getPipeline(StagePathUtils.java:40) [shared.arbitrate-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.setl.helper.StagePathUtils.getMainStem(StagePathUtils.java:119) [shared.arbitrate-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.setl.monitor.MainstemMonitor.check(MainstemMonitor.java:214) [shared.arbitrate-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.setl.MainStemArbitrateEvent.check(MainStemArbitrateEvent.java:85) [shared.arbitrate-4.2.16.jar:na]
at com.alibaba.otter.shared.arbitrate.impl.setl.MainStemArbitrateEvent$$FastClassByCGLIB$$eabb1e80.invoke() [cglib-nodep-2.2.jar:na]
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191) [cglib-nodep-2.2.jar:na]
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at com.alibaba.otter.shared.arbitrate.impl.interceptor.LogInterceptor.invoke(LogInterceptor.java:53) [shared.arbitrate-4.2.16.jar:na]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at com.alibaba.otter.shared.arbitrate.impl.setl.MainStemArbitrateEvent$$EnhancerByCGLIB$$241229ce.check() [cglib-nodep-2.2.jar:na]
at com.alibaba.otter.node.etl.select.SelectTask.run(SelectTask.java:119) [node.etl-4.2.16.jar:na]
Caused by: com.alibaba.dubbo.remoting.TimeoutException: Waiting server-side response timeout by scan timer. start time: 2018-11-29 12:05:57.419, end time: 2018-11-29 12:06:47.440, client elapsed: 0 ms, server elapsed: 50021 ms, timeout: 50000 ms, request: Request [id=55189962, version=2.0.0, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=acceptEvent, parameterTypes=[class com.alibaba.otter.shared.communication.core.model.Event], arguments=[FindChannelEvent[channelId=69,pipelineId=,type=findChannel]], attachments={path=endpoint, version=0.0.0}]], channel: /192.168.1.188:57365 -> /192.168.1.156:1099
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.returnFromResponse(DefaultFuture.java:188) ~[dubbo-2.5.3.jar:2.5.3]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:110) ~[dubbo-2.5.3.jar:2.5.3]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:84) ~[dubbo-2.5.3.jar:2.5.3]
at com.alibaba.dubbo.rpc.protocol.dubbo.DubboInvoker.doInvoke(DubboInvoker.java:96) ~[dubbo-2.5.3.jar:2.5.3]
... 25 common frames omitted
3条答案
按热度按时间vd8tlhqk1#
node 执行完binlog里面的一个begin....end 事务 后,会自动跟新位置点信息到ZK,manager上面显示的位置点信息也是从ZK上面拉取的。 从你的报错上面看仅仅是node和manager 通信有问题,不会影响node正常工作。 即使manager挂掉了,node也可以正常工作 . @luyee 能稳定重现吗?
jm2pwxwz2#
node 执行完binlog里面的一个begin....end 事务 后,会自动跟新位置点信息到ZK,manager上面显示的位置点信息也是从ZK上面拉取的。 从你的报错上面看仅仅是node和manager 通信有问题,不会影响node正常工作。 即使manager挂掉了,node也可以正常工作 . @luyee 能稳定重现吗?
能稳定重现就好了
xj3cbfub3#
@luyee 今天我对照代码故意模拟了这个超时问题,发现即使出现你上面从manager中读取channel超时错误,node节点也会取channel以前的旧值,不会影响正常流程。后面如果再遇到不同步问题,还是第一时间把JVM dump出来,通过JVM分析卡在哪里,可以@我一起分析。