当我尝试获取twitter tweets时,出现了一个接收器错误。我添加了twitterapi配置,并在hdfs中创建了一个目录。我不确定我做错了什么。我正在使用hadoop2.0.0-cdh4.2.1
ERROR flume.SinkRunner: Unable to deliver event.
ERROR hdfs.HDFSEventSink: process failed
例外情况随之而来。
java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
Flume.conf
# Naming the components on the current agent.
TwitterAgent.sources = twitter
TwitterAgent.channels = memoryChannel
TwitterAgent.sinks = HDFS
# Describing/Configuring the source
TwitterAgent.sources.twitter.type = org.apache.flume.source.twitter.TwitterSource
TwitterAgent.sources.twitter.consumerKey = Xxx
TwitterAgent.sources.twitter.consumerSecret = Xxx
TwitterAgent.sources.twitter.accessToken = Xxx
TwitterAgent.sources.twitter.accessTokenSecret = Xxx
TwitterAgent.sources.twitter.maxBatchDurationMillis = 200
TwitterAgent.sources.twitter.channels = memoryChannel
TwitterAgent.sources.twitter.keywords = lsu
TwitterAgent.channels.memoryChannel.type = memory
TwitterAgent.channels.memoryChannel.capacity = 10000
TwitterAgent.channels.memoryChannel.transactionCapacity = 1000
TwitterAgent.sinks.HDFS.type = hdfs
TwitterAgent.sinks.HDFS.channel = memoryChannel
TwitterAgent.sinks.HDFS.hdfs.path = hdfs:/user/flume/tweets/
TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream
TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text
TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
TwitterAgent.sinks.HDFS.hdfs.rollSize = 0
TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000
TwitterAgent.sinks.HDFS.hdfs.useLocalTimeStamp = true
暂无答案!
目前还没有任何答案,快来回答吧!