运行配置单元脚本“am\u stats\u analyzer\u 163”cassandra时发生wso2 bam错误

lnlaulya  于 2021-06-04  发布在  Hadoop
关注(0)|答案(3)|浏览(323)

我在两个不同的虚拟机上运行apim和bam，在bam上运行一个外部cassandra。
我有一个**“执行配置单元脚本时出错。查询返回非零代码：9，原因：失败：执行错误，从org.apache.hadoop.hive.ql.exec.mapredtask返回代码2”**执行配置单元脚本时出错（am\u stats\u analyzer\u 163）
这是完整的错误跟踪：

TID: [0] [BAM] [2014-03-18 12:34:07,948] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} -  Ended Job = job_local_0001 with errors {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2014-03-18 12:34:07,950] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} -  Error during job, obtaining debugging information... {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2014-03-18 12:34:08,041] ERROR {org.apache.hadoop.hive.ql.exec.Task} -  Execution failed with exit status: 2 {org.apache.hadoop.hive.ql.exec.Task}
TID: [0] [BAM] [2014-03-18 12:34:08,042] ERROR {org.apache.hadoop.hive.ql.exec.Task} -  Obtaining error information {org.apache.hadoop.hive.ql.exec.Task}
TID: [0] [BAM] [2014-03-18 12:34:08,042] ERROR {org.apache.hadoop.hive.ql.exec.Task} -
Task failed!
Task ID:
  Stage-0

Logs:
 {org.apache.hadoop.hive.ql.exec.Task}
TID: [0] [BAM] [2014-03-18 12:34:08,042] ERROR {org.apache.hadoop.hive.ql.exec.Task} -  /opt/wso2bam-2.4.0/repository/logs//wso2carbon.log {org.apache.hadoop.hive.ql.exec.Task}
TID: [0] [BAM] [2014-03-18 12:34:08,042] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} -  Execution failed with exit status: 2 {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2014-03-18 12:34:08,042] ERROR {org.apache.hadoop.hive.ql.Driver} -  FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.apache.hadoop.hive.ql.Driver}
TID: [0] [BAM] [2014-03-18 12:34:08,044] ERROR {org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl} -  Error while executing Hive script.
Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl}
java.sql.SQLException: Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
    at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:189)
    at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.executeHiveQuery(HiveExecutorServiceImpl.java:569)
    at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:282)
    at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:189)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
TID: [0] [BAM] [2014-03-18 12:34:08,044] ERROR {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask} -  Error while executing script : am_stats_analyzer_163 {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask}
org.wso2.carbon.analytics.hive.exception.HiveExecutionException: Error while executing Hive script.Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl.execute(HiveExecutorServiceImpl.java:115)
org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask.execute(HiveScriptExecutorTask.java:60)
    at     org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

我为cassandra使用的用户名和密码是默认的（admin/admin），它们在**[bam\u home]/repository/conf/etc/cassandra auth.xml和[bam\u home]/repository/conf/datasources/master-datasources.xml中定义**
你知道问题出在哪里吗？
mysql数据源配置：

<datasource>
            <name>WSO2AM_STATS_DB</name>
            <description>The datasource used for getting statistics to API Manager</description>
            <jndiConfig>
            <name>jdbc/WSO2AM_STATS_DB</name>
        </jndiConfig>
        <definition type="RDBMS">
            <configuration>
                <url>jdbc:mysql://GCEP-IDS:3306/APIMGTSTATS_DB</url>
                <username>ldapadmin</username>
                <password>ldapadmin</password>
                <driverClassName>com.mysql.jdbc.Driver</driverClassName>
                <maxActive>50</maxActive>
                <maxWait>60000</maxWait>
                <minIdle>5</minIdle>
                <testOnBorrow>true</testOnBorrow>
                <validationQuery>SELECT 1</validationQuery>
                <validationInterval>30000</validationInterval>
            </configuration>
        </definition>
     </datasource>

api manager.xml配置：

<Enabled>true</Enabled>
    <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
    <ThriftPort>7611</ThriftPort>

    <BAMServerURL>tcp://135.238.115.232:7611</BAMServerURL>
    <BAMUsername>admin</BAMUsername>
    <BAMPassword>admin</BAMPassword>
    <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>

表也是在mysql数据库中创建的：

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| APIMGTSTATS_DB     |
| mysql              |
| regdb              |
| test               |
+--------------------+
5 rows in set (0.02 sec)

mysql> use APIMGTSTATS_DB
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+---------------------------+
| Tables_in_APIMGTSTATS_DB  |
+---------------------------+
| API_REQUEST_SUMMARY       |
| API_RESPONSE_SUMMARY      |
| API_VERSION_USAGE_SUMMARY |
+---------------------------+
3 rows in set (0.00 sec)

mysql> select * from API_REQUEST_SUMMARY;
Empty set (0.00 sec)

mysql> select * from API_RESPONSE_SUMMARY;
Empty set (0.00 sec)

mysql> select * from API_VERSION_USAGE_SUMMARY;
Empty set (0.00 sec)

mysql> \q
Bye
[root@GCEP-IDS ~]#
hadoopHivecassandrawso2wso2bam

3条答案

按热度按时间
new9mtju

new9mtju1#

尝试在data-bridge-config.xml中将主机名设置为0.0.0.0：

<thriftDataReceiver>
    <hostName>0.0.0.0</hostName>
        <port>7611</port>
        <securePort>7711</securePort>
</thriftDataReceiver>

并在cassandra.yaml中设置rpc\u地址：0.0.0.0。
错误与不正确的主机/端口有关。

赞(0）分享  回复(0）举报  2021-06-04
kzipqqlq

kzipqqlq2#

请确保已使用正确的bamserverurl和thriftport配置了api-manager.xml，后者位于/repository/conf/api-manager.xml中

<!-- Enable/Disable the API usage tracker. -->
<Enabled>true</Enabled>  
    <PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
    <ThriftPort>7612</ThriftPort>
    <BAMServerURL>tcp://localhost:7612/</BAMServerURL>
    <BAMUsername>admin</BAMUsername>
    <BAMPassword>admin</BAMPassword>
    <!-- JNDI name of the data source to be used for getting BAM statistics. This data source should
        be defined in the master-datasources.xml file in conf/datasources directory. -->
    <DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>

</APIUsageTracking>
赞(0）分享  回复(0）举报  2021-06-04
hlswsv35

hlswsv353#

由于sql异常，与rdbms相关的配置单元查询似乎有问题。请重新检查在删除“插入覆盖…”查询时配置单元脚本是否运行。如果是这样，请重新检查脚本的这部分。在重新执行查询之前，请手动删除rdbms表（例如，使用h2数据库客户端），并在配置单元脚本顶部添加类似“drop table…”的查询，以清除rdbms Package 的配置单元表。

赞(0）分享  回复(0）举报  2021-06-04
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com