在ubuntu16.04中尝试创建从sqlserver2017到hadoop的外部表时,抛出以下错误
msg 105019,level 16,state 1,line 1 external table access failed due to internal error:'调用hdfsbridge\u isdirexist时引发java异常。java异常消息:从desktop-ve8knag/..x.调用到..x.x:54310连接失败异常:java.net.connectexception:连接被拒绝:无进一步信息;有关详细信息,请参阅:http://wiki.apache.org/hadoop/connectionrefused:error[从desktop-ve8knag/1..x.调用..x:54310连接失败异常:java.net.connectexception:连接被拒绝:无进一步信息;有关详细信息,请参阅:http://wiki.apache.org/hadoop/connectionrefused]访问外部文件时发生
外部源位置从core-site.xml获取
创建tmp文件夹,并向用户添加权限,并将其添加到core-site.xml中
所有节点都在hadoop 10625 datanode 10869 secondarynamenode 17113 resourcemanager 17434 nodemanager 10490 namenode 21566 jps中运行
SQL查询
create EXTERNAL DATA SOURCE [HDP2]
WITH (TYPE = HADOOP,
LOCATION = N'hdfs://xxx.xxx.x.x:54310',
CREDENTIAL = [HDPUser])
GO
create EXTERNAL TABLE [dbo].CLASS_DIM_EXP (
[CLASS_ID] [varchar](8) NOT NULL,
[CLASS_DESC] [varchar](100) NULL,
[INSERT_DATE] [datetime2](7) NOT NULL,
[LAST_UPDATE_DATETIME] [datetime2](7) NOT NULL)
WITH (LOCATION='/user/pdw_user',
DATA_SOURCE = HDP2,
FILE_FORMAT = TSV,
REJECT_TYPE = VALUE,
REJECT_VALUE = 0);
core-site.xml文件
<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:54310</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri's authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>
有什么需要改变的吗?
1条答案
按热度按时间vnjpjtjt1#
这里的问题是cores-site.xml文件包含hdfs://localhost:54310需要将其替换为相应的ip地址hdfs://..x.x:54310.