sqoop导出不起作用

eufgjt7s 于 2021-05-29 发布在 Hadoop

关注(0)|答案(2)|浏览(482)

我正在尝试将数据从hdfs位置导出到greenplum用户定义的模式（不是默认模式）。
尝试sqoop eval检查连接。

sqoop eval --connect "jdbc:postgresql://sample.com:5432/sampledb" --username sample_user --password xxxx --query "SELECT * FROM sample_db.sample_table LIMIT 3"

结果：工作正常
试用 --schema 选项

/usr/bin/sqoop export --connect "jdbc:postgresql://sample.com:5432/sampledb" --username sampleuser --password samplepassword --table sample_table --schema sample_schema --export-dir=/sample/gp_export --input-fields-terminated-by ',' --update-mode allowinsert

结果：

Warning: /usr/hdp/2.3.6.0-3796/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/06/25 11:09:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.6.0-3796
18/06/25 11:09:41 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Error parsing arguments for export:
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --schema
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: sample_schema
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --export-dir=/sample/gp_export
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --input-fields-terminated-by
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: ,
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --update-mode
18/06/25 11:09:41 ERROR tool.BaseSqoopTool: Unrecognized argument: allowinsert

额外增加 '--' 之前 '--schema' 基于sqoop文档https://sqoop.apache.org/docs/1.4.3/sqoopuserguide.html

/usr/bin/sqoop export --connect "jdbc:postgresql://sample.com:5432/sampledb" --username sampleuser --password samplepassword --table sample_table -- --schema sample_schema --export-dir=/sample/gp_export --input-fields-terminated-by ',' --update-mode allowinsert

结果：

Warning: /usr/hdp/2.3.6.0-3796/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/06/25 11:06:26 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.3.6.0-3796
18/06/25 11:06:26 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
Export requires an --export-dir argument or --hcatalog-table argument.
Try --help for usage instructions.

有人能给我指点一下吗。谢谢

hadoop hdfs sqoop greenplum

来源：https://stackoverflow.com/questions/51049838/sqoop-export-from-hdfs-to-greenplum-is-not-working

2条答案

按热度按时间

new9mtju1#

感谢@cricket\u 007的澄清。 --schema 参数应该是sqoop命令中的最后一个。所以下面的代码是有效的。

/usr/bin/sqoop export --connect "jdbc:postgresql://sample.com:5432/sampledb"  \ 
--username sampleuser --password samplepassword  \ 
--export-dir=/sample/gp_export --input-fields-terminated-by ','  \ 
--table sample_table -- --schema sample_schema

但是 UPSERT postgressql中不支持的操作。这里有一张开放的吉拉门票。https://issues.apache.org/jira/browse/sqoop-1270

赞(0）回复(0）举报 2021-05-29

ygya80vv2#

之后 --export-dir 你不需要 = 看看下面的例子。另一个建议是使用 --verbose 当你遇到这些问题的时候。

sqoop export --libjars /path/some.jar \
--connect 'jdbc:sqlserver://IP:1433;database=db' \
--username someName -password somePassword -m 10 \
--verbose --mysql-delimiters \
--export-dir /HDFS/Path/someFile.csv \
--table "RDBMSTABLENAME"

赞(0）回复(0）举报 2021-05-29

我来回答

sqoop导出不起作用

2条答案

相关问题

热门标签

最新问答