在aws上sqoop作业失败,出现连接错误

hgncfbus  于 2021-06-03  发布在  Sqoop
关注(0)|答案(1)|浏览(458)

我正在执行下面的sqoop命令,从另一个awsrds示例获取一个表,并将其传递给hdfs。


# !/bin/bash

sqoop import \
--connect jdbc:mysql://awsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com/financials_data \
--username someuser \
--password somepwd \
--table member_score \
--m 1 \
--target-dir /capstone/member_score

我可以使用工作台连接到这个服务器。但是,sqoop无法获取数据。
堆栈跟踪如下所示:

[ec2-user@ip-10-0-0-238 capstone]$ ./DataIngestion.txt
Warning: /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/01/03 03:56:45 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.15.1
20/01/03 03:56:45 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/01/03 03:56:45 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/01/03 03:56:45 INFO tool.CodeGenTool: Beginning code generation
20/01/03 03:58:52 ERROR manager.SqlManager: Error executing statement: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at

堆栈跟踪显示连接错误。但是,我可以使用mysql工作台连接

dvtswwa3

dvtswwa31#

由于sqoop提供了一个连接错误,我尝试ping服务器。

[ec2-user@ip-10-0-0-238 capstone]$ ping awsrds.cpclxrkdvwmz.us-east-1.rds.amazonaws.com
PING ec2-3-211-175-82.compute-1.amazonaws.com (3.211.175.82) 56(84) bytes of data.
^C
--- ec2-3-211-175-82.compute-1.amazonaws.com ping statistics ---
2935 packets transmitted, 0 received, 100% packet loss, time 2934021ms

当看到服务器不可访问时,下一步要检查的是 Security Groups aws上的设置。
出站规则应允许 All Traffic
我之前已经准备好了 Outbound 到特定ip。因为每次静态ip不工作时,aws都会分配新的ip。

相关问题