导入到hdfs时发生sqoop错误

hmae6n7t  于 2021-06-04  发布在  Hadoop
关注(0)|答案(3)|浏览(445)

我正在尝试使用sqoop将一个小表从mysql导出到hdfs。我可以通过sqoop列出数据库和表。但在将表导入hdfs时出现异常。请帮忙。下面是错误日志。

13/12/04 02:05:38 WARN conf.Configuration: session.id is deprecated. 
Instead, use   dfs.metrics.session-id
13/12/04 02:05:38 INFO jvm.JvmMetrics: 
Initializing JVM Metrics  withprocessName=JobTracker,sessionId=
13/12/04 02:05:39 INFO mapreduce.JobSubmitter: 
Cleaning up the staging area file:/tmp/hadoop-hadoop/mapred/staging/hadoop1439217057
/.staging/job_local1439217057_0001
13/12/04 02:05:39 ERROR 
security.UserGroupInformation:PriviledgedActionException  as:hadoop      
(auth:SIMPLE) 
cause:java.io.FileNotFoundException: 
File does not exist: hdfs://prat1:9000/home/hadoop/usr/sqoop-1.4.3-cdh4.3.0/lib/commons-    compress-1.4.1.jar
13/12/04 02:05:39 DEBUG util.ClassLoaderStack: 
Restoring classloader:sun.misc.Launcher$AppClassLoader@35a16869
13/12/04 02:05:39 ERROR tool.ImportTool: 
Encountered IOException running import job:  java.io.FileNotFoundException: File does not  exist:
hdfs://prat1:9000/home/hadoop/usr/
sqoop-1.4.3-cdh4.3.0/lib/commons-compress-1.4.1.jar
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:824)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
    at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:254)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:290)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:361)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1269)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1266)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1266)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1287)
    at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:173)
    at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:151)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:226)
    at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:555)
    at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:111)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
7cjasjjr

7cjasjjr1#

由于您没有写入权限,因此将获得security.usergroupinformation:priviledgedactionexception
尝试以hdfs用户身份登录,然后运行sqoop命令。
苏根
键入密码
su hdfs公司
然后运行sqoop导出命令。

avkwfej4

avkwfej42#

给予 /usr/lib/sqoop/lib/<mysqljavaconnector>-bin.jar 文件 777 权限,并确保 /usr/lib/sqoop/* 以及 /usr/local/hadoop/* 属于同一用户。

643ylb08

643ylb083#

尝试这样做: su root 键入根密码 su hdfs 然后跑 sqoop 命令,它会像冠军一样工作!

相关问题