hsql到hadoop使用sqoop/套接字创建错误

6ljaweal  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(191)

我正在尝试使用sqoop1.4.6.2.6.5.0-292版本将数据从hypersql数据库表导入hadoop dfs。我将试着总结一下我采取的步骤：
首先，我下载并解压了hsqlversion1.8.0.10（这与sqoop中使用的版本相同）。我添加了一个server.properties文件，并将数据库文件夹配置为： server.database.0=file:test/test 数据库服务已按如下方式启动：
nohup java-cp/data/programs/etu/hsqldb-1.8.0/hsqldb/lib/hsqldb.jar org.hsqldb.server&
在创建了一个名为rtu sales的表并用数据填充它之后，我能够使用 sqoop list-tables --connect "jdbc:hsqldb:hsql://localhost" --connection-manager "org.apache.sqoop.manager.HsqldbManager" --username "SA" 结果显示了我所希望的单表rt\u销售额。但是，在尝试使用

sqoop import --num-mappers 1 --connect "jdbc:hsqldb:hsql://localhost" --connection-manager "org.apache.sqoop.manager.HsqldbManager" --username "SA" --table "RT_SALES" --target-dir "/some-folder"

我得到以下错误：

19/05/08 20:16:13 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.5.0-292
19/05/08 20:16:13 INFO manager.SqlManager: Using default fetchSize of 1000
19/05/08 20:16:13 INFO tool.CodeGenTool: Beginning code generation
19/05/08 20:16:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "RT_SALES" AS t WHERE 1=0
19/05/08 20:16:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "RT_SALES" AS t WHERE 1=0
19/05/08 20:16:14 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.6.5.0-292/hadoop-mapreduce
Note: /tmp/sqoop-user1/compile/50b3c7f6e0257e6f477a23139bedcb39/RT_SALES.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
19/05/08 20:16:16 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-user1/compile/50b3c7f6e0257e6f477a23139bedcb39/RT_SALES.jar
19/05/08 20:16:16 INFO mapreduce.ImportJobBase: Beginning import of RT_SALES
19/05/08 20:16:17 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "RT_SALES" AS t WHERE 1=0
19/05/08 20:16:18 INFO client.AHSProxy: Connecting to Application History server at server1/100.00.000.00:10200
19/05/08 20:16:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 70501 for user1 on ha-hdfs:server1
19/05/08 20:16:18 INFO security.TokenCache: Got dt for hdfs://server1; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:server1, Ident: (HDFS_DELEGATION_TOKEN token 70501 for user1)
19/05/08 20:16:18 INFO client.RequestHedgingRMFailoverProxyProvider: Looking for the active RM in [rm1, rm2]...
19/05/08 20:16:18 INFO client.RequestHedgingRMFailoverProxyProvider: Found active RM [rm2]
19/05/08 20:16:21 INFO db.DBInputFormat: Using read commited transaction isolation
19/05/08 20:16:21 INFO mapreduce.JobSubmitter: number of splits:1
19/05/08 20:16:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1557311883605_0041
19/05/08 20:16:21 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:server1, Ident: (HDFS_DELEGATION_TOKEN token 70501 for user1)
19/05/08 20:16:22 INFO impl.TimelineClientImpl: Timeline service address: http://server1:8188/ws/v1/timeline/
19/05/08 20:16:23 INFO impl.YarnClientImpl: Submitted application application_1557311883605_0041
19/05/08 20:16:23 INFO mapreduce.Job: The url to track the job: http://server1:8088/proxy/application_1557311883605_0041/
19/05/08 20:16:23 INFO mapreduce.Job: Running job: job_1557311883605_0041
19/05/08 20:16:31 INFO mapreduce.Job: Job job_1557311883605_0041 running in uber mode : false
19/05/08 20:16:31 INFO mapreduce.Job:  map 0% reduce 0%
19/05/08 20:16:37 INFO mapreduce.Job: Task Id : attempt_1557311883605_0041_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: socket creation error
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:751)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
Caused by: java.lang.RuntimeException: java.sql.SQLException: socket creation error
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:220)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:165)
        ... 9 more
Caused by: java.sql.SQLException: socket creation error
        at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
        at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
        at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
        at org.hsqldb.jdbcDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:302)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
        ... 10 more

从其他帖子来看，这条消息表明hsqldb没有运行。但是，我可以使用集成的hsql客户机进行连接，而且如上所述，sqoop至少可以查询表—因此服务必须运行。
我已经试过了（没有成功）：
为hsqldb定义不同的端口（9002而不是9001）
为用户sa创建密码
设置数据库名（“test”而不是“”）
更新：将服务器属性silent设置为false后，我收到以下日志：

[Server@49476842]: [Thread[main,5,main]]: checkRunning(false) entered
[Server@49476842]: [Thread[main,5,main]]: checkRunning(false) exited
[Server@49476842]: Startup sequence initiated from main() method
[Server@49476842]: Loaded properties from [/data/programs/ETU/hsqldb-1.8.0/hsqldb/server.properties]
[Server@49476842]: [Thread[main,5,main]]: start() entered
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: run() entered
[Server@49476842]: Initiating startup sequence...
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.tls=false
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.port=9002
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.trace=true
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.database.0=file:test/test
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.restart_on_shutdown=false
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.no_system_exit=false
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.silent=false
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.default_page=index.html
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.address=0.0.0.0
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.dbname.0=test
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: server.root=.
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: openServerSocket() entered
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: Got server socket: ServerSocket[addr=0.0.0.0/0.0.0.0,localport=9002]
[Server@49476842]: Server socket opened successfully in 5 ms.
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: openServerSocket() exiting
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: openDatabases() entered
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: Opening database: [file:test/test]
[Server@49476842]: Database [index=0, id=0, db=file:test/test, alias=test] opened sucessfully in 118 ms.
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: openDatabases() exiting
[Server@49476842]: Startup sequence completed in 124 ms.
[Server@49476842]: 2019-05-10 12:59:46.675 HSQLDB server 1.8.0 is online
[Server@49476842]: To close normally, connect and execute SHUTDOWN SQL
[Server@49476842]: From command line, use [Ctrl]+[C] to abort abruptly
[Server@49476842]: [Thread[main,5,main]]: start() exiting
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: handleConnection(Socket[addr=/127.0.0.1,port=38030,localport=9002]) entered
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: handleConnection() exited
[Server@49476842]: [Thread[HSQLDB Connection @5e60432f,5,HSQLDB Connections @49476842]]: 0:trying to connect user SA
[Server@49476842]: 0:HSQLCLI:SETSESSIONATTR:AUTOCOMMIT null CONNECTION_READONLY null
[Server@49476842]: 0:HSQLCLI:SETSESSIONATTR:AUTOCOMMIT false CONNECTION_READONLY null
[Server@49476842]: 0:SQLCLI:SQLPREPARE SELECT t.* FROM "RT_SALES" AS t WHERE 1=0
[Server@49476842]: 0:SQLCLI:SQLEXECUTE:1
[Server@49476842]: 0:SQLCLI:SQLENDTRAN:COMMIT
[Server@49476842]: 0:SQLCLI:SQLFREESTMT:1
[Server@49476842]: 0:SQLCLI:SQLPREPARE SELECT t.* FROM "RT_SALES" AS t WHERE 1=0
[Server@49476842]: 0:SQLCLI:SQLEXECUTE:2
[Server@49476842]: 0:SQLCLI:SQLENDTRAN:COMMIT
[Server@49476842]: 0:SQLCLI:SQLFREESTMT:2
[Server@49476842]: 0:SELECT * FROM INFORMATION_SCHEMA.SYSTEM_PRIMARYKEYS WHERE 1=1 AND TABLE_NAME = 'RT_SALES'
[Server@49476842]: 0:SQLCLI:SQLENDTRAN:COMMIT
[Server@49476842]: 0:SELECT * FROM INFORMATION_SCHEMA.SYSTEM_PRIMARYKEYS WHERE 1=1 AND TABLE_NAME = 'RT_SALES'
[Server@49476842]: 0:SQLCLI:SQLENDTRAN:COMMIT
[Server@49476842]: 0:SQLCLI:SQLPREPARE SELECT t.* FROM "RT_SALES" AS t WHERE 1=0
[Server@49476842]: 0:SQLCLI:SQLEXECUTE:3
[Server@49476842]: 0:SQLCLI:SQLENDTRAN:COMMIT
[Server@49476842]: 0:SQLCLI:SQLFREESTMT:3
[Server@49476842]: 0:SQLCLI:SQLDISCONNECT
[Server@49476842]: [Thread[HSQLDB Connection @5e60432f,5,HSQLDB Connections @49476842]]: 0:disconnected SA
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: handleConnection(Socket[addr=/127.0.0.1,port=38190,localport=9002]) entered
[Server@49476842]: [Thread[HSQLDB Server @49476842,5,main]]: handleConnection() exited
[Server@49476842]: [Thread[HSQLDB Connection @7450cadd,5,HSQLDB Connections @49476842]]: 1:trying to connect user SA
[Server@49476842]: 1:HSQLCLI:SETSESSIONATTR:AUTOCOMMIT false CONNECTION_READONLY null
[Server@49476842]: 1:call "org.hsqldb.Library.getDatabaseProductName"()
[Server@49476842]: 1:HSQLCLI:SETSESSIONATTR:AUTOCOMMIT null CONNECTION_READONLY null
[Server@49476842]: 1:SQLCLI:SQLDISCONNECT
[Server@49476842]: [Thread[HSQLDB Connection @7450cadd,5,HSQLDB Connections @49476842]]: 1:disconnected SA
hadoopsqoophsqldb

暂无答案！

目前还没有任何答案，快来回答吧！

我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com