hadoop Spark-Submit命令需要一段时间才能运行

jdg4fx2g  于 2022-11-01  发布在  Hadoop
关注(0)|答案(1)|浏览(219)

我们已经在运行IBM AIX（版本2）操作系统的服务器集群上安装了Apache Hadoop和Spark。
Hadoop版本- Hadoop-3.2.1Spark版本-Spark-3.0.1
我们正在通过运行**$SPARK_HOME/bin文件夹下的spark-submit --version**命令来测试Spark的整体安装。此命令间歇性运行。当您第一次运行时，命令运行时没有延迟。之后多次运行该命令需要很长时间才能执行（大约30 - 40分钟）。我们已经检查了服务器CPU和内存-不存在内存不足或应用程序占用处理器资源的问题。我们无法确定此命令运行时的延迟位置。
此Hadoop/Spark设置在运行Red Hat 7.9的集群中运行。在此环境中，我们不会遇到此问题。
这是我第一次问关于堆栈溢出的问题。如果我需要提供更多的信息，请告诉我。
先谢谢你。
==========================编辑5月11日：
日志运行成功（在spark-submit命令中添加了调试行）

  1. bash-5.0$ spark-submit --version
  2. Entered spark submit
  3. About to execute spark submit command.....
  4. About to load spark env.sh
  5. Loaded spark env.sh
  6. Entered statement to create RUNNER
  7. searching spark_home/jars
  8. Loaded spark jars DIR
  9. Launching class path
  10. Launched class path
  11. Entering build command
  12. Completed build command
  13. About to enter while block
  14. Entered while block for Entered build command
  15. Entered build command
  16. CMD is
  17. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  18. Entered while block for
  19. For  changing delim to blank
  20. CMD is
  21. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  22. Entered while block for /u01/app/java8_64/bin/java
  23. Entered if condition for /u01/app/java8_64/bin/java
  24. CMD is /u01/app/java8_64/bin/java
  25. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  26. Entered while block for -cp
  27. Entered if condition for -cp
  28. CMD is /u01/app/java8_64/bin/java -cp
  29. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  30. Entered while block for /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/
  31. Entered if condition for /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/
  32. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/
  33. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  34. Entered while block for -Xmx1g
  35. Entered if condition for -Xmx1g
  36. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g
  37. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  38. Entered while block for org.apache.spark.deploy.SparkSubmit
  39. Entered if condition for org.apache.spark.deploy.SparkSubmit
  40. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit
  41. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  42. Entered while block for --version
  43. Entered if condition for --version
  44. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit --version
  45. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  46. Entered while block for 0
  47. Entered if condition for 0
  48. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit --version 0
  49. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  50. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit --version 0
  51. completed while block
  52. About to execute /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g 
  53.    org.apache.spark.deploy.SparkSubmit --version
  54. Welcome to
  55.       ____              __
  56.      / __/__  ___ _____/ /__
  57.     _\ \/ _ \/ _ `/ __/  '_/
  58.    /___/ .__/\_,_/_/ /_/\_\   version 3.0.1
  59.       /_/
  61. Using Scala version 2.12.10, IBM J9 VM, 1.8.0_251
  62. Branch HEAD
  63. Compiled by user ubuntu on 2020-08-28T08:58:35Z
  64. Revision 2b147c4cd50da32fe2b4167f97c8142102a0510d
  65. Url https://gitbox.apache.org/repos/asf/spark.git
  66. Type --help for more information.

=============================================================
失败的运行：

  1. bash-5.0$ spark-submit --version
  2. Entered spark submit
  3. About to execute spark submit command.....
  4. About to load spark env.sh
  5. Loaded spark env.sh
  6. Entered statement to create RUNNER
  7. searching spark_home/jars
  8. Loaded spark jars DIR
  9. Launching class path
  10. Launched class path
  11. Entering build command
  12. Completed build command
  13. About to enter while block
  14. Entered while block for Entered build command
  15. Entered build command
  16. CMD is
  17. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  18. Entered while block for
  19. For  changing delim to blank
  20. CMD is
  21. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  22. Entered while block for /u01/app/java8_64/bin/java
  23. Entered if condition for /u01/app/java8_64/bin/java
  24. CMD is /u01/app/java8_64/bin/java
  25. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  26. Entered while block for -cp
  27. Entered if condition for -cp
  28. CMD is /u01/app/java8_64/bin/java -cp
  29. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  30. Entered while block for /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/
  31. Entered if condition for /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/
  32. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/
  33. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  34. Entered while block for -Xmx1g
  35. Entered if condition for -Xmx1g
  36. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g
  37. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  38. Entered while block for org.apache.spark.deploy.SparkSubmit
  39. Entered if condition for org.apache.spark.deploy.SparkSubmit
  40. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit
  41. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  42. Entered while block for --version
  43. Entered if condition for --version
  44. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit --version
  45. build_command is  and org.apache.spark.deploy.SparkSubmit --version
  46. Entered while block for 0
  47. Entered if condition for 0
  48. CMD is /u01/app/java8_64/bin/java -cp /u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/conf/:/u01/app/rmb/ria/AnthemSpark/spark-3.0.1-bin-hadoop3.2/jars/*:/u01/app/rmb/ria/AnthemSpark/hadoop-3.2.1/etc/hadoop/ -Xmx1g org.apache.spark.deploy.SparkSubmit --version 0
  49. build_command is  and org.apache.spark.deploy.SparkSubmit --version

##########################

编辑-3月12日
这是运行下面的命令- truss -d时的最后几行。最后的输出显示它进入“休眠”状态。

  1. 0.9063:        lseek(0, 0, 1)                   Err#29 ESPIPE
  2. 0.9066:        fstatx(0, 0x0FFFFFFFFFFFE8F8, 176, 0) = 0
  3. 0.9068:        _sigaction(14, 0x0FFFFFFFFFFFE710, 0x0FFFFFFFFFFFE740) = 0
  4. 0.9071:        incinterval(0, 0x0FFFFFFFFFFFE640, 0x0FFFFFFFFFFFE660) = 0
  5. 0.9073:        kread(0, " o", 1)                = 1
  6. 0.9075:        kread(0, " r", 1)                = 1
  7. 0.9078:        kread(0, " g", 1)                = 1
  8. 0.9080:        kread(0, " .", 1)                = 1
  9. 0.9082:        kread(0, " a", 1)                = 1
  10. 0.9084:        kread(0, " p", 1)                = 1
  11. 0.9086:        kread(0, " a", 1)                = 1
  12. 0.9089:        kread(0, " c", 1)                = 1
  13. 0.9091:        kread(0, " h", 1)                = 1
  14. 0.9093:        kread(0, " e", 1)                = 1
  15. 0.9095:        kread(0, " .", 1)                = 1
  16. 0.9097:        kread(0, " s", 1)                = 1
  17. 0.9100:        kread(0, " p", 1)                = 1
  18. 0.9102:        kread(0, " a", 1)                = 1
  19. 0.9104:        kread(0, " r", 1)                = 1
  20. 0.9106:        kread(0, " k", 1)                = 1
  21. 0.9108:        kread(0, " .", 1)                = 1
  22. 0.9111:        kread(0, " d", 1)                = 1
  23. 0.9113:        kread(0, " e", 1)                = 1
  24. 0.9115:        kread(0, " p", 1)                = 1
  25. 0.9117:        kread(0, " l", 1)                = 1
  26. 0.9119:        kread(0, " o", 1)                = 1
  27. 0.9122:        kread(0, " y", 1)                = 1
  28. 0.9124:        kread(0, " .", 1)                = 1
  29. 0.9126:        kread(0, " S", 1)                = 1
  30. 0.9128:        kread(0, " p", 1)                = 1
  31. 0.9130:        kread(0, " a", 1)                = 1
  32. 0.9132:        kread(0, " r", 1)                = 1
  33. 0.9135:        kread(0, " k", 1)                = 1
  34. 0.9137:        kread(0, " S", 1)                = 1
  35. 0.9139:        kread(0, " u", 1)                = 1
  36. 0.9141:        kread(0, " b", 1)                = 1
  37. 0.9143:        kread(0, " m", 1)                = 1
  38. 0.9187:        kread(0, " i", 1)                = 1
  39. 0.9190:        kread(0, " t", 1)                = 1
  40. 0.9192:        kread(0, "\0", 1)                = 1
  41. 0.9195:        incinterval(0, 0x0FFFFFFFFFFFE5C0, 0x0FFFFFFFFFFFE5E0) = 0
  42. 0.9197:        _sigaction(14, 0x0FFFFFFFFFFFE690, 0x0FFFFFFFFFFFE6C0) = 0
  43. 0.9200:        kfcntl(1, F_GETFL, 0x0000000000000000) = 67110914
  44. 0.9204:        kfcntl(1, F_GETFL, 0x0000000000000000) = 67110914
  45. 0.9207:        kioctl(0, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  46. 0.9211:        lseek(0, 0, 1)                   Err#29 ESPIPE
  47. 0.9214:        fstatx(0, 0x0FFFFFFFFFFFE8F8, 176, 0) = 0
  48. 0.9216:        _sigaction(14, 0x0FFFFFFFFFFFE710, 0x0FFFFFFFFFFFE740) = 0
  49. 0.9219:        incinterval(0, 0x0FFFFFFFFFFFE640, 0x0FFFFFFFFFFFE660) = 0
  50. 0.9222:        kread(0, " -", 1)                = 1
  51. 0.9224:        kread(0, " -", 1)                = 1
  52. 0.9227:        kread(0, " v", 1)                = 1
  53. 0.9229:        kread(0, " e", 1)                = 1
  54. 0.9231:        kread(0, " r", 1)                = 1
  55. 0.9234:        kread(0, " s", 1)                = 1
  56. 0.9236:        kread(0, " i", 1)                = 1
  57. 0.9238:        kread(0, " o", 1)                = 1
  58. 0.9241:        kread(0, " n", 1)                = 1
  59. 0.9243:        kread(0, "\0", 1)                = 1
  60. 0.9245:        incinterval(0, 0x0FFFFFFFFFFFE5C0, 0x0FFFFFFFFFFFE5E0) = 0
  61. 0.9248:        _sigaction(14, 0x0FFFFFFFFFFFE690, 0x0FFFFFFFFFFFE6C0) = 0
  62. 0.9251:        kfcntl(1, F_GETFL, 0x0000000000000000) = 67110914
  63. 0.9254:        kfcntl(1, F_GETFL, 0x0000000000000000) = 67110914
  64. 0.9257:        kioctl(0, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  65. 0.9260:        lseek(0, 0, 1)                   Err#29 ESPIPE
  66. 0.9262:        fstatx(0, 0x0FFFFFFFFFFFE8F8, 176, 0) = 0
  67. 0.9265:        _sigaction(14, 0x0FFFFFFFFFFFE710, 0x0FFFFFFFFFFFE740) = 0
  68. 0.9268:        incinterval(0, 0x0FFFFFFFFFFFE640, 0x0FFFFFFFFFFFE660) = 0
  69. 0.9270:        kread(0, " 0", 1)                = 1
  70. 0.9273:        kread(0, "\0", 1)                = 1
  71. 0.9275:        incinterval(0, 0x0FFFFFFFFFFFE5C0, 0x0FFFFFFFFFFFE5E0) = 0
  72. 0.9278:        _sigaction(14, 0x0FFFFFFFFFFFE690, 0x0FFFFFFFFFFFE6C0) = 0
  73. 0.9281:        kfcntl(1, F_GETFL, 0x0000000000000000) = 67110914
  74. 0.9284:        kfcntl(1, F_GETFL, 0x0000000000000020) = 67110914
  75. 0.9287:        kioctl(0, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  76. 0.9290:        lseek(0, 0, 1)                   Err#29 ESPIPE
  77. 0.9292:        fstatx(0, 0x0FFFFFFFFFFFE8F8, 176, 0) = 0
  78. 0.9295:        _sigaction(14, 0x0FFFFFFFFFFFE710, 0x0FFFFFFFFFFFE740) = 0
  79. 0.9297:        incinterval(0, 0x0FFFFFFFFFFFE640, 0x0FFFFFFFFFFFE660) = 0
  80. 2.9303:        kread(0, "\t", 1) (sleeping...)
hadoop

1条答案

按热度按时间
q35jwt9p

q35jwt9p1#

@LorinczyZsigmond的建议最终把我们带到了正确的地方。搜索最后一行-kread(0, "\t", 1) (sleeping...)-我们找到了Issue with Bash read builtin in latest release，在那里有传言说其他人在IBM AIX Linux Toolbox网站上发布的那个版本上遇到了同样的问题。在更新到版本www.example.com后5.1.4.2，这个问题消失了，我们能够完成脚本。

赞(0）分享  回复(0）举报  2022-11-01
我来回答

相关问题

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com