我有一个python脚本，它使用pyspark来处理一些数据，然后我尝试在aws glue job上运行它，有时job成功了，但有时失败了，我是glue新手，这是我在日志中发现的成功运行和失败运行的区别之一：
有线的事情是，没有改变任何配置或脚本，这个错误只是偶尔发生，非常奇怪。
为了成功运行：

Log Contents:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/yarn/usercache/root/filecache/54/glue-assembly.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/yarn/usercache/root/filecache/19/__spark_libs__5130541944173628908.zip/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Continuous Logging: Creating cloudwatch appender.
Continuous Logging: Creating cloudwatch appender.
21/04/13 17:13:09 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 12345@ip-123-12-123-123
21/04/13 17:13:09 INFO SignalUtils: Registered signal handler for TERM
21/04/13 17:13:09 INFO SignalUtils: Registered signal handler for HUP
21/04/13 17:13:09 INFO SignalUtils: Registered signal handler for INT
.

对于失败的运行：

Log Contents:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/yarn/usercache/root/filecache/53/glue-assembly.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/yarn/usercache/root/filecache/19/__spark_libs__1502454731345361680.zip/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Continuous Logging: Creating cloudwatch appender.
Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.protocol.json.JsonErrorShapeMetadata.withExceptionUnmarshaller(Lcom/amazonaws/transform/JsonErrorUnmarshaller;)Lcom/amazonaws/protocol/json/JsonErrorShapeMetadata;
    at com.amazonaws.services.logs.AWSLogsClient.<clinit>(AWSLogsClient.java:112)
    at com.amazonaws.services.logs.AWSLogsClientBuilder.build(AWSLogsClientBuilder.java:61)
    at com.amazonaws.services.logs.AWSLogsClientBuilder.build(AWSLogsClientBuilder.java:27)
    at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
    at com.amazonaws.services.glue.cloudwatch.CloudWatchLogsAppenderCommon.<init>(CloudWatchLogsAppenderCommon.java:86)
    at com.amazonaws.services.glue.cloudwatch.CloudWatchLogsAppenderCommon$CloudWatchLogsAppenderCommonBuilder.build(CloudWatchLogsAppenderCommon.java:79)
    at com.amazonaws.services.glue.cloudwatch.CloudWatchAppender.activateOptions(CloudWatchAppender.java:58)
    at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
    at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
    at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
    at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
    at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:120)
    at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:108)
    at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:785)
    at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)
    at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:785)
    at org.apache.spark.internal.Logging$class.log(Logging.scala:46)
    at org.apache.spark.deploy.yarn.ApplicationMaster$.log(ApplicationMaster.scala:785)
    at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:800)
    at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
End of LogType:stderr
LogType:stdout
Log Upload Time:Wed Apr 14 14:33:48 +0000 2021
LogLength:0
Log Contents:
End of LogType:stdout

这一行之后就发生了变化 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 那么 Continuous Logging: Creating cloudwatch appender. ，然后出现一些错误 Exception in thread "main" java.lang.NoSuchMethodError: com.amazonaws.protocol.json.JsonErrorShapeMetadata.withExceptionUnmarshaller(Lcom/amazonaws/transform/JsonErrorUnmarshaller;)Lcom/amazonaws/protocol/json/JsonErrorShapeMetadata; 我不太明白，有人能帮我吗？谢谢。