我们有spark结构化流应用程序（在aws emr上运行）并使用spark launcher提交，我想收集spark结构化流应用程序自定义日志。为此，我将log4j.properties文件发送到外部。对于外部log4j文件，我遵循以下步骤。
https://www.youtube.com/watch?v=omeoa8c51ls 而且，http://shzhangji.com/blog/2015/05/31/spark-streaming-logging-configuration/
但是我在网络用户界面上看不到这些日志。
下面是我的log4j-driver.properties文件。

log4j.rootLogger=INFO,rolling
log4j.appender.rolling=org.apache.logg4j.RollingFileAppender
log4j.appender.rolling.layout=org.apache.log4j.PatternLayout
log4j.appender.rolling.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.rolling.maxFileSize=50MB
log4j.appender.rolling.maxBackupIndex=10
log4j.appender.rolling.file=${spark.yarn.app.container.log.dir}/${app.logging.name}-driver.log
log4j.appender.rolling.encoding=UTF-8
log4j.logger.org.apache.spark=${app.logging.level}
log4j.logger.org.eclipse.jetty=WARN

下面是我提交spark结构化流应用程序的代码片段。

class LetsLaunchSparkForStreaming {
  val logger:Logger = Logger.getLogger(this.getClass)

  def letsLaunchSpark(ag1:String,arg2:String,arg3:String,arg4:String): Unit = {
    logger.info("topicName" + arg1)
    val countDownLatch = new CountDownLatch(1)
    val log4jDriverProperties:String = "-Dapp.logging.level=DEBUG -Dlog4j.configuration=file:/location/to/log4j-driver.properties -Dapp.logging.name="+arg1
    val log4jExecutorProperties:String = "-Dapp.logging.level=DEBUG -Dlog4j.configuration=file:/location/to/log4j-executor.properties -Dapp.logging.name="+arg1
    val env = Map("HADOOP_CONF_DIR" -> "/hadoop/home/path")
    val sparkLauncher = new SparkLauncher(env.asJava)
      .setVerbose(true)
      .setAppName("Application Name"+"_Streaming._Query")
      .setSparkHome("/Spark/Home")
      .setAppResource(/spark/streaming.application.jar)
      .setMainClass("main.Class")
      .setMaster("yarn")
      .setDeployMode("cluster")
      .setConf("spark.dynamicAllocation.enabled","true")
      .setConf("spark.dynamicAllocation.minExecutors","2")
      .setConf("spark.dynamicAllocation.initialExecutors","4")
      .setConf("spark.dynamicAllocation.maxExecutors","75")
      .setConf("spark.yarn.driver.memoryOverhead","2048")
      .setConf("spark.yarn.executor.memoryOverhead","9246")
      .setConf("spark.driver.extraJavaOptions",log4jDriverProperties)
      .setConf("spark.executor.extraJavaOptions",log4jExecutorProperties)
     // .addFile("/full/path/to/log4j-driver.properties")
      .addSparkArg("--driver-memory",driver_memory)
      .addSparkArg("--driver-cores",driver_cores)
      .addSparkArg("--executor-memory",executor_memory)
      .addSparkArg("--executor-cores",executor_cores)
      .addSparkArg("--num-executors",num_executors)
      .addSparkArg("--queue",spark_queue)
      .addAppArgs(arg2,arg3,arg4)
      .startApplication()
}
}

有人能帮我收集spark结构化流媒体应用程序的定制日志吗，
提前谢谢。