apache flink-无法使用log4j创建小时/每日日志文件

fcwjkofz  于 2021-06-24  发布在  Flink
关注(0)|答案(1)|浏览(663)

我不能用log4j创建每日和每小时的日志文件(特别是taskexecutor日志)
这是我的log4j属性


# This affects logging for both user code and Flink

log4j.rootLogger=INFO, file

# Uncomment this if you want to _only_ change Flink's logging

# log4j.logger.org.apache.flink=INFO

# The following lines keep the log level of common libraries/connectors on

# log level INFO. The root logger does not override this. You have to manually

# change the log levels here.

log4j.logger.akka=INFO
log4j.logger.org.apache.kafka=INFO
log4j.logger.org.apache.hadoop=INFO
log4j.logger.org.apache.zookeeper=INFO

# Log all infos in the given file

log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.File=${log.file}
log4j.appender.file.DatePattern='.'yyyy-MM-dd-HH
log4j.appender.file.append=false
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n

${log.file}指向:

2020-02-13 17:40:51,105 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dlog.file=/.../flink-1.9.1/log/flink-tarantula-taskexecutor-0-...log

同时,还有一个logback.xml配置(flink使用哪种配置?)
logback.xml文件:

<configuration>
    <appender name="file" class="ch.qos.logback.core.FileAppender">
        <file>${log.file}</file>
        <append>false</append>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{60} %X{sourceThread} - %msg%n</pattern>
        </encoder>
    </appender>

    <!-- This affects logging for both user code and Flink -->
    <root level="INFO">
        <appender-ref ref="file"/>
    </root>

    <!-- Uncomment this if you want to only change Flink's logging -->
    <!--<logger name="org.apache.flink" level="INFO">-->
        <!--<appender-ref ref="file"/>-->
    <!--</logger>-->

    <!-- The following lines keep the log level of common libraries/connectors on
         log level INFO. The root logger does not override this. You have to manually
         change the log levels here. -->
    <logger name="akka" level="INFO">
        <appender-ref ref="file"/>
    </logger>
    <logger name="org.apache.kafka" level="INFO">
        <appender-ref ref="file"/>
    </logger>
    <logger name="org.apache.hadoop" level="INFO">
        <appender-ref ref="file"/>
    </logger>
    <logger name="org.apache.zookeeper" level="INFO">
        <appender-ref ref="file"/>
    </logger>

    <!-- Suppress the irrelevant (wrong) warnings from the Netty channel handler -->
    <logger name="org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline" level="ERROR">
        <appender-ref ref="file"/>
    </logger>
</configuration>

这是flink/lib文件夹 apache-log4j-extras-1.2.17.jar flink-dist_2.11-1.9.1.jar flink-table_2.11-1.9.1.jar flink-table-blink_2.11-1.9.1.jar log4j-1.2.17.jar slf4j-log4j12-1.7.15.jar ###最终解决方案
经过所有的尝试,我找到了解决方案,我不能创建每日日志文件,因为这个语句阻止创建每日文件。

log4j.logger.org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline=ERROR, filelog4j.rootLogger=INFO, file

在每天中午和午夜创建日志的示例。

log4j.category.org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline=ERROR, nettyFileAppender
log4j.rootLogger=INFO, file

# Uncomment this if you want to _only_ change Flink's logging

# log4j.logger.org.apache.flink=INFO

# The following lines keep the log level of common libraries/connectors on

# log level INFO. The root logger does not override this. You have to manually

# change the log levels here.

log4j.logger.akka=INFO
log4j.logger.org.apache.kafka=INFO
log4j.logger.org.apache.hadoop=INFO
log4j.logger.org.apache.zookeeper=INFO

# Log all infos in the given file

log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.file=${log.file}
log4j.appender.file.append=false
log4j.appender.file.DatePattern='.'yyyy-MM-dd-a
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n

# Suppress the irrelevant (wrong) warnings from the Netty channel handler

log4j.appender.nettyFileAppender=org.apache.log4j.FileAppender
log4j.appender.nettyFileAppender.file=/path/to/nettyLog/nettyChannelIrrelevant.log
log4j.appender.nettyFileAppender.append=false
log4j.appender.nettyFileAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.nettyFileAppender.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
ldxq2e6h

ldxq2e6h1#

您没有指定如何提供 log4j.properties 文件。但一个常见的问题是,有人在jar中有这个文件,它被忽略了,因为flink将使用 conf/log4j.properties 文件位于群集上。
假设您的jar除了 slf4j-api jar,然后Flink会拿起 slf4j-log4j12.jarflink/lib (因为它在类路径上),因此使用 log4j (不是logback),所以 logback.xml 配置文件被忽略。

相关问题