When a property is changed on Consul, all logging inside of the application stops. I utilize @RefreshScope on a few @Configuration classes, nowhere else and on nothing related to logging. The application is started inside of a docker container and a few of the logging properties such as logging location and file name are picked up at runtime via environment variables – this works fine. We use logback for logging and it is configured via xml. I’m suspecting that the logback configuration reloads on a consul refresh event and isn’t able to continue logging afterwards.

I’ve tried removing all relevant logging properties from consul which didn’t solve the issue. I’ve made sure that all @RefreshScope annotations do not have anything to do with logback and this is true. I’m attaching the logback configuration below as I believe that to be the most relevant at the moment. Attempted at printing debug logs for logback but even after adding -Dlogback.debug=true or <configuration debug="true"> in the xml, there was no debug log output.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

    <springProperty scope="context" name="LOGGING_JSON_PATH" source="logging.json.path"/>
    <springProperty scope="context" name="LOGGING_TEXT_PATH" source="logging.text.path"/>
    <springProperty scope="context" name="LOG_FILE_NAME" source="logging.file.name"/>
    <springProperty scope="context" name="LOGGING_JOB_PATH" source="logging.job.path"/>
    <springProperty scope="context" name="LOGGING_JOB_FILE_NAME" source="logging.job.file.name"/>

    <if condition='isDefined("LOGGING_JSON_PATH")'>
        <then>
            <!-- Appender for SPLUNK -->
            <appender name="JSON_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
                <append>true</append>
                <File>${LOGGING_JSON_PATH}/${LOG_FILE_NAME}</File>
                <!--https://github.com/logstash/logstash-logback-encoder#standard-fields -->
                <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
                    <providers>
                        <mdc/>
                        <timestamp />
                        <version />
                        <threadName />
                        <pattern>
                            <pattern>
                                {
                                "message": "%message",
                                "severity":"%p",
                                "component": "%c{1}",
                                "line_number": "[%file:%line]",
                                "userid": "%X{userId}",
                                "sessionId": "%X{sessionId}",
                                "app": "project",
                                "stackTrace": "%X{stack_trace}"
                                }
                            </pattern>
                        </pattern>
                        <stackTrace>
                            <throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
                                <maxDepthPerThrowable>30</maxDepthPerThrowable>
                                <maxLength>2048</maxLength>
                                <shortenedClassNameLength>20</shortenedClassNameLength>
                                <exclude>^sun.reflect..*.invoke</exclude>
                                <exclude>^net.sf.cglib.proxy.MethodProxy.invoke</exclude>
                                <rootCauseFirst>true</rootCauseFirst>
                            </throwableConverter>
                        </stackTrace>
                    </providers>
                </encoder>

                <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
                    <!-- By setting the name to .gz here, we get free compression. -->
                    <fileNamePattern>${LOGGING_JSON_PATH}/${LOG_FILE_NAME}.%d{yyyy-MM-dd}.%i.gz</fileNamePattern>
                    <timeBasedFileNamingAndTriggeringPolicy
                            class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                        <maxFileSize>20MB</maxFileSize>
                    </timeBasedFileNamingAndTriggeringPolicy>
                    <!-- https://logback.qos.ch/manual/appenders.html#tbrpMaxHistory -->
                    <maxHistory>1</maxHistory>
                    <cleanHistoryOnStart>true</cleanHistoryOnStart>
                </rollingPolicy>
            </appender>
        </then>
    </if>

    ...
</configuration>

I have omitted most of the configuration for brewity. But the loggers and appenders work as expected and log to the expected files. When a consul environment change event is fired, all logging stops.

The logback environment variables are passed through docker-compose:

environment:
  - "SPRING_PROFILES_ACTIVE={{ key "/placeholder/project/SPRING_PROFILES_ACTIVE" }}"
  - "LOGGING_FILE_NAME={{ key "/placeholder/project/LOGGING_FILE_NAME" }}"
  - "LOGGING_JOB_PATH_FILE_NAME={{ key "/placeholder/project/LOGGING_JOB_PATH_FILE_NAME" }}"

The value and property names were changed to remove sensitive information so typos and mismatched names are possible.

Has anyone experienced something similar?