Logs are getting auto deleted after the date is changed.

[payaljain2003]

Bug report

Bug description:

I have written a code that would rotate logs on midnight maintaing backup of 1 day. But the next day, the previous logs are getting deleted.

For example :
While I am checking logs on 14th Feb, I noticed that all the logs generated on 13th Feb, got deleted from instancelogs.log.2025-02-13. The first log generated on 14th Feb is getting stored in instancelogs.log.2025-02-13 file instead of instancelogs.log file , the remaining logs for 14th Feb are correctly stored in instancelogs.log file.

Please look into the issue as we are losing all the logs for 13th of Feb.

I am using below code to handle logs:

import logging
from logging.handlers import TimedRotatingFileHandler


#logger instances for PPT creator
LOG_FILE_PATH = "../STORE_LOGS/instancelogs.log"

instance_logger = logging.getLogger('instance_logger')

instance_logger_file_handler = None


instance_logger_file_handler = TimedRotatingFileHandler(LOG_FILE_PATH, when='midnight', interval=1, backupCount=1)
instance_logger_file_formatter = logging.Formatter(
    fmt="[%(asctime)s] - [%(levelname)s]  - [%(filename)s] - [%(funcName)s] - %(message)s",
    datefmt="%d-%m-%y %H:%M:%S")
instance_logger_file_handler.setFormatter(instance_logger_file_formatter)

instance_logger.setLevel(logging.DEBUG)
instance_logger.addHandler(instance_logger_file_handler)

CPython versions tested on:

3.9

Operating systems tested on:

Linux

[hugovk]

Hello, this issue tracker is for bugs and feature requests in CPython itself.

Please post in the help forum for questions about your own code:

https://discuss.python.org/c/help/7

[payaljain2003]

@hugovk this seems to be a bug for logging module. Is thi snot valid to report the bug here..

[hugovk]

Yes, if so, please can you produce a minimal, self-contained code sample that reproduces the problem?

See https://stackoverflow.com/help/minimal-reproducible-example for tips.

[payaljain2003]

@hugovk Thank you for the information.
This is the minimal code I have provided above; to best understand what are the log related config I have setup so that the bug can be reproducible.
Is is please possible for you to reopen the bug, so that it can be looked into.

[hugovk]

You have backupCount=1, the docs say:

If backupCount is nonzero, at most backupCount files will be kept, and if more would be created when rollover occurs, the oldest one is deleted. The deletion logic uses the interval to determine which files to delete, so changing the interval may leave old files lying around.

And you have a single instancelogs.log file?

If you change it to backupCount=2, do you get instancelogs.log file and one for yesterday?

[hugovk]

Oh I see:

While I am checking logs on 14th Feb, I noticed that all the logs generated on 13th Feb, got deleted from instancelogs.log.2025-02-13. The first log generated on 14th Feb is getting stored in instancelogs.log.2025-02-13 file instead of instancelogs.log file , the remaining logs for 14th Feb are correctly stored in instancelogs.log file.

Actual result:

• On 14th Feb:
  • 13th Feb logs deleted from instancelogs.log.2025-02-13
  • first log on 14th Feb stored in instancelogs.log.2025-02-13
  • remaining 14th Feb logs in instancelogs.log

And is this your expected result?

• On 14th Feb:
  • only 14 Feb logs in instancelogs.log

Or something else?

[payaljain2003]

@hugovk
What I am expecting is on 14th Feb,

• all 14th Feb logs should be only in instancelogs.log
• instancelogs.log.2025-02-13 should contain all the 13th Feb logs

since I have kept backup count 1, 13th logs should be intact.

There is another instance,(if 1 day is creating confusion) where I have kept backupCount = 5, and the last day file is getting erased.

Please observe in the attached image, ppt.log.2025-02-13 has been modified on 14th, which shouldnot have been. All the logs of 13th Feb are erased from this file and it contains only the first log of 14th Feb. Remainng logs of 14th Feb(excluding the 1st one) are in ppt.log file. Since backup count is 5, this 13th Feb file should have been intact. But all the logs of 13th are gone from this file.

[rockleona]

I would like to assign this issue, seems like a interesting bug here, would you mind if I can take time to reproduce the behavior what @payaljain2003 mentioned?

[hugovk]

@rockleona We don't usually assign issues, but please feel free to investigate further :)

[rockleona]

I've tested on my MacBook, using Python 3.9 and 3.12, with the logger that @payaljain2003 provided in the first post, changing the system time on my MacBook to observe the behavior.

This is how I trigger the logger:

def main():
    instance_logger.debug("This is a debug message")

if __name__ == "__main__":
    import time
    MINUTE= 1

    while True:
        main()
        time.sleep(MINUTE * 60)

The reproduction steps are described as below (tested with both Python 3.9 and 3.12):

1. Set system time at 23:55 on April 8th
2. Run the script in background python test_log.py &
3. Wait until 00:05 on April 9th

As the result, the logs on April 8th are contained at instancelogs.log.2025-04-08, and April 9th logs are all in instancelogs.log.

Maybe @payaljain2003 can share more details about how do you reproduce the bug?