Automatically clean up old log files

[dgw]

Requested Feature

Sopel should clean up log files older than some default number of days, with a configuration option to override it.

Problems Solved

Our main Sopel instance started spitting out errors tonight when its jail ran out of disk quota. This feature would prevent similar issues from happening to other users who don't set up an external mechanism to prune logs.

Alternatives

As alluded to above, a cron job or other recurring task could take care of this—at the cost of extra effort from the bot owner.

Notes

The TimedRotatingFileHandler's backupCount option would be a simple way to implement this.

[SnoopJ]

Not sure if the option has been suggested previously but the `logging` module has handlers that can rotate based on time and/or size, they might be suitable for an in-band solution to the problem (i.e. not needing to get an external check involved)

[dgw]

Sopel already uses TimedRotatingFileHandler to get a new log file each day. It's just not configured to delete old logs, ever, so they build up indefinitely. Our main instance has had raw and debug logs turned on for a long time, which made it a problem much faster than it should ever impact normal use. 😁

[Exirel]

That's why logrotate exists! We can document its usage for Sopel, the same way we document it to create a service with systemd.

I don't like when a software tries to deal itself with logs, it tends to be messy and never really satisfactory. And on the dev side, it means we would have to deal with the maintenance burden of that.

[dgw]

it tends to be messy and never really satisfactory

What's "messy and never really satisfactory" is defaulting to fill up the user's entire disk if left alone for long enough.

If Sopel shouldn't delete its own old logs because you don't like when software tries to deal with that itself, why are you OK with Sopel creating its own log files for each day?

Requiring the user to set up an external tool is not (IMO) in the spirit of being "simple". Recovering from a full disk also can be not "simple", depending on how Sopel was deployed. Sopel must clean up its own mess, or change to not create the mess in the first place.

[Exirel]

I strongly disagree with that. First, because rm -rf *.log.* isn't that hard (it's annoying, I've to admit), and second I don't really like that Sopel write logs to files, and I consider it a tradeoff, because for most people this is, indeed, too complex to setup themselves. Beside, Python already deals with log file with the TimedRotatingFileHandler.

That being said, logrotate is pretty standard and quite easy to setup. The configuration file should look like this (I need to check what I used for the filename pattern):

~/.sopel/*.log.* {
  rotate -1
  weekly
  maxage 15
}

This removes log files older than 15 days every week. Can be customized easily to being monthly and older than 30 or 60 days.

[Exirel]

I can, begrudgingly, accept a config option to use TimedRotatingFileHandler's backupCount option.

[dgw]

I can, begrudgingly, accept a config option to use TimedRotatingFileHandler's backupCount option.

So my original suggestion from May, then? 😁

The TimedRotatingFileHandler's backupCount option would be a simple way to implement this.

Not a terribly involved patch. Hardest part will be deciding what to call the new option (log_max_age?)