Async appender

[tisonkun]

Migrate the current NonBlocking impl to a composible async appender.

Ref -

• https://github.com/SpriteOvO/spdlog-rs/blob/main/spdlog/src/sink/async_sink/async_pool_sink.rs

But I don't think we need a thread pool or something complicated here - just migrate the current nonblocking impl and file/syslog appender impl.

[Phoenix500526]

I'm interesting in that. Can you assign it to me?

[tisonkun]

Just go ahead and submit a patch for review :D

You don't need get assigned to start. And you can follow rust-lang/log#696 for more background.

I suppose this can introduce more design and impls over an owned record.

[SpriteOvO]

Nice to see spdlog-rs's idea inspiring elsewhere.

Regarding spdlog-rs's ThreadPool, every AsyncPoolSink all use the (only) one internal thread to perform tasks (write and flush) by default, but when the user creates a bit more async sinks, one thread may be a bit overstretched. So we provided an abstraction -- ThreadPool, that allows the user to set up a separate thread for an async sink instead of sharing the internal thread.

Currently, one ThreadPool runs only one thread; it neither gives the user the ability to make a Pool run multiple threads, nor does it do so internally. Because for now, I haven't found any indication that there would be any performance benefit to running multiple threads for a single async sink. But for ease of understanding and extensibility, I've named it as such and reserved the possibility of supporting multiple threads in the future.

I hope this helps.

[tisonkun]

Nice to see spdlog-rs's idea inspiring elsewhere.

@SpriteOvO "Privately" speaking, I like spdlog-rs macro and record design (over log's, or yet, lol), while logforth's Append/Layout/Diagnostic/Filter fits my scenarios better :D

Maybe an ErrorHandle would be introduced later also, but so far, since log macros never return Err, I don't think it's quite necessary rather than leave an eprinln!.

Anyway, log is the official lib and I'd try to improve it directly first. You may read this blog (in Chinese) about the story of logforth if you're interested in.

[tisonkun]

Currently, one ThreadPool runs only one thread; it neither gives the user the ability to make a Pool run multiple threads, nor does it do so internally. Because for now, I haven't found any indication that there would be any performance benefit to running multiple threads for a single async sink. But for ease of understanding and extensibility, I've named it as such and reserved the possibility of supporting multiple threads in the future.

Good to know. I think typically one thread nonblocking impl is enough. When you let users configure the thread numbers, they don't actually know how to do it properly, or even there is no proper way. Options in open-source are sometimes evil, and cheat users into playing with them with no practical reasons >~<

[SpriteOvO]

I have read that blog two weeks ago, my friend found it then told me about it 😆. I honestly admire your motivation to improve log crate.

And I'll also take a look at logforth's implementation when I have time, some things I still haven't figured out how to appropriately design, such as filters, and the log-crate-proxy for spdlog-rs still feels worth refining.

[tisonkun]

@Phoenix500526 FYI I'm looking into this issue these days. But it should not block you from spending your time, since we can combine both thoughts together and the credits can be shared with Co-authored-by attr.

[Phoenix500526]

@Phoenix500526 FYI I'm looking into this issue these days. But it should not block you from spending your time, since we can combine both thoughts together and the credits can be shared with Co-authored-by attr.

Ok, thanks! I’m tied up with some personal things for the next couple of days. I’ll probably check it on Monday or Tuesday.

[tisonkun]

I'm going to remove non-blocking and let syslog/rolling_file appender with a global mutex first.

Then I'll try to add an Async appender like spdlog-rs' one. However, I noticed:

• [Bug Report] AsyncPoolSink may lose the last log records and panic on program exit SpriteOvO/spdlog-rs#64
• SpriteOvO/spdlog-rs@cac28cc

Currently, nonblokcing uses a WorkerGuard and users should hold it explicitly:

let (rolling_writer, _guard) = RollingFileBuilder::new("logs", "my_app")
        .layout(JsonLayout::default())
        .rotation(Rotation::Daily)
        .build()
        .unwrap();

Seems spdlog-rs uses a global logger setup while the guard is held internally. This may cause some issues when the global static variable getting dropped on program tearing down.

@SpriteOvO I'd appreciate it if you can give some context here how things around Drop works upon program tearing down and where the issues may occur.