A NtCreateWaitCompletionPacket and NtAssociateWaitCompletionPacket realisation to poll CreateWaitableTimer and CreateEvent handles

[eesekaj]

Hello,

Since this request doesn't break the API, I decided not to open an issue. Perhaps the implementation of this feature is not yet completely sensible, so I do not insist on its adoption.

I am working on project which uses Mio crate (from private repo but anyway). I was tidying up the "zoo" of IO subsystems of different OSes (with timers and stuff) and ran into the problem with this crate that it's impossible to poll the timer/events if handles are not files. In order to make it fully compatible with Windows, I developed some experimental code which adds a feature to 'poll' the timers and events.

A SourceEventHndl in source_hndl.rs is acting as a wrapper for any type which holds a 'handle' to object which was created by: CreateEvent or CreateWaitableTimer.

Windows-sys crate is missing some calls to NTDLL, so ffi.rs covers this. Tests pass successfully on Windows 10, but I don't have an opportunity to check if it runs on Windows 11 because, usually I am not writing a code for Windows. (And I don't have access to Win11 machine)

The code can be enabled using feature os-extended.

I tried my best not to touch the base code and integrate it as extension. For more details please check attached comments and test file win_event.rs and example windows_event.rs

Problems:

1. There is at least one problem that I haven't solved yet: WaitCompletionPacket identification, which is carried out not by the descriptor, but by the token.
2. It is not possible to tell if handle is from Event* API. So if a wrong handle is passed then program probably would crash.
3. Every time an event triggers it should be re-associated. It is ok to de-associate CompletionPacket many times, but in reverse it returns the error which is general and masking it create problems.

Example:

#[derive(Debug)]
pub struct PrimitiveTimer
{
    hndl_timer: OwnedHandle
}

impl AsHandle for PrimitiveTimer
{
    fn as_handle(&self) -> std::os::windows::prelude::BorrowedHandle<'_> 
    {
        return self.hndl_timer.as_handle();
    }
}

impl PrimitiveTimer
{
    fn new(name: &str) -> PrimitiveTimer
    {
        let mut label_cstr: Vec<u16> = name.encode_utf16().collect();
        label_cstr.push(0);

        let hndl_timer = 
            unsafe
            { 
                HANDLE(
                    CreateWaitableTimerExW(
                        null(),  
                        label_cstr.as_ptr(),
                        0,
                        EVENT_ALL_ACCESS
                    )
                )
                .try_into_owned()
                .unwrap()
            };

        return Self{ hndl_timer: hndl_timer};
    }

    fn arm_relative(&self, timeout: i64) 
    {
        let time: i64 = timeout / 100;
        unsafe
        {
            SetWaitableTimer(
                self.hndl_timer.as_raw_handle(), 
                &time as *const i64,
                0,
                None,
                null(),
                false.into()
            )
        };
    }
}

#[test]
fn test_event_win()
{
   // timer 1
    let mut se_hndl_timer = 
        SourceEventHndl::new(PrimitiveTimer::new("timer_1")).unwrap();

  //  ....

    // poll
    let mut poll = Poll::new().unwrap();

    // registering
    poll.registry().register(&mut se_hndl_timer, Token(1), Interest::READABLE).unwrap();
}

[eesekaj]

I decided to adapt some changes from closed source MIO crate (i.e from the organisation's private repository) to your OSS crate, but I had to change some things so as not to conflict with the interested parties and avoid making changes to the public API.

I'd like to know what you think about these changes. And how likely is it that they will be accepted, or are those changes unsuitable?

All changes are related to Windows only!

Changes:

1. Internal Token generator in /tokens.rs. Produces "mapped" tokens for every IO source i.e NamedPipe, Event, Waker, Afd.
2. ./source_hndl.rs was re-implemented. The SourceHndl is acting as SourceFd. And SourceEventHndl is a safer realization of the former. SourceEventHndl is no longer using a locked list and a callback is now used. SourceFd stores the data on the shared list.
3. ./seletor.rs all code from previous PR was removed and the feed_events() function was re-implemented to determine the event source by mapped token.

Now, a user provided token and internal token are separated. The former is used for identification of event outside of the crate. The latter is used for internal use only. The selector's event feeder receives the internal token and checks to which IO/event (i.e NamedPipe, Event, Waker etc. a facility) is it mapped. Then it routes a received data for parsing to this facility. A facility is holding the user provided token in _inner_ structures and emits the Event (to user) replacing an internal token with user token. In some cases i.e in NamedPipe, when it is required to reschedule the event, the internal token is stored is indicated in the Event in order for selector's event feeder to determine where to send this rescheduled event.

Token Flow diagram

+--------------+                                                             
|              |                                                             
|  AFD token   ----------+                                                   
|              |         |                                                   
+--------------+         |                                                   
                         | +--------------------+                            
+--------------+         | |     OVRLAPPED_ENTRY|                            
|              |         | |                    |                            
|  Pipe token  ---------->-> lpCompletionKey    |                            
|              |    +----->|                    |                            
+--------------+    |  +--->                    |                            
                    |  |   |                    |                            
+--------------+    |  |   +---------|----------+                            
|              |    |  |             |                                       
|  Event token -----+  |             |                                       
|              |       |             |                                       
+--------------+       |    +--------v---------+                             
                       |    |                  |                             
+--------------+       |    |  IoCompletionPort<----------------------------+
|              |       |    |                  |                            |
|  Waker token --------+    +--------|---------+                            |
|              |                     |                                      |
+--------------+                     |                                      |
                            +--------v---------+                            |
                            |                  |                            |
          +------------------  event_feed()    |              Rescheduled   |
          |                 |                  |              event         |
          |                 +----------|-------+                            |
          |Internal token              |                                    |
          |                            |                                    |
          |                            |                                    |
   +------v----------+               +-v----------------+                   |
   |    WAKER        |               |    Pipe          |       ...         |
   | from_overlapped |               |  event_*()       |                   |
   |                 |               |                  |                   |
   +------|----------+               +|----------------|+                   |
          | ▯                         |                |                    |
          |           User token      |                | Internal token     |
          |                           |                |                    |
   +------v----------+          +-----v---------+     +v---------------+    |
   | Event{}         |          | Event{}       |     |▯Event{}        |    |
   |                 |          |               |     |                |----+
   |     OUT         |          |    OUT        |     |  RESCHEDULE    |     
   +-----------------+          +---------------+     +----------------+     
                                                                             

[Darksonn]

I'm confused. Is this a new feature, or is it fixing a bug? If so, what is the bug?

[eesekaj]

This is a new feature which allows to poll handles created by CreateWaitableTimerExW and CreateEventA which are not a file type.

For example CreateWaitableTimerExW can be used with WaitMultipleObject call or IOCP via NtCreateWaitCompletionPacket and NtAssociateWaitCompletionPacket to IOCP instance.

Another thing is a token for event identification. In order to introduce new feature, I decided to amend the token generation system and substitute it with a "mapped" internal token for internal events and resolve it to user defined token only when the event is emitted to the "poll" caller. (this is temporary solution) Because, I don't want to represent struct as "C" and count the offset using magic numbers instead (at least) using "offset_of()". There is a better solution for that, but cannot copy it from closed source mio fork. I need to re-implement it somehow. I don't have time for that.

I need that feature to avoid calling WaitMultipleObject on separate thread for timers and events. This allows to poll it in the same instance.

[eesekaj]

In the recent commit I have fixed some issues with tests and errors. But I didn't format the code and will do it later if it will be approved.

Related issue 5390