Cannot start shared-memory branch on Windows

[tsculpt]

Hi Pietro.

Master at ba226c seems fine. However I cannot start the shared-memory branch.

With attempting to run a stripped bot (just passing the api key), it may run but does not respond (some kind of race condition?). Other times, setting one simple component or the about / owner bot attribute, I get any of the three errors pasted bellow:

...
\runner\__init__.py", line 85, in _boot_processes
    worker.start()
  File "C:\Python34\Lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Python34\Lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Python34\Lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "C:\Python34\Lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Python34\Lib\multiprocessing\reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python34\Lib\multiprocessing\connection.py", line 940, in reduce_pipe_connection
    dh = reduction.DupHandle(conn.fileno(), access)
  File "C:\Python34\Lib\multiprocessing\connection.py", line 170, in fileno
    self._check_closed()
  File "C:\Python34\Lib\multiprocessing\connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

...
\runner\__init__.py", line 85, in _boot_processes
    worker.start()
  File "C:\Python34\Lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Python34\Lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Python34\Lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "C:\Python34\Lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Python34\Lib\multiprocessing\reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'weakref'>: attribute lookup weakref
on builtins failed

...
\runner\__init__.py", line 85, in _boot_processes
    worker.start()
  File "C:\Python34\Lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Python34\Lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Python34\Lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "C:\Python34\Lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Python34\Lib\multiprocessing\reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function BaseManager._finalize_manager at 0
x03214F18>: attribute lookup _finalize_manager on multiprocessing.managers failed

Hope this helps,
Brad

[tsculpt]

Toying around, I replaced the SharedMemoryManager 's _manager attribute with multiprocessing.Manager() and arrived at a different, possibly helpful error:

20:03.53 -   DEBUG   - Component botogram just loaded into the bot
20:03.55 -   DEBUG   - Component botogram just loaded into the bot
20:03.57 -   INFO    - The botogram runner is booting up.
20:03.57 -   INFO    - Press Ctrl+C to exit.
Traceback (most recent call last):
...
\runner\__init__.py", line 56, in run
    self._shared_memory.start()
  File "C:\.....\botogram\runner\shared.py", line 60, in start
    self._manager.start()
  File "C:\Python34\Lib\multiprocessing\managers.py", line 475, in start
    assert self._state.value == State.INITIAL
AssertionError

Brad

[emilyalbini]

Just a random note, I've figured out how to simulate the Windows behavior on my Linux machine, so it should be easier for me to simulate what's going wrong.

[emilyalbini]

Toying around, I replaced the SharedMemoryManager 's _manager attribute with
multiprocessing.Manager() and arrived at a different, possibly helpful error:

No, that's because Manager() automatically calls the start() method, and obviously you can't start an already-started manager.

[tsculpt]

Ah, I see that now.

I've figured out how to simulate the Windows behavior on my Linux machine,

Very good, no doubt this will be more efficient than my actions. :)

[emilyalbini]

I think I've found the problem (hopefully the only one).

When the runner's shared memory driver is serialized, it requires to the master process a new communication queue. Unfortunately, the master process starts accepting shared memory's commands after all processes are booted up. This doesn't cause any problem on Linux and OSX, but on Windows the driver object is serialized while booting up processes, so a deadlock happens.

Now I'm just tired, tomorrow I'll see how I can fix this problem (I wish this doesn't require a brand new process just for this thing).

Pietro.

[emilyalbini]

Just a quick note, I've managed to get over the the deadlock. Unfortunately, a new error came out just now :-(

[tsculpt]

That's good news. I'm interested in seeing how you got around it.

[emilyalbini]

I've pushed 10833fa, which doesn't bring full compatibility (there is the error I mentioned yesterday), but resolves the ones you reported I was able to reproduce. Can you try directly on Windows which errors it's giving?

[tsculpt]

Here it is:

05:08.19 -   DEBUG   - Component botogram just loaded into the bot
05:08.21 -   INFO    - The botogram runner is booting up.
05:08.21 -   INFO    - Press Ctrl+C to exit.
05:08.22 -   DEBUG   - Component botogram just loaded into the bot
05:08.24 -   DEBUG   - Component botogram just loaded into the bot
05:08.26 -   DEBUG   - Shared process is ready! (pid: 7048)
Traceback (most recent call last):
  File "C:\Python34\Lib\multiprocessing\queues.py", line 242, in _feed
    obj = ForkingPickler.dumps(obj)
  File "C:\Python34\Lib\multiprocessing\reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
  File "C:\Python34\Lib\multiprocessing\queues.py", line 57, in __getstate__
    context.assert_spawning(self)
  File "C:\Python34\Lib\multiprocessing\context.py", line 347, in assert_spawning
    ' through inheritance' % type(obj).__name__
RuntimeError: Queue objects should only be shared between processes through inheritance

[emilyalbini]

Perfect. Later in the afternoon I'll try to migrate from queues to pipes.

[emilyalbini]

Current status: I can't find a way to transfer pipes to different processes via other pipes. Uff.