Support dynamic reloading of plugins

[renepickhardt]

Currently every time I change the code of the plugin I need to restart the lightning node which takes some time. It would be cool if there was a way to reload plugins in a running lightning node

[conscott]

The correct way to do this in lightningd is to get inotify events from the filesystem for registered plugins, and I suppose it's an option that should only be compiled in debug mode, HOWEVER, there is a poor man's version of this that will work just fine (for simple testing in regtest)

Here is my hacky version:
https://github.com/conscott/lightning/blob/plugin_restarter/reload_plugins.py

I just made a wrapper for lightningd that will kill it and restart it when any of your registered plugins are modified. You can use its like:

./reload_plugins.py lightningd/lightningd --plugin-dir ~/c-lightning-plugins/

For now I hardcoded that plugindir, but with just a bit more work you can make it pull from command line args or from the lightning config file

Edit: Updated the script so it can detect your plugins from config file and command line.

Seems to work okay. If there is more interest in this, I can pull the script out to separate repo (or in c-lightning if there is a good place to put it) and polish it up a bit.

[conscott]

I've also wanted this for myself, so this issue has given me the motivation :)

[cdecker]

There are three ways I can see that'd allow a plugin to dynamically reload:

• lightningd detects a change, kills the old instance and starts a new one with the new code
• the plugin itself manages the reload, detecting its own changes and reloading when something changes (could be part of the framework)
• we create a --plugin-runner argument that is prepended to the exec call that is run by lightningd, this could then detect changes and handle the killing and restarting of the plugin on its own.

Since this is mostly useful for plugin developers I think I'd vote against the first, in favor of the runner or the self-reloading plugins.

Notice however that there are a few things that cannot be changed when reloading the plugin:

• Methods, notification subscriptions and hooks that are registered may not be added or removed upon reload (this could be changed and made more dynamic but it'd require support for manifest-notifications in lightningd)
• Command line options cannot be changed without restarting lightningd, this is in the nature of the command line options themselves, and I don't see a way around this.

Lastly I think I see 2 possible reload triggers:

• the runner/plugin/lightningd detects changes on the filesystem
• the plugin or an external entity notifies lightningd of changes to the plugin and that it should be reloaded

[ZmnSCPxj]

Closed in #2771