App Manager Integration

[stonier]

Proposal for how workflow between capability server and rapp manager might happen.

Rapp Manager Initialisation

• Query the capability server

Query the capability server to get information about the capabilities it provides. If we're assuming that capabilities are static as part of a bootstrap constant, then the rapp manager probably only needs to do this at startup and cache/save the information.

• Prune list of both local and installable apps

Check against capabilities and prune apps which can't be run because they lack capabilities. Again, if we're assuming static capabilities, this can be done at startup to save having to do it every time list_apps is called.

Start Rapp

• Check rapp is in the pruned list when /start_app is called, [1] [2]
• Rapp manager reads the rapp definition for preferred, required, optional capabilities [1] [2]

Am envisioning three kinds of capability specifications as listed above. The preferred should always come listed with a fallback. I think this would get predominately used with semantic capabilities. One or two simple example rapps might be useful in the rocon_apps package (it's probably better thought of as rocon_tutorial_apps).

• Rapp manager makes requests to start capabilities on the capability server
• Rapp manager starts the app with capability server remapping [1]

Some information should get to the rapp somehow. Remapping the capability server for the rapp so that it can make it's own queries and decisions would be a good minimum point. I'm not sure we can intelligently do any remapping for the rapp since the rapp doesn't provide its own private interface specification, so let it work things out for itself.

• Stop app process should work in reverse in all the expected ways

Variations

The rapp manager only does single rapp management right now. With capabilities, capability counts, locks or sharing priveleges, we could feasibly do multiple apps which would be great.

Demonstrables

Integrating stuff next week, should focus on what we might want to show. Options:

• Portability: same app on two robots utilising the same capability via different capability providers.
• Tuning your app: using semantics to choose capability more intelligently.
• Simplified Bringup/Teardown : graphical tool which helps bring up capabilities in a modular way and set application meaningful parameters (the rest hide of course in the capability provider).

The first one I could integrate here on two robots - navigation would be the best example. The second one would probably take more work to make it meaningful. The last would be quite useful for robotics researchers and developers in general I think. It would also mean we could delay integration into the app manager until after roscon.

Any other ideas? I'll have a couple of days next week to work on integration for the demo, which is right after roscon.

[bit-pirate]

I agree with most thoughts and suggestions. Here are my few remarks:

_@ Remapping_:
We have already agreed on the rapp being remapped according to the capability's interface [1]. Hence, we need to find a way to retrieve the required information for the app manager.

Proposal:
Requiring a capability means for the rapp to provide the topic names for its side of the interface as well. The workflow could then look like this:
App manager stores capability information retrieved from the capability server, e.g. capability: mobile_base; cmd_vel_topic: mobile_base/commands/velocity; ... The rapp requiring the mobile_base capability (needs to) specifies its own interface topics, e.g. cmd_vel_topic: cmd_vel. With the combined info the app manager is then able to do the remapping cmd_vel -> mobile_base/commands/velocity.

_@ Starting a Rapp_:
How do we determine the status of a capability, i.e. when do we consider it as started/stopped?

@wjwwood what's your opinion on the proposed workflow?

[wjwwood]

@ The Proposal:

We can certainly do this, but I think this is a bit fancier than it has to be, I think it would be easier to define these Capabilities in a standard set of packages, which then the capability providers and rapps both adhere to. Then there is no need for automated remappings. While this is a feature we may well need/want in the future, I don't see it as required for the first round of integration.

That being said, if you see it as a flexibility we will definitely need, we can work on getting that integrated. You can access the information which the capability server has using the Python API currently. I can work on a set of ROS services which expose this information too, if we think that the app manager and capability server will run on different machines. See the API here:

http://ros.org/doc/groovy/api/capabilities/html/capabilities.html#module-capabilities.discovery

@ Starting a Rapp:

Once calling the /start_capability service call, you can watch the /events topic for capability start ups and terminations. Unfortunately we can't know how long it will take between launching roslaunch as a subprocess and when the capability is "ready".

[bit-pirate]

@ proposal:
I would prefer to do automated remappings for the sole reason of giving the robot and app developers as much freedom as possible, e.g. structure topic names as they wish. I do agree, that we don't need to implement this right away.

Regarding your thought about the app manager and capability provider running in different places, I would say that in most case this won't happen. However, on more powerful machines/robots, such as the PR2 with multiple PCs, this might be the case - or maybe once ROS on Arm gets mature and we will see robots with highly distributed processing units (would love to see this soon :-) ). Making the capability-related information available via services will also give C++ nodes access to it, e.g. someone wants to use only the capability server and not the app manager. Then again, we can implement this, whenever we have time to.

@ capability statuses:
This is actually a wider issue (for us). AFAIK there is currently no way to determine the status of a node in ROS. This gives us headaches on all levels - is the node/capability/app/service/solution ready?

While I like your solution for its simplicity, I also see that this imposes extra design restrictions on the capability providers/apps/services, i.e. everyone has to implement a status publisher with predefined statuses. Do you know of any other more generic/automated ways to do this? Do you know of any former discussions addressing this topic?

For the first demo(s) I would say that your implementation is enough. We might even want to extend this to the app level as well (@stonier)?!

[bit-pirate]

@ stopping apps

I guess stopping an app should also stop unused capabilities, what would require some kind of counter. Any suggestions on how to implement this? Do we want this feature for the upcoming demo?

[mikeferguson]

@bit-pirate on stopping apps

You actually probably want to do something like reference counting on the capabilities and then shut down / garbage collect unused capabilities after a short time period (maybe use a heuristic like 2x the startup time attributed to that capability).

I would be concerned that if you have an app that starts/stop frequently, with a desired capability that is quite heavy to start/stop, you might thrash the heck out of the system (imagine for instance, a capability that brings up all of MoveIt!, and an app that does a simple pick & place.. and you keep running the app to move lots of things, you'll spend most of your CPU time starting moveit over and over again loading the meshes for collision avoidance).

[bit-pirate]

@mikeferguson That's an important point. Two thoughts: First, I'm not a big fan of this heuristic, but I can't come up with a nicer solution, since there is currently no clean and simple way to decide on the status of a node -> capability -> app.

Second, wouldn't that app be "inefficiently" designed, if it does what you describe? I.e. if I want to be able to move multiple things around, shouldn't that app be running continuously? I'm just concerned, that this heuristic (x times startup time) might add unnecessary shutdown delays.

[tfoote]

The problem is that the app may be "efficiently" designed to do that, but
if the user picks something up, switches apps to something else. like a
picture taking app, then switches back to pick and place something else you
will get this condition.

I agree that the heuristic is awkward, but finding a way such that the
startup/shutdown costs can be avoided could be valuable. For example
android will usually keep processes in the background until it runs out of
available memory and then it will start knocking off the older ones. I
don't think that's the right behavior for this case but as another example.

On Thu, Oct 10, 2013 at 2:51 AM, Marcus Liebhardt
notifications@github.comwrote:

@mikeferguson https://github.com/mikeferguson That's an important
point. Two thoughts: First, I'm not a big fan of this heuristic, but I
can't come up with a nicer solution, since there is currently no clean and
simple way to decide on the status of a node -> capability -> app.

Second, wouldn't that app be "inefficiently" designed, if it does what you
describe? I.e. if I want to be able to move multiple things around,
shouldn't that app be running continuously? I'm just concerned, that this
heuristic (x times startup time) might add unnecessary shutdown delays.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-26041019
.

[wjwwood]

I'm not sure about the shutdown heuristic either, because there may be scenarios where the user actually wants the capability to shutdown (to reset the connected devices or something) and that won't be deterministic anymore.

I'll give some thought to the shutdown costs, but I can simply add reference counting to capabilities in order for them to not get shutdown prematurely/get shutdown when no one is using them.

[wjwwood]

#19

[wjwwood]

As I was writing #19 I thought about where this reference counting should live. Certainly it can be implemented into ~/start_capability and ~/stop_capability, but that sort of breaks the semantics of ~/stop_capability. It would be odd to me if ~/stop_capability returns true but the capability hasn't actually shutdown. We may need a ~/use_capability and ~/free_capability (or something similar) which has the reference counting semantic.

The other thought is that this could be managed in the app server which wouldn't call ~/stop_capability until its own reference count was zero, but I'm not sure about that either.

[bit-pirate]

I like the idea of a ~/use_capability and ~/free_capability. Maybe add those additionally, so that one could force a shutdown with ~/stop_capability, while ~/free_capability will just decrease the reference counter.

I'm not in favour of adding the reference counting to the app manager, since I'd like to keep the dependencies between them as little as possible, i.e. one can use the capability server or app manager w/o the need of using the other.

[bit-pirate]

Regarding alternatives for the heuristic, what about simply adding an option to keep the capabilities running even when the app requiring it is shut down? (could be implemented on the app manager side using ~/stop_capability and ~/free_capability)

[wjwwood]

I am going to close this issue, can we open new tickets with more focused scopes? I think we have tickets for most of these things.

If you guys disagree I can reopen it.