Proposal: Enhance docker info output with running and exited/stopped containers

[abronan]

Currently the output of docker info looks like:

$ docker info
Containers: 1362
Images: 145
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.4-1-ARCH
Operating System: Arch Linux
CPUs: 8
Total Memory: 15.6 GiB
Name: archbox
ID: OUQD:OBQE:7KQO:2FQ5:YJIC:ILXS:O7SE:PWEK:3ZWT:PG7I:RUPA:2EIW

This does not really help at a first glance to assess how many containers are running on a machine. Indeed on those 1362 containers, I only have 10 running containers, the rest are stopped. I might have a few paused containers too.

This would be useful in docker info to show more informations on the current state of containers:

$ docker info
Containers: 1362
 Running: 8
 Paused: 2
 Stopped: 1352
Images: 145
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.4-1-ARCH
Operating System: Arch Linux
CPUs: 8
Total Memory: 15.6 GiB
Name: archbox
ID: OUQD:OBQE:7KQO:2FQ5:YJIC:ILXS:O7SE:PWEK:3ZWT:PG7I:RUPA:2EIW

This also means adding this to the remote API on /info. This way we can also use it in Swarm as a quick way to assess the load on machines (see related issue on the Swarm side docker-archive/classicswarm#1140)

I agree that this information is provided somehow by docker ps but this could be an added convenience to have this on info 😃

[thaJeztah]

I wonder if we're going to need a --verbose flag (or similar) for docker info on the command line; is this information relevant to the end user on a daily base?

I can see the use case here, just wondering if we're not overloading this endpoint with too much information

[abronan]

Adding a flag for this output to be optional seems like a good idea as long as this information is also available with the remote API through a query parameter. It is just to offer an accurate overview of the machine/cluster to the end user. Especially in Swarm where we manage tens/hundreds of machines and where docker info is used far more often. It also makes sense for a single Docker daemon running thousands of short lived batch jobs without cleaning stopped containers.

[thaJeztah]

Yes, I understand. Bit of brainstorming here; is Swarm interested in a "snapshot" of this information, or a stream? i.e. could a global "metrics" endpoint be useful, not only including number of containers, but also system load/memory usage?

[duglin]

If we're only talking about a small number of properties I'm not sure its worth introducing a --verbose flag.

[duglin]

But +1 to adding the new fields

[thaJeztah]

@duglin true, we can handle a few more lines, but since there's also the PR to expand the Operating System node to add more information, at some point this may become an unwieldy list. Doesn't have to be introduced with this proposal, but we may want to start thinking about this.

[duglin]

Yup, I'm just not a fan of us deciding for people what's important. Especially when its due to a random time-based thing - like "new stuff is hidden" and "existing stuff is important/shown", when its possible the existing stuff could be less important than the new stuff. E.g. The "ID" is pretty useless (I think) to most people, but (as a proof point) that's just my opinion.

[thaJeztah]

Yes, splitting up in "default" and "verbose" could be a challenge. Hence I was wondering if we need a "metrics" endpoint #15639 (comment) to have the "info" endpoint return "static" information (version, daemon config, platform), and the metrics endpoint return "live" / "runtime" information.

[abronan]

@thaJeztah Just wondering if this is worth splitting this in two separate endpoints considering the amount of information that a metrics endpoint would provide.

I like the idea of a metrics endpoint though, that would return the current live status of a docker daemon with runtime informations: running/stopped containers, used CPU and MEM, etc. This would allow external tools to rely on this endpoint rather than having to, for example, calculate the used CPU and MEM (by iterating through running containers and adding reserved CPU shares like we do in Swarm right now).

[abronan]

ping @aluzzardi @vieux @chanwit @jimmyxian

[dhiltgen]

Instead of arbitrarily picking what we think users want for a verbose flag, could we follow the pattern of inspect with --format=... to allow CLI consumers to select which subset they want to see? In fact, I'd love to see this or some similar pattern make its way into the API so apps can also subset response payloads to only the data they care about.