Data Only Containers/No Run Containers

[kojiromike]

I propose that Compose be able to support a way to create volume containers that never run. Right now, if you want to describe a volume container in your figfile, you must expect it to run. The best you can do is

data:
    image: tianon/true
    volumes:
        - /my/volume
service:
    image: whatever
    volumes_from:
        - data

But it shouldn't be necessary to run a DVC at all. In the commandline Docker client I can do:

c=$(docker create --entrypoint=_ -v /my/volume scratch)
docker run --rm --volumes-from="$c" busybox dd if=/dev/urandom of=/my/volume/foo bs=1M count=1
docker run --rm --volumes-from="$c" busybox du -hs /my/volume/foo
docker rm -v "$c"

It's admittedly a little strange to have to provide an entrypoint to scratch, but it doesn't ever run, so it doesn't make a difference. It would be nice to be able to do this in Compose as well.

data:
    image: scratch
    create_only: true
    volume: …

Related:

• Proposal: Allow fig up (no other arguments) to run a subset of containers #697 asks to be able to run a subset of containers. This is similar in that it asks not to run a container. Perhaps this provides a use case for that, but that issue still implies that the containers can be run. This proposal is to mark containers for creation, but never to run them.
• Fig data-only containers #613 - Despite the similar title this is a feature request with significantly different implications.

[immortal-tofu]

+1 for this feature

[MrMMorris]

+1

[aanand]

I think this would be valuable, especially for data-only containers. To bikeshed: I don't like create_only: true. Perhaps start: false?

[Vrakfall]

I find start: false not very self-explaining. What about data_container: true or data: true. Even data_only: true would fit.

[MrMMorris]

@Vrakfall I would say that data_container, data_only and especially data are even less self explanatory. There may be instances where you would want to create a container and not run it that isn't a data only container.

I +1 start: false as it is very easy to understand what that means. Maybe even running: false or something like stopped: true similar to the Ansible module syntax.

[kojiromike]

I was shooting for something to distinguish the concept of running from automatically running. Maybe that's unnecessary. start: false seems OK.

[dnephin]

I am still really skeptical of any new config setting for this. I think we can solve these issues in a better way. For one, you can already do this by just listing the services you want to operate on. fig up -d <servicea> <serviceb>

I don't quite understand the problem with having a data volume container run echo or true. It seems like that isn't going to really harm anything, and it lets us keep things simple. fwiw, I use data volume containers extensively, and I just use CMD echo.

I think #988 is a better way to address the "administrative container that can run, but shouldn't run by default" issue. You can put those services into separate files, and just include the linked services.

[MrMMorris]

@dnephin that is a fair suggestion, but it will not fit all use cases such as:

• image used for db has an entrypoint set that doesn't allow for it to run echo or true
• I believe that a container running true/echo would be started again when compose is run unless you run with --no-recreate?

Let me know if I'm misunderstanding

[aanand]

@MrMMorris Regarding the first point: If we remove the need for intermediate containers - either by using renaming (#874) or manually copying volume config (#858) - then the only requirement for the image used is that it can run a command of some kind. I don't think that's a huge issue?

Regarding the second: does it matter that the container is started again when Compose is run?

[MrMMorris]

@aanand just so I understand: does #874 and #858 mean that you won't have to use the same container for the data as you do for the db as pointed out by @cpuguy83? http://www.tech-d.net/2014/11/18/data-only-container-madness/

as for the second point, I actually didn't consider whether or not the container starting again would be an issue or not. More of a note 👍

[aanand]

@dnephin I'm surprised that containers with CMD echo work, given that docker-compose up is currently supposed to error out if a container exits.

Does it only work with -d? If it works without -d, is it only because of the race condition fixed by #738? If so, we should merge that and then address #741.

[dnephin]

Yes, true. It only works with -d, which is where #741 came from.

[kojiromike]

@dnephin I don't think your suggestion to use fig up -d <servicea> <serviceb> applies. If I have the following figfile:

svc1:
    image: scratch
    volumes:
        - /foo
svc2:
    image: tianon/true
    volumes_from:
        - svc1

Then even if I explicitly tell fig only to run svc2 with fig up -d svc2, it will still try to run svc1 because it is a dep of svc2. So how does specifying svc2 help in this case?

[dnephin]

I'm pretty sure that up only considers links as dependencies. volumes_from is only considered for building of containers, so doing docker-compose up -d svc2 should only start the one container (and any linked containers).