links-like behavior for containers using net=host.

[razic]

hey. i'm running some containers with net: host and i need some containers to start running before others but obviously cant use links with net host to specify that sort of dependency. any ideas?

[thaJeztah]

You'll have to start them manually in that case. (Having something in the yaml to define dependencies would indeed be nice).

You can create/start individual services manually, using docker-compose up -d [servicename(s)]. The same applies to other commands (e.g. docker-compose stop foo bar baz)

[dnephin]

#686 would make this possible

[bonitao]

You can workaround with volumes_from, as explained here: #933

[razic]

@bonitao think you referenced the wrong issue. this is issue #933 which does not contain a work around.

[bonitao]

Hi @razic,

Sorry for that. I can't find the link I intended to post anymore. But all it says is that instead of links you can use volumes_from to make containers depend on each other.

# this ensures docker-compose starts container_a before container_b
container_a:
  ...
container_b:
  links:
     a:a

# this does not run because links are not allowed with net: host
container_a:
  net: host
  ...
container_b:
  links:
     a:a

# this ensures docker-compose starts container_a before container_b
container_a:
  net: host
  ...
container_b:
  volumes_from:
     - a

I hope this helps.

[aanand]

Why do some containers need to start before others? Best practice is to make the dependent container poll until the container it relies on is ready, so that the starting order is irrelevant.

[razic]

@aanand unfortunately i am using open source software who probably care nothing about implementing such features. besides, even if they did, it would have to be some complex implementation that is able to poll anything you throw at it, allowing you to specify whether it be TCP, UDP, the endpoint, etc... basically, while what you say sounds great, it's not the typical use case, since majority of people will be using open source software as well.

[aanand]

@razic Regardless of what software you're using, you can make your containers independent of starting order by setting a wrapper script as the container command, which polls until it's ready. Something like:

#!/bin/bash

until nc -w 1 -z postgres 5432 2>/dev/null; do
    echo "$(date) - waiting for postgres..."
    sleep 1
done

exec actual-process-to-run

What you say about the complexity of implementation is another reason it shouldn't be Compose's job: the job of determining whether a service is "ready" is complex, and should be left to application code.

[razic]

What about when the port is dynamic?

On Tuesday, April 28, 2015, Aanand Prasad notifications@github.com wrote:

@razic https://github.com/razic Regardless of what software you're
using, you can make your containers independent of starting order by
setting a wrapper script as the container command, which polls until it's
ready. Something like:

#!/bin/bash
until nc -w 1 -z postgres 5432 2>/dev/null; do
echo "$(date) - waiting for postgres..."
sleep 1done
exec actual-process-to-run

What you say about the complexity of implementation is another reason it
shouldn't be Compose's job: the job of determining whether a service is
"ready" is complex, and should be left to application code.

—
Reply to this email directly or view it on GitHub
#933 (comment).

[aanand]

If the port is dynamic, then however you normally determine it, do that and then call nc with the appropriate arguments.

[razic]

If the container was spun up using docker/compose and the port was
configured to be dynamic; then the only way I know of to determine the port
number is to mount the docker socket and binary and run something like
'docker inspect' on the container with the unknown port. Is there another
way to determine ports? You could probably scan for open ports but how
would you match it up with a service? Also, seems like you would introduce
a lot of duplicate code wrapper scripts in each of your images. Unless you
suggest everyone use the same base image too. Also, mounting the docker
docker and binary can prove to be a security risk as now your container has
full access to your docker environment.

The whole point of linking docker containers together is so that you can
specify ordered dependencies and not have to worry about manually
determining the port via some wrapper script. Docker even sets environment
variables and modifies the hosts file for each container with links to make
it super easy for developers to obtain the ports of the linked
containers without a wrapper script.

What we are lacking is a way to get the same benefits of using linked
containers.

On Wednesday, April 29, 2015, Aanand Prasad notifications@github.com
wrote:

If the port is dynamic, then however you normally determine it, do that
and then call nc with the appropriate arguments.

—
Reply to this email directly or view it on GitHub
#933 (comment).

[aanand]

If you're looking for a replacement for everything that links do, why not use links? I'm pretty sure I don't understand your use case.

[razic]

I want to use --links but I can't. the image I'm using requires --net host.

Docker doesn't provide support for --links with --net host.

I'm not looking for a replacement, I want the real thing.

I guess it's more of a docker issue itself.

On Wednesday, April 29, 2015, Aanand Prasad notifications@github.com
wrote:

If you're looking for a replacement for everything that links do, why
not use links? I'm pretty sure I don't understand your use case.

—
Reply to this email directly or view it on GitHub
#933 (comment).

[thaJeztah]

If I have understood correctly, Docker networking will get a massive overhaul (no time estimate yet). The foundations for the underlying framework are currently being laid in this repo: https://github.com/docker/libnetwork. You might want to follow progress there to see what's possibly coming. Afaik, the initial implementation will not bring changes from a user-perspective, but more features might be added in future.