Multiple file parameter does not use the correct working directory

[LoicMahieu]

Related: #2051

I would like to use multiple -f parameter for running multiple docker-compose in one.
Example:
Project A needs Project B. There are not in the same directory.

• Project A docker-compose.yml:

app:
  image: some/image
  command: /project-A-start.sh
  working_dir: /app
  volumes:
    - .:/app
  ports:
    - "3001:3000"
  links:
    - api

• Project B docker-compose.yml:

api:
  image: some/other/image
  command: /project-B-start.sh
  working_dir: /app
  volumes:
    - .:/app
  ports:
    - "3000:3000"

So I use the command in project A:

docker-compose -f docker-compose.yml -f ../project-b/docker-compose.yml up

But api container from project B does not start because command /project-B-start.sh can not be found. It appears the volume/cwd used for ../project-b/docker-compose.yml is not ../project-b but the initial directory, the project A. (Hope it's clear)
Furthermore, it turns out that it is the first -f parameter that define the directory. If I inverse the two parameter, the directory used is now project-b and now it is /project-A-start.sh that can not be found.

Is this behavior by design ? In my case the perfect behavior would be to use a separate "cwd" for each docker-compose.

Thanks for your help.

---

Docker: 1.8.0
Docker Compose: 1.5.0rc2

[dnephin]

Thanks for the report!

I'm actually working on better documentation for this feature as we speak, so this feedback is helpful.

The order of the files changing the working directory is by design. The first file is always the base and the following files are applied over it in the order they are included. This isn't important for your example case, but it is when a service is defined in both files. We need a well defined order so that we know which file overrides the other.

I think the current working directory being relative to the first file is also necessary. There's no requirement that the overriding files are completely valid compose files. They may contain services that are missing a build or image (effectively abstract), or they may contain links to services that aren't defined locally.

Since each file is just a fragment, it's extremely difficult to keep track of which fragments are relative to which path.

I'll make sure to include both of these details in the documentation.

---

To support your use case, you can remove the volumes when including it from a separate project, and use the code in the image.

To support that workflow, you could split out the Compose file into two:

Project B docker-compose.yml:

api:
  image: some/other/image
  command: /project-B-start.sh
  working_dir: /app

Project B docker-compose.override.yml:

api:
  volumes:
    - .:/app
  ports:
    - "3000:3000"

(ports could go into either file really, but for this example I moved it into the override)

Inside the project b directory, if you run any command like docker-compose up it will automatically use both of these files (without having to specify -f).

Now in Project A you can use it with:

docker-compose -f docker-compose.yml -f ../project-b/docker-compose.yml up

Since the override file is not being included, it should work.

Another option might be to have the project A include an override for the volumes, to set it to the correct relative path, but that's assumes that the relative paths will always be the same, which would break if a developer uses a different directory structure. You could put those relative paths into yet another file (maybe a user.override.yml) in project a, so that users can add those overrides for their path structure.

[LoicMahieu]

Thanks @dnephin for your useful and complete response. Your explanations helped me to better understand docker-compose.
I am thinking to refactor my projects into one for simplifying things.

Once again thank you. Feel free to close the issue or keep it open for doc.

[dnephin]

Awesome! I've put up a review for the new docs, the relevant section is here: https://github.com/docker/compose/pull/2290/files#diff-e80ac8ac46156826332990f59e0444d4R215

[dnephin]

The docs from (#2290) will be going live today at http://docs.docker.com/compose/extends/

[pwaterz]

@dnephin I appreciate your explanation above, but I would like to know if this could be re opened. I understand the reason for the current implementation, with having to order of precedence, but I feel honoring the location of the compose file in respect to volumes would be a great feature to have. For example, I and have seen many others maintain a git repos with many different docker setups for different applications with a flat file structure like so

• app1
  • docker-compose.yml
• app2
  • docker-compose.yml
  • somefile-that-get-mounted.txt

Sometimes I need app2 with app1 to create launch a larger piece of my development stack. With the logic that currently exists, the only way to get this to work is to copy somefile-that-get-mounted.txt in app1 and now I have to maintain the same file two places or I have to launch each compose files separately and join them into the same network. Thoughts? Is there a better way to solve my problem that I am unaware of?