It should allow extended services to define links

[docteurklein]

Hi!

I'm in a use-case where I have a basic docker-compose.yml file that defines a php service talking to a db service.

Now I added a second file named "dev.yml" where I extend all the services from docker-compose.yml and I add a volumes key to the php service.

This scenario does not work (services with 'links' cannot be extended) and I don't think the docs are quite clear on this behavor. (EDIT: it's clear but stay with me :))

Docs says that links won't be copied:

Compose copies configurations from the original service over to the local one, except for links and volumes_from. These exceptions exist to avoid implicit dependencies—you always define links and volumes_from locally.

What I'd like is to not being forced to create a prod.yml file, just to add a links section.
I'm perfectly OK with the idea of always defining links locally (and not merging the array like it's done with expose f.e).
But I think the base file can also be considered a perfectly valid leaf.

I'd like my docker-compose.yml file to be the standard, and dev.yml to add some volumes (and define links locally in it).

I hope I'm clear :)
I think there is no reasion why I can't define links in the base file. It would be great if I just had to redefine them if needed in any other file that extends it.

[BinaryMiscreant]

I run into this development scenario as well, wanting some dev.yml to just add a few things (volume, variables, etc. ) It would be nice to specify you want to inherit links in some way.

[docteurklein]

@BinaryMiscreant The idea is not to inherit the links, (you'll still have to redefine them at each extends).
The reason (and I think it's a valid one) is to be able to know the exact dependencies by reading only one file.

But my proposition is not about merging links with parent file, It's to let the parent file be able to define links on its own. And if a child overrides this parent, let it redefine all the links.

@aanand thoughts?

[dunk]

This seems overly restrictive. I have just come across this problem myself. On the one hand you have services all specified in one place, and on the other better code factorisation. I'd give people the choice, different circumstances will warrant different things.

[viranch]

+1 I agree. Extending links is seldom a destructive action anyway.

[mvallerie]

+1 for this.

[lucernae]

+1 for this.
From the documentation, it is clear that we have to define the links again locally. But compose doesn't accepts services with predefined links. I'm not against the idea of defining the links again. But, if I have to extend it from services which doesn't have links defined, then I end up having to create 3 yml configuration: common yml without links, production yml with links, and development yml with links. I'd rather not repeat myself in those configuration files.

A general use case why I need an extend features is because I want to expose a port of certain services without modifying its link in dev environment, or change some environment variables. Having to redefine the links actually feels complicating things.

[samuelololol]

+1 very agree!

As https://docs.docker.com/compose/extends/#reference says, links and volumes_from should be defined locally. As I know, the only way to extends those docker-compose.yml is to refactor them into common and link parts(maybe 2+ files) and use the common part to extend.

I am just wondering are there any shallow exdends to accomplish these things ?

[c0deaddict]

+1

[michaeljs1990]

kind of crazy that it doesn't just ignore links / volume_from and tell you that you must redefine it. This is a huge pain point and cases me to have duplicated docker-compose code all over the place that I must not manually watch and update.

Without supporting these features extend is almost completely worthless.

[aanand]

Thanks for your patience with this one, folks. Have a look at the new functionality added in #2051 - you can pass multiple files to Compose with -f, each acting as a file-level extends. Links etc. are allowed.

[michaeljs1990]

this is awesome @aanand. thanks for the info. hard to keep up with all the changes.

[samuelololol]

@aanand Is this override feature also applied in docker-compose.yml file for extends ? I mean will it settings override parents' service?

[aanand]

@samuelololol We need to nail down how extends and multiple -f args should interact. We have a few options:

1. Don't allow them to be used together. Nice and simple.
2. Fully resolve extends in every file as part of reading it in, then go through the files in sequence.
3. Go through the files in sequence to produce a service->config mapping, allowing each file's extends keys to override the previous one's, then resolve extends in each service that has it defined.

What's the intention of the following config?

docker-compose.yml

web:
  extends:
    file: common.yml
    service: foo

docker-compose.overrides.yml

web:
  extends:
    file: common.yml
    service: bar

To me, it looks like the user probably wants web to have (by default) all the config from bar, and none of the config from foo. Which would be a case for option 3.

But I'm not sure anyone wants to do this in the first place. Which would be a case for option 1 - don't let people combine the features unless it turns out they want to, then figure out what they're trying to achieve, then decide if either of options 2 or 3 is therefore a good idea.

Would you want to combine them? How, and why?

[samuelololol]

Actually, what I want is extended configs in service->config mapping could be replaced in extended docker-compose.yml. They can help me to reuse existing docker-compose.yml without spliting them into two separate parts, docker-compose-common.yml and docker-compose.yml. As an example described in this issue, links cannot be redefined in extended config.

#2051 is a great feature to help modulize configs, but to me, it probably will not gracefully solve this case, I don't see helps to this by using -f argument to extends multiple docker-compsoe files. I would like to know the detail of extends in yaml and see if it's possible, but from your description it apparently not. :(

I still provide my thought to this. It is simple and easy as well. I would use extends in yaml as much as I could because the purpose of my using docker-compose is to reduce command line work of docker command, and I don't want it back again in docker-compose. And also, I don't see any possible yet to combine them together.