ConfigMapGenerator should not load values from the build environment

[KnVerey]

It recently came to @natasha41575 and my attention that the configMapGenerator has a bug where it will load values from the environment when a line in an env file has a key but no value. This bug dates back to the origins of Kustomize, when some of the original code was copied out of kubectl, and this undesirable behaviour (for kustomize, not kubectl) went unnoticed in the adaptation.

This behaviour is a clear violation of one of Kustomize's core principles, as documented in our Eschewed Features list:

Very unfortunately, someone not only discovered this bug, but made a PR to document it as a feature on kubernetes.io: kubernetes/website#30348. Unfortunately no Kustomize maintainers were tagged on the PR, and it has been live since January 2022. 😞

As stated in Kustomize documentation screenshot above, any need for environment reading should be handled as part of a pre-build workflow. We can consider adding a kustomize edit subcommand for this specifically if there is demand from those currently relying on the bug. For example, it could look like kustomize edit add configmap my-configmap --from-env-file=env/path.env --from-env=ONE,TWO.

Since this behaviour has been around unnoticed by us for so long, our plan is to to deprecate it and remove it with a major version bump rather than treating it like a bug fix. We are reverting the docs change in kubernetes/website#35522 and emitting a warning to affected users in #4730. The final step is to actually fix the behaviour in the next major release, i.e. Kustomize v5.

[k8s-ci-robot]

@KnVerey: This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[afirth]

Hi @KnVerey, thanks for writing this up!

I am the documentor. Frankly I think it's a bad feature to eschew, and leaving in the ability to fetch manifests from remote targets while pulling this out feels like just making my life as a user hard. Kustomize is currently a single, fully usable tool for manifests. If we remove this, we will have to add another one on top of it. I'm totally sick of that. Judging by the number of issues I responded to, which resulted in me documenting it, other users don't want you to eschew this feature either.

The feature has been critical several times over the last couple years. The workaround proposed requires CD to know both the env var name, the configmap name, and the folder the configmap is in in the CD pipeline definition. This means that the CD pipeline must be custom for the particular project, which feels ridiculous and exactly the problem that skaffold + kustomize solve by allowing me to declare that in yaml next to the code instead. So either we need a hook system or another system to communicate into kustomize. This one works fine, has no unexpected consequences (you can still set to empty by including the =), and is a desired feature by your users. The major benefit is that grepping for the variable name leads to something in the codebase, rather than existing solely in the CD pipe.

I'd like to hear stronger reasons for removing it than "we didn't think it was a good idea 4 years ago". I totally understand the stance "we don't want to be a templating engine" but replacing this interface with calls to the CLI that take two pieces of info the caller doesn't have is not great - nor is suggesting users write a transformer to run sed. Maybe I'm missing some key info. I just really don't see how calling kustomize edit in a CD pipeline which may be remote from the codebase leads to a better developer experience.

A reasonable compromise would perhaps be a kustomize edit command to set a var at the top level kustomization - but those are going/gone too.

My 2c as a long-time user and fan currently running kustomize+skaffold as CD for ~25 services. Here's another user's perspective, which I pretty much agree with.

p.s. apologies for not tagging any maintainers on the docs PR. Perhaps you want to add an OWNERS for that page?
p.p.s. nice to meet you, however briefly, at kubecon, hope your gateway api experiments went ok.

cc @justinsb - this is the one we were talking about at kubecon and also 👋

use case

Here is the use case I originally dug it out for - extracting the build tag to provide to an operator which spawns more pods at the same tag, but needs the tag in a config file:

      - name: Extract skaffold image tags to the environment
        # some things need to know their tag;
        # this let's kustomize or skaffold populate it from the env
        continue-on-error: true # continue if no build artifacts are found in skaffold.yaml
        run: |
          skaffold build -q --dry-run | \
            jq -r '.builds | map(.tag |= split(":"))[0] |"BUILD_SKAFFOLD_TAG="+.tag[1]' >> $GITHUB_ENV

[afirth]

Just to be clear, I don't think the environment fallthrough is a good way - but kustomize edit feels like a much worse way. Maybe I'm building my pipes wrong. Also, sorry for saying "it's intentional" in previous issues and the docs PR - I didn't consider the possibility of copypasta nor chase the git blame as far as I could have.

[Gikkman]

This change is going to be a massive issue in our organization. We've considered it a feature for a long time (no idea how we found it originally though), and rely on it for templating ingress URLs, since most of our testing environments are generated on-demand. It's been very handy, since at least for us, the namespace and the cluster host has many times been necessary during manifest rendition.

Basically, a developer requests a testing environment. A namespace is generated on kubernetes with a random namespace name. That name is then part of the URL (e.g. product-$(namespace).$(cluster-host).com). I know there are some other places where namespace and cluster-host has been necessary (for example, if an application needs to send a callback URL to an external service, then it needs to know what the address of its own ingress became)

I could image a command kustomize edit set var KEY=VALUE or something would be a possible solution... but I've frankly never really understood why the kustomize edit... commands are fine because they also move data out of version control, don't they? I don't see how having a single file which declares all env variables we rely on is a worse practise than having several kustomize edit... commands as part of our CI pipeline before calling kustomize build.

I agree pretty much with @afirth on this topic, and really hope you reconsider this

[ciis0]

In our project we use this feature to inject credentials into ConfigMaps, using kustomize edit risks exposing the credentials in left-over files (we pipe kustomize build directly into kubectl apply).

We also hope you reconsider this.

[cedric-orange]

I can understand your choice for ConfigMapGenerator, but this warning is also for secretGenerator.

Today, our secrets are stored in Gitlab CI variables and are masked. Edit kustomize file to set these secrets variables is not a good way.

[max-wittig]

Sad to see this feature go. Please reconsider not changin it. It's really useful, when deploying from pipelines where secrets are injected. In addition its one of the 12 factor principles: https://12factor.net/config