I cannot for the life of me figure out if there is an existing way to do this. It appears the env context is locked down, so not sure if this is actually going to require a code change.
FWIW, I'd like to set it to GOARM64="v9.0,lse,crypto"
Edit: See #4557 with what I think is how to do this.