Treat static, index-creation-only settings as read-only in normalizer - #1146
Closed
jwils wants to merge 1 commit into
Closed
Treat static, index-creation-only settings as read-only in normalizer#1146jwils wants to merge 1 commit into
jwils wants to merge 1 commit into
Conversation
jwils
requested review from
BrianSigafoos-SQ,
ayousufi,
bsorbo,
jwondrusch and
myronmarston
as code owners
April 22, 2026 18:09
jwils
force-pushed
the
joshuaw/normalizer-ignore-routing-partition-size
branch
from
April 22, 2026 19:01
d9aa9f0 to
66d9700
Compare
`index.routing_partition_size` and `index.codec` are both static (non-dynamic) index settings. OpenSearch exposes them on read but rejects PUTs against `_settings` for them on an open index: - `routing_partition_size` is index-creation-only and defaults to `1` when not set, so any PUT for it (including PUTs of null to restore the default) is rejected outright with `Can't update non dynamic settings`. - `index.codec` can only be changed on a closed index, so PUTs against an open index fail the same way (`Can't update non dynamic settings [[index.codec]] for open indices [...]`). The admin update loop diffs current vs desired settings and PUTs null for anything present in current but absent from desired, so running `clusters:configure:perform` against a cluster that surfaces either setting fails with a 400. Add both to `READ_ONLY_SETTINGS` so the normalizer drops them from current_settings and the update loop leaves them alone, matching the pattern established in #1144.
jwils
force-pushed
the
joshuaw/normalizer-ignore-routing-partition-size
branch
from
April 22, 2026 19:45
66d9700 to
de51043
Compare
myronmarston
requested changes
Apr 22, 2026
Collaborator
There was a problem hiding this comment.
I'm confused by this.
- IIRC, on the earliest EG projects, we used
index.codecstarting in 2021 or 2022 and never ran into this. What changed? What error are you seeing and why? - There are lots of static, non-dynamic settings. What's special about these two which need to be filtered out? For example,
shard_countis a static, non-dynamic setting, but you're not excluding it here. It's commonly managed by EG. Why doesn't it have the same problem? - I might be missing something, but if you add these settings to
READ_ONLY_SETTINGS, then I don't think you can use EG to configure these on index creation anymore, because (presumably) EG will filter them out when creating the index. (I haven't dug through the code to confirm that'll happen, but that's how I recall the normalizer being used).
Historically, the only settings we put in here are ones that "appeared" on a managed cluster (e.g. AWS added it behind the scenes) and which we can't change anyway.
Collaborator
|
Put a different way: I think there's a difference between a read-only setting (that is, a setting which ElasticGraph can only ever read) and a write-once-at-index-creation setting. I wouldn't consider a setting which EG is able to write as part of creating an index to be read-only. |
Collaborator
Author
|
Was trying to deploy a change to use zstd, but totally right just need to update the index settings to actually show what's applied. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
index.routing_partition_sizeandindex.codecare both static (non-dynamic) index settings. OpenSearch exposes them on read but rejects PUTs against_settingsfor them on an open index.routing_partition_sizeis index-creation-only and defaults to1when unset, so any PUT (including PUTs of null to restore the default) is rejected outright.index.codeccan only be changed on a closed index, so PUTs against an open index fail withCan't update non dynamic settings [[index.codec]] for open indices [...].clusters:configure:performagainst a cluster that surfaces either setting fails with a 400.READ_ONLY_SETTINGSinIndexConfigNormalizerso they are dropped from current_settings and the update loop leaves them alone. Follows the pattern established in Treat shrink/split/clone source settings as read-only in normalizer #1144.Observed failure (motivating
index.codec)Surfaced from
ElasticGraph::OpenSearch::Client#put_index_settingsviaIndexDefinitionConfigurator::ForIndex#update_settingswhile runningclusters:configure:perform.Test plan
index_config_normalizer_spec.rbto include bothindex.routing_partition_sizeandindex.codecin the read-only filter case;bundle exec rspec spec/unit/elastic_graph/datastore_core/index_config_normalizer_spec.rbpasses (10 examples, 0 failures).References:
_routing/routing_partition_size: https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping-routing-field.html