Skip to content

Treat static, index-creation-only settings as read-only in normalizer - #1146

Closed
jwils wants to merge 1 commit into
mainfrom
joshuaw/normalizer-ignore-routing-partition-size
Closed

Treat static, index-creation-only settings as read-only in normalizer#1146
jwils wants to merge 1 commit into
mainfrom
joshuaw/normalizer-ignore-routing-partition-size

Conversation

@jwils

@jwils jwils commented Apr 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • index.routing_partition_size and index.codec are both static (non-dynamic) index settings. OpenSearch exposes them on read but rejects PUTs against _settings for them on an open index.
    • routing_partition_size is index-creation-only and defaults to 1 when unset, so any PUT (including PUTs of null to restore the default) is rejected outright.
    • index.codec can only be changed on a closed index, so PUTs against an open index fail with Can't update non dynamic settings [[index.codec]] for open indices [...].
  • The admin update loop diffs current vs desired settings and PUTs null for anything present in current but missing from desired, so running clusters:configure:perform against a cluster that surfaces either setting fails with a 400.
  • Adds both settings to READ_ONLY_SETTINGS in IndexConfigNormalizer so they are dropped from current_settings and the update loop leaves them alone. Follows the pattern established in Treat shrink/split/clone source settings as read-only in normalizer #1144.

Observed failure (motivating index.codec)

[400] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't update non dynamic settings [[index.codec]] for open indices [[orders_rollover__2026/aq2jMQpZTRiLiBhsCaRbUA]]"}],"type":"illegal_argument_exception","reason":"Can't update non dynamic settings [[index.codec]] for open indices [[orders_rollover__2026/aq2jMQpZTRiLiBhsCaRbUA]]"},"status":400}

Surfaced from ElasticGraph::OpenSearch::Client#put_index_settings via IndexDefinitionConfigurator::ForIndex#update_settings while running clusters:configure:perform.

Test plan

  • Updated index_config_normalizer_spec.rb to include both index.routing_partition_size and index.codec in the read-only filter case; bundle exec rspec spec/unit/elastic_graph/datastore_core/index_config_normalizer_spec.rb passes (10 examples, 0 failures).

References:

@jwils
jwils force-pushed the joshuaw/normalizer-ignore-routing-partition-size branch from d9aa9f0 to 66d9700 Compare April 22, 2026 19:01
@jwils jwils changed the title Treat index.routing_partition_size as read-only in normalizer Treat static, index-creation-only settings as read-only in normalizer Apr 22, 2026
`index.routing_partition_size` and `index.codec` are both static
(non-dynamic) index settings. OpenSearch exposes them on read but
rejects PUTs against `_settings` for them on an open index:

- `routing_partition_size` is index-creation-only and defaults to `1`
  when not set, so any PUT for it (including PUTs of null to restore the
  default) is rejected outright with `Can't update non dynamic
  settings`.
- `index.codec` can only be changed on a closed index, so PUTs against
  an open index fail the same way (`Can't update non dynamic settings
  [[index.codec]] for open indices [...]`).

The admin update loop diffs current vs desired settings and PUTs null
for anything present in current but absent from desired, so running
`clusters:configure:perform` against a cluster that surfaces either
setting fails with a 400.

Add both to `READ_ONLY_SETTINGS` so the normalizer drops them from
current_settings and the update loop leaves them alone, matching the
pattern established in #1144.
@jwils
jwils force-pushed the joshuaw/normalizer-ignore-routing-partition-size branch from 66d9700 to de51043 Compare April 22, 2026 19:45

@myronmarston myronmarston left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused by this.

  • IIRC, on the earliest EG projects, we used index.codec starting in 2021 or 2022 and never ran into this. What changed? What error are you seeing and why?
  • There are lots of static, non-dynamic settings. What's special about these two which need to be filtered out? For example, shard_count is a static, non-dynamic setting, but you're not excluding it here. It's commonly managed by EG. Why doesn't it have the same problem?
  • I might be missing something, but if you add these settings to READ_ONLY_SETTINGS, then I don't think you can use EG to configure these on index creation anymore, because (presumably) EG will filter them out when creating the index. (I haven't dug through the code to confirm that'll happen, but that's how I recall the normalizer being used).

Historically, the only settings we put in here are ones that "appeared" on a managed cluster (e.g. AWS added it behind the scenes) and which we can't change anyway.

@myronmarston

Copy link
Copy Markdown
Collaborator

Put a different way: I think there's a difference between a read-only setting (that is, a setting which ElasticGraph can only ever read) and a write-once-at-index-creation setting. I wouldn't consider a setting which EG is able to write as part of creating an index to be read-only.

@jwils

jwils commented Apr 23, 2026

Copy link
Copy Markdown
Collaborator Author

Was trying to deploy a change to use zstd, but totally right just need to update the index settings to actually show what's applied.

@jwils jwils closed this Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants