Backend error: strict_dynamic_mapping_exception, ([publisher] or [rating] within [attachment]), fileIndexManager usecase

[fellowseb]

Symptoms / detection

The backend strict_dynamic_mapping_exception error was detected with the following specific fields:

• cause_message: Update indexing fail

• cause_attributes_cause_message:

strict_dynamic_mapping_exception
	Root causes:
		strict_dynamic_mapping_exception: [1:538] mapping set to strict, dynamic introduction of [publisher] within [attachment] is not allowed

• cause_attributes_cause_stack:

ResponseError: strict_dynamic_mapping_exception
	Root causes:
		strict_dynamic_mapping_exception: [1:538] mapping set to strict, dynamic introduction of [publisher] within [attachment] is not allowed
    at SniffingTransport._request (/opt/opencti/build/back.js:790:194528)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async /opt/opencti/build/back.js:790:197793
    at async SniffingTransport.request (/opt/opencti/build/back.js:790:197613)
    at async _Client.IndexApi [as index] (/opt/opencti/build/back.js:853:172288)
    at async elIndex (/opt/opencti/build/back.js:1829:21943)
    at async elIndexFiles (/opt/opencti/build/back.js:1878:105502)
    at async indexImportedFiles (/opt/opencti/build/back.js:4285:18107)
    at async fileIndexHandler (/opt/opencti/build/back.js:4285:20286)
    at async /opt/opencti/build/back.js:4285:22052
    at async _SetIntervalAsyncTimer.runHandlerAndScheduleTimeout_fn (/opt/opencti/build/back.js:894:3825)
    at async Timeout._onTimeout (/opt/opencti/build/back.js:894:3612)

• cause_message: Simple indexing fail

Happened in versions 6.9.15 to 6.9.20.

Analysis / explanation

The general explanation of the strict_dynamic_mapping_exception error in Elasticsearch is that it occurs when you try to index a document with a new field that doesn't match the existing mapping rules in a strict dynamic mapping context. This exception is thrown to prevent unintended schema changes that could lead to data corruption or inconsistent mappings.

This specific instance of strict_dynamic_mapping_exception occurs in the fileIndexManager use case. Docs here.

More investigation needed to understand the root cause.

Resolution brainstorm

See PR description.