[CAI-876] Update synced files for Strapi v5

[mdciri]

Depends on #2134

List of Changes

Update of the reading of the API and Products files in AWS S3 generated by Strapi v5

Motivation and Context

Strapi v5 saves the files differently from the previous version

How Has This Been Tested?

Locally with Docker compose

Screenshots (if appropriate):

Types of changes

• Chore (nothing changes by a user perspective)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[changeset-bot]

🦋 Changeset detected

Latest commit: cb26dc6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
chatbot-index	Patch
chatbot	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Copilot]

Pull request overview

This PR updates the chatbot and indexing pipeline to consume the new Strapi v5 S3 sync JSON structure (notably the top-level data array and flattened fields), and refreshes the local Docker fixtures accordingly.

Changes:

• Update product list parsing to read from safe_json_load(...)[ "data" ] and use flattened fields (isVisible, slug).
• Update API docs parsing in the indexer to consume Strapi v5 API payload shape.
• Refresh local Docker S3 fixture JSON files and adjust the local index creation script to include API docs.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
apps/chatbot/src/modules/documents.py	Switch product parsing to Strapi v5 JSON shape; change empty-content handling.
apps/chatbot/docker/files/s3/it/synced-products-response.json	Update products fixture to Strapi v5 response format.
apps/chatbot/docker/files/s3/it/synced-apis-data-response.json	Replace APIs fixture with Strapi v5 response format (much larger payload).
apps/chatbot-index/src/modules/documents.py	Update product + API docs parsing to match Strapi v5 response format.
apps/chatbot-index/scripts/run.create.index.sh	Run index creation with --static --api instead of --static --clean-redis.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Branch is not up to date with base branch

@mdciri it seems this Pull Request is not updated with base branch.
Please proceed with a merge or rebase to solve this.

[MarBert]

is this related to the updates for Strapi 5?

[mdciri]

yes, I emulated the flow in the docker compose to test that everything works just fine. Indeed, I edited the json files of products and API as Strapi v5 does, and running the create index for the developer portal, the scripts consider the API file too.

[github-actions]

Jira Pull Request Link

This Pull Request refers to the following Jira issue CAI-876

[github-actions]

This PR exceeds the recommended size of 800 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.