Skip to content

[backend/frontend] Support relationship-based neighbor resolution in CSV Feed columns (#14680)#14681

Merged
SamuelHassine merged 6 commits intomasterfrom
issue/14680
Mar 4, 2026
Merged

[backend/frontend] Support relationship-based neighbor resolution in CSV Feed columns (#14680)#14681
SamuelHassine merged 6 commits intomasterfrom
issue/14680

Conversation

@SamuelHassine
Copy link
Member

Summary

Implements #14680 — Adds support for relationship-based neighbor resolution in CSV Feed columns. Users can now configure feed columns that resolve first-degree neighbors through relationships and extract their attributes at runtime, enabling enriched, flat CSV output without additional API calls.

What changed

New column mapping mode: Relationship (neighbor)

Each entity type mapping within a feed column can now be toggled between two modes:

  • Direct (existing behavior) — maps to a direct attribute of the entity (e.g., indicator.value, indicator.confidence)
  • Relationship (new) — resolves a neighbor entity through a relationship and extracts an attribute from it (e.g., Indicator → indicates → Malware → name)

Configurable multi-match behavior

When multiple neighbors match (e.g., an Indicator indicates 2 Malware), users can configure per column:

  • Strategy: "All (list)" to join all matches, or "First match" to take only the first
  • List separator: configurable separator for joining multiple values (defaults to ,), validated to differ from the feed's CSV separator to prevent parsing ambiguity

Backend changes (5 files)

  • GraphQL schema (opencti.graphql) — Extended FeedMapping with relationship_type and target_entity_type; extended FeedAttribute with multi_match_strategy and multi_match_separator
  • Elasticsearch index mapping (internalObject-registrationAttributes.ts) — Registered new fields in the Feed entity attribute definitions for proper strict-mapping indexing
  • TypeScript types (store.d.ts) — Updated BasicStoreEntityFeed with the new optional fields
  • Domain validation (feed.ts) — Validates relationship_type/target_entity_type must both be set or both absent; validates multi_match_strategy and multi_match_separator (must differ from feed CSV separator)
  • Feed engine (httpRollingFeed.ts) — Core runtime logic:
    • resolveNeighborsForFeed() — batch-queries relationships for all loaded entities across both directions (FROM and TO), then batch-resolves target entities via elFindByIds. Returns a pre-computed lookup map
    • Updated buildCsvLines() to handle neighbor-based mappings using the pre-resolved lookup map, applying the configurable multi-match strategy and separator

Frontend changes (3 files + generated)

  • FeedCreation.tsx and FeedEdition.jsx — Redesigned column configuration UI:
    • Entity type label with a clickable Chip toggle (Direct / Relationship)
    • In Relationship mode: 3-column grid with Relationship type → Target type → Attribute dropdowns, populated from schemaRelationsTypesMapping (both directions)
    • Multi-match strategy dropdown and list separator text field appear when any mapping uses Relationship mode
    • Dividers between entity type sections for visual clarity
  • FeedLine.jsx — Updated Relay fragment with new fields

Performance

  • Batch relationship queries: one fullRelationsList call per unique (rel_type, target_type) pair — not per entity
  • Batch entity resolution: one elFindByIds call for all target IDs
  • Pre-computed lookup map before iterating CSV rows
  • Bounded by existing SIZE_LIMIT (5000 entities)

Backward compatibility

  • All new fields are optional (nullable in GraphQL, optional in TypeScript)
  • Existing feeds with only direct mappings continue to work unchanged
  • multi_match_strategy defaults to list, multi_match_separator defaults to ,

Closes #14680

Test plan

  • Create a feed with only direct attribute columns → verify CSV output is unchanged (backward compatibility)
  • Create a feed with a relationship column (e.g., Indicator → indicates → Malware → name) → verify the neighbor's attribute appears in the CSV
  • Test with multiple neighbors matching → verify comma-separated list output with "All (list)" strategy
  • Switch to "First match" strategy → verify only one value appears
  • Change the list separator (e.g., to |) → verify it is used in the CSV output
  • Attempt to set list separator to same as CSV separator → verify validation error
  • Toggle between Direct and Relationship mode → verify smooth switching in both directions
  • Edit an existing feed with neighbor columns → verify data loads correctly in the edition form
  • Test with a feed that has multiple entity types → verify per-type mapping works independently

@SamuelHassine SamuelHassine changed the title [backend/frontend] Support relationship-based neighbor resolution in CSV Feed columns [backend/frontend] Support relationship-based neighbor resolution in CSV Feed columns (#14680) Feb 28, 2026
@codecov
Copy link

codecov bot commented Feb 28, 2026

Codecov Report

❌ Patch coverage is 12.03501% with 402 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.47%. Comparing base (c1b2181) to head (fe11d7e).
⚠️ Report is 24 commits behind head on master.

Files with missing lines Patch % Lines
...atform/opencti-graphql/src/http/httpRollingFeed.ts 20.41% 152 Missing ⚠️
...src/private/components/data/feeds/FeedCreation.tsx 0.00% 115 Missing ⚠️
.../src/private/components/data/feeds/FeedEdition.jsx 0.00% 115 Missing ⚠️
...pencti-platform/opencti-graphql/src/domain/feed.ts 47.82% 12 Missing ⚠️
...encti-platform/opencti-front/src/utils/Relation.ts 11.11% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #14681      +/-   ##
==========================================
+ Coverage   32.36%   32.47%   +0.10%     
==========================================
  Files        3097     3099       +2     
  Lines      210976   212616    +1640     
  Branches    38232    38864     +632     
==========================================
+ Hits        68280    69037     +757     
- Misses     142696   143579     +883     
Flag Coverage Δ
opencti-client-python 44.38% <ø> (-1.10%) ⬇️
opencti-front 2.86% <0.41%> (+0.03%) ⬆️
opencti-graphql 67.95% <24.77%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds relationship-based neighbor resolution for CSV Feed columns so feeds can output attributes from first-degree related entities (with configurable multi-match behavior) across backend runtime, schema/types, and frontend configuration UI.

Changes:

  • Extend Feed GraphQL schema/types and strict-mapping attribute registration to store relationship-based mapping configuration and multi-match settings.
  • Implement backend neighbor resolution + CSV rendering logic using batched relationship queries and entity resolution.
  • Update frontend feed creation/edition UIs and Relay fragments to configure relationship mappings and multi-match options; add i18n/doc updates.

Reviewed changes

Copilot reviewed 27 out of 29 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
opencti-platform/opencti-graphql/src/types/store.d.ts Adds optional fields for relationship mapping + multi-match settings on Feed types.
opencti-platform/opencti-graphql/src/modules/attributes/internalObject-registrationAttributes.ts Registers new Feed attribute fields for strict ES mapping/indexing.
opencti-platform/opencti-graphql/src/http/httpRollingFeed.ts Implements neighbor resolution and integrates it into CSV line generation.
opencti-platform/opencti-graphql/src/generated/graphql.ts Updates generated GraphQL TS types for new Feed fields.
opencti-platform/opencti-graphql/src/domain/feed.ts Adds validation for relationship mapping integrity and multi-match config.
opencti-platform/opencti-graphql/config/schema/opencti.graphql Extends Feed schema inputs/outputs with new fields.
opencti-platform/opencti-front/src/schema/relay.schema.graphql Mirrors schema changes for frontend Relay schema.
opencti-platform/opencti-front/src/private/components/data/feeds/FeedLine.jsx Updates feed fragments to fetch new mapping fields.
opencti-platform/opencti-front/src/private/components/data/feeds/FeedEdition.jsx Adds UI for Direct/Relationship toggle + multi-match configuration in edit form.
opencti-platform/opencti-front/src/private/components/data/feeds/FeedCreation.tsx Adds UI for Direct/Relationship toggle + multi-match configuration in create form.
opencti-platform/opencti-front/lang/front/.json / lang/back/.json Adds/updates translations for new UI labels/fields.
docs/docs/usage/feeds.md Documents relationship-based columns and multi-match behavior for CSV feeds.

@fellowseb fellowseb self-requested a review March 2, 2026 13:36
Copy link
Member

@fellowseb fellowseb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat feature !
Tested on my side 👍 .

Consider augmenting httpRollingFeed-test.ts with a least one test with a populated neighbor map or better that calls resolveNeighborsForFeed to get the entire thing tested.

Also, from a UX perspective here are two suggestions:

  • (bonus) Preventing from validating the form if a "List separator" matches the main "Separator"
  • (ultra bonus) Having a local state to keep track of the most recent "List separator" so that next column definition gets populated with it instead of ,

…esolution (#14680)

- Extract relationship utils to shared Relation.ts (reuse getRelationshipTypesForEntityType, add getTargetTypesForRelationship)

- Add frontend validation preventing form submit when list separator collides with CSV separator

- Fix hardcoded comma in extractAttributeFromEntity to use configurable separator

- Simplify mapping validation checks

- Add unit tests for neighbor-based CSV line generation (list, first-match, empty)

- Add i18n key for separator collision error message

Made-with: Cursor
@SamuelHassine
Copy link
Member Author

Added 3 new test cases in httpRollingFeed-test.ts covering neighbor-based CSV generation: list strategy with multiple neighbors, first-match strategy, and empty neighbors map. All addressed in 83c012d.

@SamuelHassine SamuelHassine requested a review from fellowseb March 4, 2026 11:46
@SamuelHassine SamuelHassine merged commit 20f9a6f into master Mar 4, 2026
36 checks passed
@SamuelHassine SamuelHassine deleted the issue/14680 branch March 4, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CSV Feed] Support relationship-based neighbor resolution in CSV Feed columns

3 participants