[backend/frontend] Support relationship-based neighbor resolution in CSV Feed columns (#14680)#14681
[backend/frontend] Support relationship-based neighbor resolution in CSV Feed columns (#14680)#14681SamuelHassine merged 6 commits intomasterfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #14681 +/- ##
==========================================
+ Coverage 32.36% 32.47% +0.10%
==========================================
Files 3097 3099 +2
Lines 210976 212616 +1640
Branches 38232 38864 +632
==========================================
+ Hits 68280 69037 +757
- Misses 142696 143579 +883
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…umns (#14680) Made-with: Cursor
There was a problem hiding this comment.
Pull request overview
Adds relationship-based neighbor resolution for CSV Feed columns so feeds can output attributes from first-degree related entities (with configurable multi-match behavior) across backend runtime, schema/types, and frontend configuration UI.
Changes:
- Extend Feed GraphQL schema/types and strict-mapping attribute registration to store relationship-based mapping configuration and multi-match settings.
- Implement backend neighbor resolution + CSV rendering logic using batched relationship queries and entity resolution.
- Update frontend feed creation/edition UIs and Relay fragments to configure relationship mappings and multi-match options; add i18n/doc updates.
Reviewed changes
Copilot reviewed 27 out of 29 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| opencti-platform/opencti-graphql/src/types/store.d.ts | Adds optional fields for relationship mapping + multi-match settings on Feed types. |
| opencti-platform/opencti-graphql/src/modules/attributes/internalObject-registrationAttributes.ts | Registers new Feed attribute fields for strict ES mapping/indexing. |
| opencti-platform/opencti-graphql/src/http/httpRollingFeed.ts | Implements neighbor resolution and integrates it into CSV line generation. |
| opencti-platform/opencti-graphql/src/generated/graphql.ts | Updates generated GraphQL TS types for new Feed fields. |
| opencti-platform/opencti-graphql/src/domain/feed.ts | Adds validation for relationship mapping integrity and multi-match config. |
| opencti-platform/opencti-graphql/config/schema/opencti.graphql | Extends Feed schema inputs/outputs with new fields. |
| opencti-platform/opencti-front/src/schema/relay.schema.graphql | Mirrors schema changes for frontend Relay schema. |
| opencti-platform/opencti-front/src/private/components/data/feeds/FeedLine.jsx | Updates feed fragments to fetch new mapping fields. |
| opencti-platform/opencti-front/src/private/components/data/feeds/FeedEdition.jsx | Adds UI for Direct/Relationship toggle + multi-match configuration in edit form. |
| opencti-platform/opencti-front/src/private/components/data/feeds/FeedCreation.tsx | Adds UI for Direct/Relationship toggle + multi-match configuration in create form. |
| opencti-platform/opencti-front/lang/front/.json / lang/back/.json | Adds/updates translations for new UI labels/fields. |
| docs/docs/usage/feeds.md | Documents relationship-based columns and multi-match behavior for CSV feeds. |
opencti-platform/opencti-front/src/private/components/data/feeds/FeedEdition.jsx
Outdated
Show resolved
Hide resolved
opencti-platform/opencti-front/src/private/components/data/feeds/FeedCreation.tsx
Outdated
Show resolved
Hide resolved
fellowseb
left a comment
There was a problem hiding this comment.
Neat feature !
Tested on my side 👍 .
Consider augmenting httpRollingFeed-test.ts with a least one test with a populated neighbor map or better that calls resolveNeighborsForFeed to get the entire thing tested.
Also, from a UX perspective here are two suggestions:
- (bonus) Preventing from validating the form if a "List separator" matches the main "Separator"
- (ultra bonus) Having a local state to keep track of the most recent "List separator" so that next column definition gets populated with it instead of
,
opencti-platform/opencti-front/src/private/components/data/feeds/FeedCreation.tsx
Outdated
Show resolved
Hide resolved
opencti-platform/opencti-front/src/private/components/data/feeds/FeedCreation.tsx
Outdated
Show resolved
Hide resolved
opencti-platform/opencti-front/src/private/components/data/feeds/FeedEdition.jsx
Outdated
Show resolved
Hide resolved
…esolution (#14680) - Extract relationship utils to shared Relation.ts (reuse getRelationshipTypesForEntityType, add getTargetTypesForRelationship) - Add frontend validation preventing form submit when list separator collides with CSV separator - Fix hardcoded comma in extractAttributeFromEntity to use configurable separator - Simplify mapping validation checks - Add unit tests for neighbor-based CSV line generation (list, first-match, empty) - Add i18n key for separator collision error message Made-with: Cursor
|
Added 3 new test cases in |
Summary
Implements #14680 — Adds support for relationship-based neighbor resolution in CSV Feed columns. Users can now configure feed columns that resolve first-degree neighbors through relationships and extract their attributes at runtime, enabling enriched, flat CSV output without additional API calls.
What changed
New column mapping mode: Relationship (neighbor)
Each entity type mapping within a feed column can now be toggled between two modes:
indicator.value,indicator.confidence)indicates→ Malware →name)Configurable multi-match behavior
When multiple neighbors match (e.g., an Indicator
indicates2 Malware), users can configure per column:,), validated to differ from the feed's CSV separator to prevent parsing ambiguityBackend changes (5 files)
opencti.graphql) — ExtendedFeedMappingwithrelationship_typeandtarget_entity_type; extendedFeedAttributewithmulti_match_strategyandmulti_match_separatorinternalObject-registrationAttributes.ts) — Registered new fields in the Feed entity attribute definitions for proper strict-mapping indexingstore.d.ts) — UpdatedBasicStoreEntityFeedwith the new optional fieldsfeed.ts) — Validates relationship_type/target_entity_type must both be set or both absent; validates multi_match_strategy and multi_match_separator (must differ from feed CSV separator)httpRollingFeed.ts) — Core runtime logic:resolveNeighborsForFeed()— batch-queries relationships for all loaded entities across both directions (FROM and TO), then batch-resolves target entities viaelFindByIds. Returns a pre-computed lookup mapbuildCsvLines()to handle neighbor-based mappings using the pre-resolved lookup map, applying the configurable multi-match strategy and separatorFrontend changes (3 files + generated)
schemaRelationsTypesMapping(both directions)Performance
fullRelationsListcall per unique (rel_type, target_type) pair — not per entityelFindByIdscall for all target IDsSIZE_LIMIT(5000 entities)Backward compatibility
multi_match_strategydefaults tolist,multi_match_separatordefaults to,Closes #14680
Test plan
|) → verify it is used in the CSV output