Skip to content

[Server] Add live NodeManager shadow and immediate reload - #4147

Merged
marcschier merged 296 commits into
marcschier/wot-04-generator-wotfrom
marcschier/wot-05-lifecycle
Aug 5, 2026
Merged

[Server] Add live NodeManager shadow and immediate reload#4147
marcschier merged 296 commits into
marcschier/wot-04-generator-wotfrom
marcschier/wot-05-lifecycle

Conversation

@marcschier

Copy link
Copy Markdown
Collaborator

Summary

Adds PR 5 of the stack splitting #4093 into independently reviewable pieces, targeting master directly.

This PR extends the NodeManager lifecycle with live shadow reload support and RuntimeNodeSet lifecycle helpers. ShadowReloadAsync stages a replacement generation and routes new service requests to it while monitored items already owned by the retired generation continue there until subscriptions drain.

ImmediateReloadAsync keeps the immediate policy but implements it on the existing detach machinery at the maintainer's request: affected non-durable monitored items are detached/marked deleted with IDetachableMonitoredItem, queue BadNodeIdUnknown, and are not migrated to the replacement generation.

The prior NodeManager generation is disposed before ImmediateReloadAsync returns in the covered lifecycle path. The bespoke permanent retirement-error path is intentionally removed, so follow-up operations use the existing detached-item semantics rather than a separate retirement state.

Validation

  • dotnet build src\Opc.Ua.Server\Opc.Ua.Server.csproj -c Release -f net10.0 -v:m
  • dotnet build tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -p:CustomTestTarget=net10.0 -v:m
  • dotnet test tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -p:CustomTestTarget=net10.0 --no-build — 3914 passed, 9 skipped, 0 failed
  • dotnet build tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -p:CustomTestTarget=net48 -v:m
  • dotnet test tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -p:CustomTestTarget=net48 --no-build — full run hit local durable queue file-lock flake; the failed test passed in isolation. Full run otherwise reported 3913 passed, 9 skipped, 1 failed.

Copilot AI review requested due to automatic review settings July 31, 2026 10:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Adds shadow-reload and immediate-reload support to the NodeManager lifecycle, including request-drain coordination so existing monitored items can continue to be serviced by a retired generation while new requests route to a replacement generation.

Changes:

  • Introduces ShadowReloadAsync / ImmediateReloadAsync lifecycle paths and runtime NodeSet extension helpers.
  • Adds request-lifecycle coordination + shutdown sequencing updates to drain/admit requests safely during lifecycle operations.
  • Expands test coverage for runtime NodeSet configuration hooks, shadow reload behavior, and monitored-item transfer rollback.

Reviewed changes

Copilot reviewed 35 out of 38 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/Opc.Ua.Server.Tests/SubscriptionTests.cs Updates mocks to match new monitored-item transfer signature.
tests/Opc.Ua.Server.Tests/RuntimeNodeSet/RuntimeNodeSetLifecycleTests.cs Adds an end-to-end shadow reload test covering routing vs. existing monitored items.
tests/Opc.Ua.Server.Tests/RuntimeNodeSet/RuntimeNodeSetLifecycleExtensionsTests.cs New unit tests for runtime NodeSet lifecycle extension method argument contracts and wiring.
tests/Opc.Ua.Server.Tests/RuntimeNodeSet/RuntimeNodeSetConfigureAsyncTests.cs New tests for RuntimeNodeSetOptions.ConfigureAsync ordering + generation-owner disposal semantics.
tests/Opc.Ua.Server.Tests/Opc.Ua.Server.Tests.csproj Removes WotCon server project reference.
tests/Opc.Ua.Server.Tests/NodeManager/*Tests.cs Updates/extends adapter delegation + rollback coverage for monitored-item transfers.
tests/Opc.Ua.Server.Tests/Hosting/HostedNodeManagerLifecycleTests.cs Adds ShadowReloadAsync delegation/guard tests.
tests/Opc.Ua.Server.Tests/Hosting/CombinedHostingTests.cs Removes WotCon hosting test.
src/Opc.Ua.Server/Subscription/SubscriptionManager.cs Adds internal test hooks around condition refresh worker shutdown/reset.
src/Opc.Ua.Server/Server/StandardServer.cs Refactors shutdown/disposal orchestration, adds deferred shutdown observer + new lifecycle registration.
src/Opc.Ua.Server/Server/RequestManager.cs Adds lifecycle admission/drain coordination and waiter exclusion support.
src/Opc.Ua.Server/Server/RequestManagerLifecycleExtension.cs New request lifecycle coordination helper for shutdown/lifecycle waits.
src/Opc.Ua.Server/RuntimeNodeSet/* Adds ConfigureAsync, lifecycle-from-request option, and shadow/immediate reload extension methods.
src/Opc.Ua.Server/NodeManager/MasterNodeManager.cs Adds monitored-item transfer transaction/rollback and ownership-sensitive dispatch to support shadow-retired generations.
src/Opc.Ua.Server/NodeManager/Lifecycle/* Adds routing table Contains, lifecycle interface surface for shadow/immediate reload, and a committed-reload exception type.
src/Opc.Ua.Server/NodeManager/NodeManager.cs Adds transfer options + rollback hooks to sync/async node managers and adapters.
samples/Quickstarts.Servers/SampleNodeManager/SampleNodeManager.cs Updates sample node manager transfer handling to honor new transfer options and rollback hook.
Suppressed comments (1)

tests/Opc.Ua.Server.Tests/Opc.Ua.Server.Tests.csproj:1

  • This PR removes the WotCon server project reference (and a related hosting test elsewhere), but the PR description focuses on NodeManager shadow/immediate reload and runtime NodeSet helpers. Either update the PR description to explicitly call out the WotCon test/reference removal, or move that change into a separate PR to keep scope aligned.

Comment thread src/Opc.Ua.Server/Server/RequestManagerLifecycleExtension.cs Outdated
Comment thread src/Opc.Ua.Server/NodeManager/MasterNodeManager.cs Outdated
Comment thread src/Opc.Ua.Server/NodeManager/MasterNodeManager.cs Outdated
@marcschier

Copy link
Copy Markdown
Collaborator Author

Pushed commit 0382d0a with the two requested corrections:

  • Reverted NodeStateLookupExtensions.cs to origin/master; git grep -n "FindByDataType" is clean.
  • Restored the RequestManager admission/drain tests and StandardServer shutdown coverage. The imported files did not reference the dropped retirement APIs, so no retirement tests were removed.

Validation: net10.0 server build passed, net10.0 server tests passed (3927 passed / 9 skipped). net48 server tests built and full-suite run had only the known DurableDataValueQueueVerifyReferenceBatchingAsync local file-lock flake; rerun in isolation passed (1 passed / 1 skipped).

@marcschier

Copy link
Copy Markdown
Collaborator Author

Flagging a genuine regression on this branch that I have not yet fixed, with the evidence so far.

Symptom

NodeManagerLifecycleNamespaceRefreshTests.LiveNodeManagerAddRefreshesTheClientNamespaceTable
fails with System.TimeoutException after 60s. The test waits for a ModelChanged event with
NamespaceTableRefreshed == true following AddRuntimeNodeSetAsync; the event never arrives.

It is not a flake, and not pre-existing

Run Result
origin/master (f0ab787), full suite, net10.0 3889 passed, 0 failed
this branch merged with master, full suite 3982 passed, 1 failed
repeated locally same failure, identical
Azure DevOps build 16182 same failure, identical counts
the same test in isolation passes in 8s

Three reproductions, and the master baseline is clean - so this is caused by this PR, not by the
environment and not by an existing defect.

It is the code, not the added tests

This PR adds only two test files (RuntimeNodeSet/RuntimeNodeSetConfigureAsyncTests.cs and
RuntimeNodeSet/RuntimeNodeSetLifecycleExtensionsTests.cs). Excluding both still reproduces the
failure (3961 passed, 1 failed), so the added tests are not poisoning it.

Running only Opc.Ua.Server.Tests.NodeManager (314 tests, including the failing one) passes, so
something in the wider suite leaves state that this branch is sensitive to and master is not. The
fixture is [NonParallelizable], so this is ordering/shared state rather than parallel contention.

Why local validation missed it

The test was added to master by 636912b (#4115, "refresh the client namespace table on model
change"), which post-dates this branch's base. Validating the branch standalone never ran it.

This generalises to the whole stack: PR CI builds refs/pull/N/merge, i.e. head merged with
base, so every branch must be validated merged with current master, not standalone. I have now
merged master into this branch (21b22e9) so CI and local runs agree.

Where I would look next

This PR modifies ModelChangeTracker.cs and IModelChangeTracker.cs, and #4115 modified the same
files to add the namespace-table refresh. The merge succeeded textually with no conflict, which is
exactly the situation where a semantic conflict hides. The refresher wiring does survive the merge
(ModelChangeTracker still takes an INamespaceTableRefresher, and both Session and
ManagedSession still implement it), so the fault is more subtle than a dropped interface - most
likely the refresh path is conditional on state that an earlier test leaves behind.

Not resolving this comment; it is a real blocker for merging this PR.

@marcschier

Copy link
Copy Markdown
Collaborator Author

Correcting and sharpening my earlier analysis of the
LiveNodeManagerAddRefreshesTheClientNamespaceTable failure. One claim I made was too strong.

What I got wrong. I said "it is the code, not the added tests". That inference was based on
excluding this PR's two new test files, but the PR also adds tests inside modified files - even
with those two excluded the run still had 77 more tests than master, so extra load had not actually
been ruled out. The claim was not justified by that experiment.

What has since been established. I re-ran the full suite with
NUnit.NumberOfTestWorkers=1, which removes parallel contention entirely. It still fails, with the
identical signature (3982 passed, 1 failed, the test taking 1m1s). So the failure is ordering or
accumulated state, not load and not parallel contention
.

Consolidated evidence:

Experiment Result
origin/master f0ab787, full suite 3889 passed, 0 failed
this branch + master, full suite (x2 local, x1 ADO 16182) 3982 passed, 1 failed
the test alone passes, 8s
Opc.Ua.Server.Tests.NodeManager only (314 tests) passes
all 18 remaining [NonParallelizable] fixtures + this test (296 tests) passes
ReferenceServerTest + this test (45 tests) passes
full suite, single worker fails, same signature
full suite minus this PR's two new test files fails

So: some fixture that runs earlier in the full suite leaves state this branch is sensitive to and
master is not. It is not any of the non-parallel fixtures, and not ReferenceServerTest.

What is ruled out. The client side is not involved. #4115, which added both this test and the
namespace-refresh feature, is entirely under src/Opc.Ua.Client/, and this branch's client-side
delta against master is now empty - git diff origin/master HEAD -- src/Opc.Ua.Client/ returns
nothing. So the tracker, INamespaceTableRefresher, and the NamespaceTableRefreshed flag are
byte-identical to master. The fault is on the server side, in whether the model change is emitted
in a form the client recognises as needing a namespace-table refresh.

Still open. I have not identified the specific poisoning predecessor. Bisecting further needs a
deterministic-order run with per-test timing; my attempts to capture a TRX for the ordering did not
produce output, and each full run costs roughly eight minutes.

Leaving this unresolved - it remains a blocker for this PR, and the characterisation above should
make the remaining search short for whoever picks it up.

Share ServerInternalData disposal cleanup so sync and async paths leave the same observable state, while async disposal still awaits async subscription managers when available.

Protect synchronous SubscriptionManager disposal with the same worker shutdown and semaphore capture used by async disposal, and make failed transfer rollback preserve unrelated destination publish requests while clearing stale source claims.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
@marcschier

Copy link
Copy Markdown
Collaborator Author

Note on the second CI failure, so it is not confused with the namespace-refresh regression.

Opc.Ua.Security.Certificates.Tests (net10.0) failed in build 16204 with:

##[error]Test host did not exit cleanly and produced no TRX results.

That is a test-host crash, not an assertion failure - no test actually reported a result. This PR
does not touch that area at all:

git diff --stat origin/master origin/marcschier/wot-05-lifecycle \
    -- src/Opc.Ua.Security.Certificates tests/Opc.Ua.Security.Certificates.Tests

returns nothing. So it should clear on the next run.

The only substantive failure on this PR remains
LiveNodeManagerAddRefreshesTheClientNamespaceTable, analysed in the comments above.

@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.31727% with 309 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.91%. Comparing base (8a10139) to head (d3c8030).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
src/Opc.Ua.Server/NodeManager/MasterNodeManager.cs 81.58% 101 Missing and 29 partials ⚠️
src/Opc.Ua.Server/Server/StandardServer.cs 80.74% 52 Missing and 26 partials ⚠️
...Server/RuntimeNodeSet/RuntimeNodeSetNodeManager.cs 57.44% 17 Missing and 3 partials ⚠️
....Server/Server/RequestManagerLifecycleExtension.cs 77.90% 9 Missing and 10 partials ⚠️
...pc.Ua.Server/NodeManager/AsyncCustomNodeManager.cs 21.05% 15 Missing ⚠️
src/Opc.Ua.Server/NodeManager/CustomNodeManager.cs 16.66% 14 Missing and 1 partial ⚠️
.../Opc.Ua.Server/Subscription/SubscriptionManager.cs 0.00% 10 Missing and 2 partials ⚠️
...ver/NodeManager/Adapters/SyncNodeManagerAdapter.cs 56.25% 7 Missing ⚠️
src/Opc.Ua.Server/Server/RequestManager.cs 94.54% 3 Missing and 3 partials ⚠️
...pc.Ua.Server/Hosting/HostedNodeManagerLifecycle.cs 50.00% 2 Missing ⚠️
... and 4 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4147      +/-   ##
==========================================
- Coverage   80.23%   79.91%   -0.33%     
==========================================
  Files        1515     1527      +12     
  Lines      209980   212839    +2859     
  Branches    36213    36693     +480     
==========================================
+ Hits       168479   170090    +1611     
- Misses      28867    29983    +1116     
- Partials    12634    12766     +132     
Files with missing lines Coverage Δ
...er/NodeManager/Adapters/AsyncNodeManagerAdapter.cs 97.18% <100.00%> (+0.15%) ⬆️
...r/NodeManager/Lifecycle/IDynamicNodeManagerHost.cs 100.00% <ø> (ø)
...rver/NodeManager/Lifecycle/NodeManagerLifecycle.cs 79.95% <ø> (+4.82%) ⬆️
...untimeNodeSet/RuntimeNodeSetLifecycleExtensions.cs 100.00% <100.00%> (ø)
....Ua.Server/RuntimeNodeSet/RuntimeNodeSetOptions.cs 100.00% <ø> (ø)
src/Opc.Ua.Server/Subscription/Subscription.cs 92.41% <100.00%> (+0.10%) ⬆️
src/Opc.Ua.Server/NodeManager/INodeManager.cs 87.50% <85.71%> (-12.50%) ⬇️
...r/Lifecycle/NodeManagerReloadCommittedException.cs 66.66% <66.66%> (ø)
...RuntimeNodeSet/RuntimeNodeSetNodeManagerFactory.cs 83.69% <90.90%> (+2.27%) ⬆️
...pc.Ua.Server/Hosting/HostedNodeManagerLifecycle.cs 92.59% <50.00%> (-7.41%) ⬇️
... and 10 more

... and 69 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

An empty collection expression binds to ImmutableDictionary<TKey, TValue> only on
the newer framework, so the registry model compiled on net10.0 but failed on net48
with CS1729 and CS0019. Naming the Empty singleton explicitly works on every target
and matches how the labels defaults are already written.

The ImmutableArray collection expressions elsewhere in the registry are unaffected
and keep their existing form.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
@marcschier

Copy link
Copy Markdown
Collaborator Author

Bisection update on LiveNodeManagerAddRefreshesTheClientNamespaceTable, so this work is not
repeated. No single fixture group reproduces it.

I split Opc.Ua.Server.Tests into disjoint groups and ran each one together with the failing test:

Subset run with the failing test Tests Result
NodeManager namespace 314 pass
all 18 remaining [NonParallelizable] fixtures 296 pass
ReferenceServerTest 45 pass
folders A-L (Alarms...Location) 1164 pass
folders R-S (RateLimiting...Subscription) 189 pass
root fixtures A-M (Aggregate*...MonitoredNode2Tests) 1116 pass
root fixtures N-U (NodeState*...UserManagement*) 713 pass
full suite 3992 fail
full suite, NUnit.NumberOfTestWorkers=1 3992 fail
full suite minus this PR's two new test files 3971 fail

So it is neither parallel contention nor one poisoning predecessor - it only manifests under the
complete ~4000-test run. That points at cumulative state or resource pressure (sockets,
certificate stores, handles) making the 60s wait for the ModelChanged /
NamespaceTableRefreshed event insufficient, rather than a logic error on a specific path.

Worth stressing what this does not change: origin/master runs the same full suite at
3889 passed / 0 failed, so whatever the mechanism, this branch is more sensitive to it than
master is. It remains a real regression to fix, not something to wave through.

Two concrete suggestions for whoever picks this up:

  1. Instrument the server side to log whether the ModelChangeEvent is emitted at all after
    AddRuntimeNodeSetAsync, and whether the client's model-change subscription is active at that
    moment. The failure mode (60s timeout, no event) does not distinguish "never emitted" from
    "emitted before the subscription was ready".
  2. The earlier log line Failed to apply subscription changes; will retry from the client during
    the full run suggests the model-change subscription may not be active when the event fires,
    which would make this a race the full-suite timing exposes rather than a lost notification.

marcschier and others added 4 commits August 1, 2026 08:08
…to marcschier/wot-14-materialization

# Conflicts:
#	src/Opc.Ua.Types/Utils/FileSystem/LocalFileSystem.cs
#	src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
Adds the materialization runtime that turns registered WoT Thing Descriptions into
live OPC UA nodes: a dependency graph over the registered documents, a coordinator
that drives resolution and projection, and a binding runtime that attaches protocol
channels to the projected variables.

Projection runs through the NodeManager lifecycle rather than mutating a running
address space in place, so a refresh either commits a complete new generation or
leaves the previous one untouched. The resolver, converter, contributor and target
variable seams are interfaces so a host can substitute its own document sources.

Also adds the registry NodeManager that exposes the registry itself in the address
space, and the hosting extensions that register the whole runtime with dependency
injection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
Bring the PR14 materialization, dependency graph, binding runtime, registry projection lifecycle, and NodeManager coverage across from the stacked integration branch.

Keep the tests compiling for both net10.0 and net48 by naming ImmutableDictionary.Empty on the binding plan models and updating the memory binding test seam to the current codec resolver contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
marcschier and others added 2 commits August 1, 2026 08:47
…arcschier/wot-16-samples

# Conflicts:
#	src/Opc.Ua.Types/Utils/FileSystem/VirtualFileSystem.cs
The Opc.Ua.WotCon.Bindings library and its test project were added without being
listed in UA.slnx, so the solution build never compiled them and their tests never
ran in CI. Registering both restores that coverage: 310 binding tests now execute.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
marcschier and others added 27 commits August 4, 2026 17:58
## Summary
- add optional WoT Connectivity asset TD mirroring into the WoT
xRegistry
- add DI opt-in via AddWotRegistryBridge and direct
WotConnectivityServerOptions.RegistryBridge fallback
- document default-off, best-effort failure policy and usage
- add NUnit/Moq coverage for create, update, delete, disabled and
failure paths

## Validation
- dotnet build src\\Opc.Ua.WotCon.Server\\Opc.Ua.WotCon.Server.csproj -c
Release -v:m (all TFMs, 0 warnings/errors)
- dotnet test tests\\Opc.Ua.WotCon.Tests\\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 (0 failed / 983 passed)

---------

Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
…ransfer' into marcschier/wot-09-xregistry-client
…ion (#4156)

Adds the runnable WoT Connectivity samples and completes the
documentation. This is **PR 16 of a
stack** splitting #4093, and the last one.

## Samples

Three projects under `samples/WotCon/`:

* **FlatTagServer** — a plain OPC UA server exposing flat tags, used as
an aggregation source.
* **AggregationServer** — registers Thing Descriptions with the WoT
registry and materializes two
flat sources into a runtime-loaded DI/Machinery/Pumps Pump model. It
also ships a small in-memory
protocol binding, which is the worked example the binding contributor
guide walks through.
* **AggregationClient** — drives the aggregated model: commands,
`Refresh`, monitoring, and
  generation replacement.

## Documentation

* New `docs/WotBindings.md` — the bindings that ship today
(planner/executor architecture, bundled
vs separate packages, operation coverage, target mapping, lazy channels,
generation lifetime) plus
the contributor guide for adding your own, with diagnostics, tests,
packaging, TFM, trimming and
  NativeAOT guidance.
* Expanded `docs/WoTConnectivity.md` covering the registry,
dependency-closure materialization and
  runtime NodeSet projection.
* `docs/RuntimeNodeSets.md` gains the shadow-reload section and
`docs/XRegistry.md` the shared
byte-store section. Both document APIs introduced earlier in the stack;
they are collected here
  because this is the documentation PR.
* `docs/README.md` links the new pages.

## Solution registration

Registers the three samples and `Opc.Ua.WotCon.Samples.Tests` in
`UA.slnx` so CI actually builds
and runs them.

## Stack position

Sits on top of **#4154** (materialization) and additionally merges
**#4142** (registry client) and
**#4144** (protocol executors), because the samples exercise the whole
stack — `AggregationServer`
needs the OPC UA executor and the client sample needs the registry
client. Its diff therefore shows
their content until they land.

It also carries `WotRegistryProjectionLiveTests.cs`, which #4154
deliberately left out: those tests
need both the materialization runtime *and* the registry client, so this
is the first branch where
they compile.

## Validation

* All three samples build clean on every target framework.
* `Opc.Ua.WotCon.Tests`: **830 passed**, 0 failed (769 from #4154 plus
61 from the live projection
  tests).
* `Opc.Ua.WotCon.Samples.Tests`: 6 of 7 pass locally; the remaining one
fails with
`CryptographicException: The system cannot find the path specified`, a
known certificate-store
problem on this machine that was previously reproduced against a clean
baseline worktree and is
  not related to this change. CI will confirm.
* 0 warnings, 0 errors.

One fix was needed while bringing the sample across: its memory binding
called `ResolveCodec` with
the pre-`out WotPayloadDescriptor` signature. The API gained the payload
out-parameter and a
`bool` result during review of the binding PRs, so the sample now
follows the same
`if (!ResolveCodec(...)) return Unsupported(...)` pattern as the shipped
planners.

Integration PR: #4093.
…pace (#4154)

PR 14 of the stacked series splitting the large WoT Connectivity
integration PR #4093.

This PR adds the server-side materialization runtime that turns WoT
Thing Description resources from the xRegistry snapshot into OPC UA
address-space projections. The coordinator builds the dependency closure
for each resource, plans binding metadata, activates binding channels,
and publishes the resulting projection through the registry NodeManager
so refresh, retire, and reload operations remain observable and
deterministic.

Projection intentionally goes through the NodeManager lifecycle reload
API instead of mutating live nodes directly. That keeps complex type
reload, runtime NodeSet publication, shadow reload, immediate reload,
and retirement on the same lifecycle path as other dynamic server
address-space changes.

This branch has a triple dependency. It is based on #4146 (registry),
and additionally needs #4147 (lifecycle reload API) and #4128
(2-argument TryGetStructure). Both #4147 and #4128 are merged into this
branch, so this PR's diff will show their content until those
dependencies land.

Validation performed:
- dotnet build src\Opc.Ua.WotCon.Server\Opc.Ua.WotCon.Server.csproj -c
Release -f net10.0 -v:m
- dotnet build tests\Opc.Ua.WotCon.Tests\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 -v:m
- dotnet test tests\Opc.Ua.WotCon.Tests\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 --no-build
- dotnet build tests\Opc.Ua.WotCon.Tests\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net48 -v:m
- dotnet test tests\Opc.Ua.WotCon.Tests\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net48 --no-build
- dotnet build tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 -v:m
- dotnet test tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 --no-build (one known inherited
failure: LiveNodeManagerAddRefreshesTheClientNamespaceTable)
PR 15 of a stacked split of integration PR #4093 into independently
reviewable pieces.

This branch is stacked on #4139 (`marcschier/wot-10-wotcon-model`) and
additionally has #4128 (`marcschier/wot-01-core-types`) and #4130
(`marcschier/wot-09-xregistry-client`) merged in so this client slice
can build.

Those extra dependencies are intentional:
- Without #4130: `WotRegistryClient.cs:83` — `error CS1729:
'XRegistryClient' does not contain a constructor that takes 4
arguments`.
- Without #4128: `WotRegistryClient.cs:346,375` and
`WotRegistryResourceClient.cs:144` — `error CS1501: No overload for
method 'TryGetStructure' takes 2 arguments`.

The extra diff from #4128 and #4130 disappears once those PRs land.

This PR adds the WoT Connectivity registry client surface for browsing
and reading registry groups, resources and versions, uploading documents
through FileTransfer, invoking registry methods, and registering the
client through DI/builder helpers.
Adds the WoT Connectivity registry service and stores for Thing
Description and Thing Model documents.

This is PR 13 of the stacked split of integration PR #4093 and is
stacked on #4140 (`marcschier/wot-11-bindings-core`). The
materialization runtime that consumes this registry follows separately.

The registry has no dependency on the binding runtime or the NodeManager
lifecycle; the stacking is only because this change shares the WoT
Connectivity server project file with the preceding PRs.

Validation:
- `dotnet build src\Opc.Ua.WotCon.Server\Opc.Ua.WotCon.Server.csproj -c
Release -f net10.0 -v:m`
- `dotnet build tests\Opc.Ua.WotCon.Tests\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 -v:m`
- `dotnet test tests\Opc.Ua.WotCon.Tests\Opc.Ua.WotCon.Tests.csproj -c
Release -p:CustomTestTarget=net10.0 --no-build -v:q`
- `dotnet build src\Opc.Ua.WotCon.Server\Opc.Ua.WotCon.Server.csproj -c
Release -v:m`
## Summary
- Adds the concrete WoT binding transport executors for HTTP, Modbus
TCP, OPC UA and MQTT.
- Includes executor/channel tests and in-process HTTP, Modbus, MQTT and
OPC UA harness coverage.
- Leaves CoAP, BACnet, PROFINET and LoRaWAN planner-only because this
stack split intentionally lands validation/planning before executable
transports.

## Stack context
This is PR 12 of the stacked split of integration PR #4093. It is
stacked on #4140, where the protocol-agnostic planner layer landed
separately, so this PR is limited to the concrete transport
implementations.

## Validation
- `dotnet build
tests\Opc.Ua.WotCon.Bindings.Tests\Opc.Ua.WotCon.Bindings.Tests.csproj
-c Release -p:CustomTestTarget=net10.0 -v:m`
- `dotnet test
tests\Opc.Ua.WotCon.Bindings.Tests\Opc.Ua.WotCon.Bindings.Tests.csproj
-c Release -p:CustomTestTarget=net10.0 --no-build` (512 passed)
- `dotnet build src\Opc.Ua.WotCon.Bindings\Opc.Ua.WotCon.Bindings.csproj
-c Release -v:m`
- `dotnet build
src\Opc.Ua.WotCon.Bindings.Mqtt\Opc.Ua.WotCon.Bindings.Mqtt.csproj -c
Release -v:m`
## Summary

PR 11 of the stacked split for integration PR #4093. This adds the
protocol-agnostic core of `Opc.Ua.WotCon.Bindings`: binder, planner,
executor and channel-factory contracts; payload codec and credential
provider seams; the binding plan model; polling subscriptions for forms
without native observation; and DI/builder registration helpers.

This PR is stacked on #4139, which itself depends on #4132 and #4134.

## Scope

The included planners validate forms and produce binding plans without
transport I/O:

- HTTP planner and validation only; the concrete HTTP executor follows
in the next PR.
- MQTT planner and validation only; the separate MQTT executor package
follows in the next PR.
- Modbus TCP planner and shared Modbus type/limit definitions only; the
Modbus executor and TCP client follow in the next PR.
- OPC UA planner and validation only; the concrete OPC UA executor
follows in the next PR.
- CoAP, BACnet, PROFINET and LoRaWAN are planner/validation-only in this
split.

Keeping concrete transport executors out of this PR makes the
abstractions and planning layer reviewable without a network stack.

## Validation

- `dotnet build
tests\Opc.Ua.WotCon.Bindings.Tests\Opc.Ua.WotCon.Bindings.Tests.csproj
-c Release -p:CustomTestTarget=net10.0 -v:m`
- `dotnet test
tests\Opc.Ua.WotCon.Bindings.Tests\Opc.Ua.WotCon.Bindings.Tests.csproj
-c Release -p:CustomTestTarget=net10.0 --no-build`
- `dotnet build src\Opc.Ua.WotCon.Bindings\Opc.Ua.WotCon.Bindings.csproj
-c Release -v:m`
This is PR 10 of the stacked split of integration PR #4093 into
independently reviewable pieces.

Stack/dependency shape:

- This branch is stacked on #4134 (`marcschier/wot-04-generator-wot`).
- It also requires #4132 (`marcschier/wot-02-sourcegen`), which is
merged into this branch so the model can build.

The #4132 dependency is functional, not cosmetic. Without #4132,
building the WoT Connectivity model on #4134 alone fails with duplicate
generated declarations, for example:

```text
CS0102: The type 'Methods' already contains a definition for 'CreateAssetMethodType'
```

That occurs because the combined WoT-Con NodeSet ships the 1.02
`CreateAssetMethodType` node explicitly; without #4132's
NodeSet-to-ModelDesign fix, the generator synthesizes a second
declaration with the same name.

Once #4132 and #4134 land, the extra diff from those dependencies
disappears and this PR reduces to the three WoT Connectivity model
files:

- `src\Opc.Ua.WotCon\Design\Opc.Ua.WotCon.NodeSet2.xml`
- `src\Opc.Ua.WotCon\Design\Opc.Ua.WotCon.NodeSet2.csv`
- `src\Opc.Ua.WotCon\Opc.Ua.WotCon.csproj`

The generated C# is produced from the pinned NodeSet2 at build time. I
verified it builds from clean after deleting the WotCon `bin`/`obj`, the
source-generation tool `bin`/`obj` directories, and shutting down dotnet
build servers.
This extracts the xRegistry client generalisation from #4093 as PR 9 of
the stacked split into independently reviewable pieces.

The change lets the xRegistry client base accept an explicit registry
root NodeId while preserving the existing well-known-root default.
GenericXRegistryClient exposes the same explicit-root path so callers
can drive registries whose root Object is discovered separately, and the
tests cover the supplied-root and fallback behavior across the lifecycle
helpers.

This PR is independent of the other stack PRs and contains no WoT
registry implementation or Opc.Ua.WotCon.* references. A later PR adds
the WoT registry client on top of this general xRegistry client surface.
…4150)

This is PR 7 of the stacked series splitting integration PR #4093,
stacked on #4147 (`marcschier/wot-05-lifecycle`).

This PR makes subscription transfer between sessions transactional:
subscriptions are prepared before any ownership move, publish-queue
transfer claims block stale source-session publishing, and rollback
restores monitored-item resend-data trigger state.

`ServerInternalData` also gains asynchronous disposal so the
asynchronously disposable subscription manager is shut down without
blocking, with sync and async dispose paths guarded so repeated disposal
is a no-op.

It also brings the transfer-focused coverage for subscription transfer,
publish-queue transfer claims, and `ServerInternalData`
async/double-disposal behavior.

The immediate monitored-item retirement mechanism present in #4093 is
deliberately excluded per maintainer decision. `IRetirableMonitoredItem`
and `INodeManagerMonitoredItemRetirementTracker` do not appear in this
PR.
@marcschier
marcschier merged commit f229529 into marcschier/wot-04-generator-wot Aug 5, 2026
marcschier added a commit that referenced this pull request Aug 5, 2026
…ation, Server and xRegistry work it builds on (#4128)

## Summary

Adds OPC UA WoT Connectivity 1.1 to the stack, together with the Types,
source generation, Server and xRegistry work it is built on. This branch
is the merge point for the fourteen-PR stack that replaced integration
PR #4093; every constituent PR was reviewed and approved separately and
is listed below.

414 files changed, +108,936 / -3,199. Roughly 46k added lines of product
code, 57k of tests, plus samples, tools and documentation.

## Types

- Complete `ExtensionObject` raw-body decoding for binary, XML and JSON
bodies by resolving the concrete type through the message context's
encodeable factory, and simplify the `Variant` structure helpers by
delegating to it (#4128).
- Add `IAtomicFileReplace` as an optional `IFileSystem` capability so
existing external implementations keep working, with atomic publish for
`LocalFileSystem` via `File.Replace`/`File.Move` and for
`VirtualFileSystem` by re-keying the in-memory entry (#4128).
- Add lossless conversion between WoT documents and NodeSet2
(`Opc.Ua.Wot.WotNodeSetConverter`). A byte-exact `uav:nodeSet` envelope
is preserved when requested, the structured `uav:nodes` projection is
used when the readable vocabulary is incomplete, and NodeSet2 is
otherwise synthesized from readable WoT terms. Unmapped JSON members
survive a round trip as pointer-addressed residue (#4131).

## Source generation

- Improve the NodeSet to ModelDesign conversion and the generated node
state (#4132).
- Generate OPC UA models directly from WoT Thing Description files
(#4134).

## Server

- Add live NodeManager shadow and immediate reload. A reload
materializes a new generation beside the active one and switches
atomically; the superseded generation is retired gracefully so existing
MonitoredItems keep being served until they drain, or immediately when
the caller asks for it. Includes the request admission and drain
machinery that lets an orderly shutdown wait for admitted requests
instead of tearing down underneath them (#4147).
- Make subscription transfer between sessions transactional, so a failed
transfer leaves neither session holding a partially moved subscription
(#4150).

## xRegistry

- Allow clients to use explicit registry roots rather than assuming a
well-known location (#4130).

## WoT Connectivity

- Add the WoT Connectivity 1.1 information model: a registry-first
revision layered on the abstract xRegistry base model, incorporating the
complete published OPC 10100-1 v1.02 surface as deprecated nodes in the
same namespace so existing 1.02 clients keep working (#4139).
- Add the protocol binding abstractions and planners that compile WoT
forms into executable plans (#4140).
- Add the HTTP, Modbus, OPC UA and MQTT binding executors (#4144).
- Add the registry and its stores, including document versioning,
validation, dependency resolution and refresh (#4146).
- Add the registry client (#4142).
- Materialize WoT Thing Descriptions into the server address space,
deriving types from Thing Models and instances from Thing Descriptions
(#4154).
- Restore the WoT asset registry bridge so the deprecated 1.02 asset
surface is backed by the registry (#4171).
- Add the WoT Connectivity samples and complete the documentation
(#4156), and restore the remaining coverage and docs (#4172).

## New projects

`src/Opc.Ua.WotCon.Bindings`, `src/Opc.Ua.WotCon.Bindings.Mqtt`,
`tests/Opc.Ua.WotCon.Bindings.Tests`,
`tests/Opc.Ua.WotCon.Samples.Tests`, and the `samples/WotCon`
AggregationServer, AggregationClient and FlatTagServer samples.

## Constituent pull requests

Merged into this branch top-down, each independently approved:

| PR | Title |
| --- | --- |
| #4128 | [Types] Complete ExtensionObject decoding and add an atomic
file replace capability |
| #4131 | [Types] Add lossless conversion between WoT documents and
NodeSet2 |
| #4132 | [SourceGeneration] Improve NodeSet to ModelDesign conversion
and generated node state |
| #4134 | [SourceGeneration] Generate OPC UA models from WoT Thing
Description files |
| #4147 | [Server] Add live NodeManager shadow and immediate reload |
| #4150 | [Server] Make subscription transfer between sessions
transactional |
| #4130 | [XRegistry] Allow clients to use explicit registry roots |
| #4139 | [WotCon] Add the WoT Connectivity 1.1 information model |
| #4140 | [WotCon] Add the WoT protocol binding abstractions and
planners |
| #4144 | [WotCon] Add the HTTP, Modbus, OPC UA and MQTT binding
executors |
| #4146 | [WotCon] Add the WoT Connectivity registry and its stores |
| #4142 | [WotCon] Add the WoT Connectivity registry client |
| #4154 | [WotCon] Materialize WoT Thing Descriptions into the server
address space |
| #4156 | [WotCon] Add the WoT Connectivity samples and complete the
documentation |
| #4171 | Restore WoT asset registry bridge |
| #4172 | Restore WoT coverage and docs |

Supersedes #4093.

## Follow-up

The WoT Connectivity and WoT Binding drafts have moved to 1.1-draft2
since this work was authored. Aligning with that revision - the removed
group vocabulary, the new projection/View construct, the remaining model
vocabulary terms and the two new portable-identity validation rules - is
tracked separately and will follow in its own pull request.

## Validation

Built for every target framework with zero warnings and zero errors, and
validated at the tip of the stack with `Opc.Ua.WotCon.Tests` at 999
passed / 0 failed and `Opc.Ua.Server.Tests` at 4024 passed / 0 failed.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready Ready to merge once CI Passes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants