Add C# lock-free shared-memory store profile (v2.0.0)

.github/workflows/ci.yml

@@ -42,7 +42,10 @@ jobs:
         run: dotnet format SharedMemoryStore.slnx --verify-no-changes --no-restore
 
       - name: Test
-        run: dotnet test SharedMemoryStore.slnx -c Release --no-build
+        # Keep the projects sequential: individual integration scenarios retain
+        # their real 8-12 process concurrency without unrelated test assemblies
+        # oversubscribing the hosted runner's one-second cold-open budget.
+        run: dotnet test SharedMemoryStore.slnx -c Release --no-build --maxcpucount:1
 
       - name: Validate documentation
         shell: pwsh

.specify/feature.json

@@ -1,3 +1,3 @@
 {
-  "feature_directory": "specs/008-cpp-python-implementations"
+  "feature_directory": "specs/009-lock-free-publish-read"
 }

AGENTS.md

@@ -1,5 +1,5 @@
 <!-- SPECKIT START -->
 For additional context about technologies to be used, project structure,
 shell commands, and other important information, read the current plan
-at specs/008-cpp-python-implementations/plan.md
+at specs/009-lock-free-publish-read/plan.md
 <!-- SPECKIT END -->

CHANGELOG.md

@@ -5,6 +5,37 @@ chronological order.
 
 ## Unreleased
 
+## 2.0.0 - 2026-07-13
+
+### Added
+
+- Added an explicitly selected C# lock-free key-value profile using mapped
+  layout `2.0` and resource protocol `2`, with direct reservation publication,
+  shared zero-copy leases, generation-fenced removal/reuse, participant
+  incarnations, explicit recovery, and bounded diagnostics.
+- Added `StoreProfile`, `CreateLockFree`, profile-aware sizing,
+  `ParticipantRecordCount`, immutable `StoreProtocolInfo`, and the appended
+  `ParticipantTableFull` open status.
+
+### Changed
+
+- NuGet `SharedMemoryStore` advances to `2.0.0`. Lock-free wait options bound
+  local retry, revalidation, helping, and backoff; they do not acquire a named
+  operation lock or wait for keys/capacity to appear.
+- Documented reservation tokens as exclusive single-producer lifetimes and
+  clarified logical removal, `RemovePending`, borrowed-view, recovery, and
+  local-handle disposal semantics.
+
+### Compatibility
+
+- The legacy profile remains the default and preserves layout `1.2`, resource
+  protocol `1`, public workflow signatures, and existing status numbers.
+- Layout `1.2` and `2.0` are never reinterpreted in place or used by mixed
+  synchronization participants. Same-name upgrade and rollback require draining
+  handles, recreating the mapping, and republishing application-owned values.
+- C++ and Python `0.1.0` remain layout-1.2-only and reject layout `2.0`; their
+  independently versioned packages and C ABI `1.0` are unchanged.
+
 ## 1.0.2 - 2026-07-10
 
 ### Added

README.md

@@ -8,17 +8,21 @@ payloads through a broker process.
 
 Distribution identities:
 
-- NuGet: `SharedMemoryStore` `1.0.2`, targeting `net10.0` with .NET BCL
-  runtime dependencies only.
+- NuGet: `SharedMemoryStore` `2.0.0`, targeting `net10.0` with .NET BCL
+  runtime dependencies only. The legacy layout remains the default; C# callers
+  opt in to the lock-free layout explicitly.
 - CMake: `SharedMemoryStore` `0.1.0`, exposing a C++20 RAII API and fixed-width
   C ABI `1.0` over the native shared library.
 - Python: `shared-memory-store` `0.1.0`, requiring Python 3.10 or newer and
   using standard-library `ctypes` with the packaged native library.
-- Shared protocol: mapped layout `1.2`, resource naming `1`, little-endian
-  64-bit Windows and Linux targets.
+- Shared protocol: C# supports mapped layouts `1.2` and `2.0`; C++ and Python
+  remain layout-`1.2` clients and reject layout 2.0. Resource naming versions
+  are `1` and `2` respectively.
 - License: MIT, see the [license file](LICENSE).
 
-The managed `1.0.0` line establishes the production .NET API contract. The
+The managed `1.0.0` line established the original production .NET API contract;
+the `2.0.0` line adds the explicit lock-free profile while retaining the legacy
+profile and its layout. The
 native and Python `0.1.0` lines are independently versioned initial
 distributions; they do not change or ship inside the NuGet package. Linux and
 Windows are implementation targets. Same-host Linux Docker containers require
@@ -56,6 +60,41 @@ The store does not parse frame headers, own application schemas, provide a
 cross-host cache, persist data beyond process and mapping lifetime, or turn
 Docker into distributed storage.
 
+## Explicit lock-free C# profile
+
+`SharedMemoryStoreOptions.CreateLockFree(...)` creates or opens mapped layout
+2.0. Existing helpers and manually constructed options remain on the legacy
+layout unless `StoreProfile.LockFree` is selected explicitly. Processes using
+different profiles cannot participate in the same live mapping; incompatible
+opens fail before payload projection.
+
+The lock-free profile is still a bounded key-value store. An application broker
+may load-balance work by sending keys to 6-12 workers, while observers and other
+processes independently acquire the same immutable values. The store does not
+enqueue keys, choose workers, acknowledge work, or turn a read lease into an
+exclusive claim. See the runnable
+[`LockFreeBrokerKeys`](samples/LockFreeBrokerKeys/Program.cs) sample.
+
+Steady-state layout-2.0 publish, acquire, release, remove, and helping paths do
+not enter the named cross-process lifecycle lock. Progress is system-wide
+lock-free, not wait-free: a paused or terminated participant cannot own a
+store-wide data-path lock, but one caller can still exhaust its bounded retry
+budget under sustained contention and receive `StoreBusy`. Reservations belong
+to one producer; leases are shared exact-generation protections; recovery is an
+explicit caller-controlled operation rather than a background worker.
+
+The trust boundary is same-host cooperating processes with write access to the
+mapping. Layout 2.0 is neither cross-host storage nor protection against a
+malicious mapped writer. Performance depends on payload size, key distribution,
+contention, lease duration, capacity pressure, process placement, and recovery;
+qualification reports separate those workloads instead of treating one
+throughput number as universal.
+
+To migrate under the same public name, drain and close every legacy handle,
+recreate the mapping as layout 2.0, and republish application-owned values. A
+side-by-side cutover uses a distinct store name. Rollback recreates layout 1.2;
+neither direction reinterprets mapped bytes in place.
+
 ## First Use
 
 Start with [Getting started](docs/getting-started.md). It separates the NuGet,
@@ -205,6 +244,7 @@ Runnable samples:
 - [Docker shared-memory sample](samples/DockerSharedMemory/README.md)
 - [C++ basic usage sample](samples/CppBasicUsage/README.md)
 - [Python basic usage sample](samples/PythonBasicUsage/README.md)
+- [Lock-free broker-key sample](samples/LockFreeBrokerKeys/Program.cs)
 
 ## Project Policies
 
@@ -248,5 +288,5 @@ pwsh ./scripts/validate-interoperability.ps1 -Configuration Release -Stress
 
 Documentation changes must keep package metadata, README content, release notes,
 support policy, security policy, compatibility metadata, and contract links
-aligned across managed `1.0.2`, native `0.1.0`, Python `0.1.0`, ABI `1.0`, and
-layout `1.2`.
+aligned across managed `2.0.0`, native `0.1.0`, Python `0.1.0`, ABI `1.0`, and
+layouts `1.2` and `2.0`.

SharedMemoryStore.slnx

@@ -6,12 +6,14 @@
   </Configurations>
   <Folder Name="/benchmarks/">
     <Project Path="benchmarks/SharedMemoryStore.Benchmarks/SharedMemoryStore.Benchmarks.csproj" />
+    <Project Path="benchmarks/SharedMemoryStore.SyncProbe/SharedMemoryStore.SyncProbe.csproj" />
   </Folder>
   <Folder Name="/samples/">
     <Project Path="samples/BasicUsage/BasicUsage.csproj" />
     <Project Path="samples/DockerSharedMemory/DockerSharedMemory.csproj" />
     <Project Path="samples/FrameValue/FrameValue.csproj" />
     <Project Path="samples/HostedServiceIntegration/HostedServiceIntegration.csproj" />
+    <Project Path="samples/LockFreeBrokerKeys/LockFreeBrokerKeys.csproj" />
     <Project Path="samples/ZeroCopyIngest/ZeroCopyIngest.csproj" />
   </Folder>
   <Folder Name="/src/">
@@ -23,6 +25,8 @@
     <Project Path="tests/SharedMemoryStore.InteropAgent/SharedMemoryStore.InteropAgent.csproj" />
     <Project Path="tests/SharedMemoryStore.InteropTests/SharedMemoryStore.InteropTests.csproj" />
     <Project Path="tests/SharedMemoryStore.LeaseOwnerTool/SharedMemoryStore.LeaseOwnerTool.csproj" />
+    <Project Path="tests/SharedMemoryStore.LinearizabilityTests/SharedMemoryStore.LinearizabilityTests.csproj" />
+    <Project Path="tests/SharedMemoryStore.LockFreeAgent/SharedMemoryStore.LockFreeAgent.csproj" />
     <Project Path="tests/SharedMemoryStore.UnitTests/SharedMemoryStore.UnitTests.csproj" />
   </Folder>
 </Solution>

benchmarks/SharedMemoryStore.Benchmarks/LockFreeProfileBenchmarks.cs

@@ -0,0 +1,194 @@
+using BenchmarkDotNet.Attributes;
+
+using Store = SharedMemoryStore.MemoryStore;
+
+namespace SharedMemoryStore.Benchmarks;
+
+/// <summary>
+/// Side-by-side profile benchmarks for the v1.2 synchronized engine and the
+/// v2 lock-free engine. Keys and buffers are prepared during setup so the
+/// MemoryDiagnoser reports library-path allocations rather than harness noise.
+/// </summary>
+[MemoryDiagnoser]
+public class LockFreeProfileBenchmarks
+{
+    private const int SlotCount = 64;
+    private const int PayloadBytes = 256;
+    private const int PrimaryBucketCount = 64;
+
+    private Store _store = null!;
+    private byte[][] _keys = null!;
+    private byte[][] _collidingKeys = null!;
+    private byte[] _payload = null!;
+    private int _cursor;
+
+    [Params(StoreProfile.Legacy, StoreProfile.LockFree)]
+    public StoreProfile Profile { get; set; }
+
+    [GlobalSetup]
+    public void Setup()
+    {
+        string name = $"sms-profile-benchmark-{Profile}-{Guid.NewGuid():N}";
+        SharedMemoryStoreOptions options = Profile == StoreProfile.LockFree
+            ? SharedMemoryStoreOptions.CreateLockFree(
+                name,
+                SlotCount,
+                PayloadBytes,
+                maxDescriptorBytes: 8,
+                maxKeyBytes: 8,
+                leaseRecordCount: 128,
+                participantRecordCount: 16,
+                openMode: OpenMode.CreateNew,
+                enableLeaseRecovery: true)
+            : SharedMemoryStoreOptions.Create(
+                name,
+                SlotCount,
+                PayloadBytes,
+                maxDescriptorBytes: 8,
+                maxKeyBytes: 8,
+                leaseRecordCount: 128,
+                openMode: OpenMode.CreateNew,
+                enableLeaseRecovery: true);
+
+        StoreOpenStatus open = Store.TryCreateOrOpen(options, out Store? store);
+        if (open != StoreOpenStatus.Success || store is null)
+        {
+            throw new InvalidOperationException($"Cannot open {Profile} benchmark store: {open}.");
+        }
+
+        _store = store;
+        _keys = Enumerable.Range(0, SlotCount).Select(BitConverter.GetBytes).ToArray();
+        _payload = new byte[PayloadBytes];
+        for (var index = 0; index < _payload.Length; index++)
+        {
+            _payload[index] = unchecked((byte)(index * 31));
+        }
+
+        _collidingKeys = FindCollidingKeys(9);
+        for (var index = 0; index < 8; index++)
+        {
+            Ensure(_store.TryPublish(_collidingKeys[index], _payload));
+        }
+
+        Ensure(_store.TryPublish(_keys[0], _payload));
+    }
+
+    [GlobalCleanup]
+    public void Cleanup()
+    {
+        _store.Dispose();
+    }
+
+    [Benchmark(Baseline = true)]
+    public StoreStatus AcquireProjectRelease()
+    {
+        StoreStatus acquire = _store.TryAcquire(_keys[0], out ValueLease lease);
+        if (acquire != StoreStatus.Success)
+        {
+            return acquire;
+        }
+
+        _ = lease.ValueSpan[PayloadBytes - 1];
+        return lease.Release();
+    }
+
+    [Benchmark]
+    public StoreStatus PublishRemove()
+    {
+        int index = 1 + (Interlocked.Increment(ref _cursor) & 31);
+        byte[] key = _keys[index];
+        StoreStatus publish = _store.TryPublish(key, _payload);
+        return publish == StoreStatus.Success ? _store.TryRemove(key) : publish;
+    }
+
+    [Benchmark]
+    public StoreStatus ZeroCopyReserveCommitRemove()
+    {
+        int index = 33 + (Interlocked.Increment(ref _cursor) & 15);
+        byte[] key = _keys[index];
+        StoreStatus reserve = _store.TryReserve(key, PayloadBytes, [], out ValueReservation reservation);
+        if (reserve != StoreStatus.Success)
+        {
+            return reserve;
+        }
+
+        _payload.CopyTo(reservation.GetSpan());
+        StoreStatus advance = reservation.Advance(PayloadBytes);
+        if (advance != StoreStatus.Success)
+        {
+            _ = reservation.Abort();
+            return advance;
+        }
+
+        StoreStatus commit = reservation.Commit();
+        return commit == StoreStatus.Success ? _store.TryRemove(key) : commit;
+    }
+
+    [Benchmark]
+    public StoreStatus CollisionOverflowPublishRemove()
+    {
+        byte[] key = _collidingKeys[8];
+        StoreStatus publish = _store.TryPublish(key, _payload);
+        return publish == StoreStatus.Success ? _store.TryRemove(key) : publish;
+    }
+
+    [Benchmark]
+    public StoreStatus CurrentOwnerLeaseRecovery()
+    {
+        int index = 49 + (Interlocked.Increment(ref _cursor) & 7);
+        byte[] key = _keys[index];
+        StoreStatus publish = _store.TryPublish(key, _payload);
+        if (publish != StoreStatus.Success)
+        {
+            return publish;
+        }
+
+        StoreStatus acquire = _store.TryAcquire(key, out _);
+        if (acquire != StoreStatus.Success)
+        {
+            _ = _store.TryRemove(key);
+            return acquire;
+        }
+
+        StoreStatus recovery = _store.TryRecoverLeases(
+            new LeaseRecoveryOptions(RecoverCurrentProcessLeases: true),
+            out _);
+        StoreStatus remove = _store.TryRemove(key);
+        return recovery == StoreStatus.Success ? remove : recovery;
+    }
+
+    private static byte[][] FindCollidingKeys(int count)
+    {
+        var keys = new List<byte[]>(count);
+        for (long candidate = 0; keys.Count < count; candidate++)
+        {
+            byte[] key = BitConverter.GetBytes(candidate);
+            if ((Hash(key) & (PrimaryBucketCount - 1)) == 0)
+            {
+                keys.Add(key);
+            }
+        }
+
+        return keys.ToArray();
+    }
+
+    private static ulong Hash(ReadOnlySpan<byte> key)
+    {
+        ulong hash = 14_695_981_039_346_656_037UL;
+        foreach (byte value in key)
+        {
+            hash ^= value;
+            hash *= 1_099_511_628_211UL;
+        }
+
+        return hash;
+    }
+
+    private static void Ensure(StoreStatus status)
+    {
+        if (status != StoreStatus.Success)
+        {
+            throw new InvalidOperationException("Benchmark setup failed: " + status);
+        }
+    }
+}

benchmarks/SharedMemoryStore.Benchmarks/Program.cs

@@ -47,7 +47,10 @@
     return;
 }
 
-BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
+// Assembly discovery intentionally includes LockFreeProfileBenchmarks alongside
+// the existing reliability suites; use --filter *LockFreeProfileBenchmarks* for
+// the v1.2-versus-v2 allocation/collision/recovery comparison.
+BenchmarkSwitcher.FromAssembly(typeof(LockFreeProfileBenchmarks).Assembly).Run(args);
 
 static FrameThroughputValidationResult RunSimplePublishSustained()
 {