Replace MD5-based deterministic GUIDs with XxHash128

[danielmarbach]

Introduces a new DeterministicGuid implementation based on XxHash128 (non-cryptographic, RFC 9562 version 8 GUIDs) and migrates host ID generation and the learning saga persister away from the legacy MD5-based algorithm.

Why move away from MD5?

• Performance: XxHash128 is significantly faster than MD5 for non-cryptographic use cases.
• FIPS compliance: MD5 is a cryptographic hash and can cause FIPSAlgorithmIsSuitedException on systems with FIPS policy enabled (see HostingComponent.Settings.cs). Using a cryptographic hash for something that doesn't require cryptographic properties is incorrect.
• Proper GUID formatting: The new implementation produces version 8 / RFC 9562 variant GUIDs, whereas the legacy MD5 implementation produced unversioned GUIDs without proper variant bits.

Alternatives considered

• Copy the tool into downstream packages: Duplicate the new DeterministicGuid into each package that needs it. Rejected because it creates maintenance burden and divergent implementations.
• Keep DeterministicGuid public in NServiceBus.Core but leave LegacyDeterministicGuid forever: Rejected because keeping MD5 around indefinitely means carrying a FIPS-incompatible, unnecessarily expensive code path that shouldn't be used by anyone.
• Current approach: Introduce the new DeterministicGuid in core, migrate callers where IDs are local/ephemeral (learning persister), use an AppContext switch for host IDs where backward compatibility matters, and mark the legacy path for removal in v12.

New DeterministicGuid API

The NServiceBus.Utils.DeterministicGuid class provides several overloads:

• Create(string) / Create(ReadOnlySpan<char>) — hash the UTF-8 encoding of the string
• Create(ReadOnlySpan<byte>) — hash raw binary data directly
• Create(params ReadOnlySpan<string>) — hash multiple values with length-prefix framing to prevent concatenation ambiguity (e.g., Create("ab", "c") ≠ Create("a", "bc"))

All overloads produce deterministic version 8 GUIDs. The same input always yields the same GUID.

Performance optimizations in Create(params ReadOnlySpan<string>)

• Single allocation: Uses GetMaxByteCount (O(1)) for buffer sizing instead of GetByteCount (O(n)), enabling one ArrayPool.Rent for the entire operation instead of per-value rent/return cycles.
• Single-pass encoding: Each string is encoded once via GetBytes directly into the contiguous buffer — no separate GetByteCount scan.
• Contiguous hashing: Length prefix and encoded bytes are a single contiguous slice, reducing XxHash128.Append calls from two per value to one.
• Hardcoded empty-input result: Create() with no values returns a precomputed static readonly Guid instead of allocating a hash state.

Breaking change: Learning Saga Persister IDs

The LearningSagaIdGenerator now uses DeterministicGuid instead of LegacyDeterministicGuid (MD5). This means saga IDs in the learning persister will change. Existing learning persister saga state files will not be found after upgrading.

This is an acceptable breaking change because:

• The learning persister is not a production persister; it is designed for local development and testing only.
• Learning persister data is ephemeral and not migrated between environments.
• The learning persister directory can be cleaned up between runs during development.

Host ID changes (opt-in via AppContext switch)

Host ID generation (HostingComponent.Settings.GenerateHostId) now defaults to the legacy MD5 algorithm for backward compatibility. Endpoint host IDs will remain unchanged unless the AppContext switch NServiceBus.Core.Hosting.UseV2DeterministicGuid is explicitly set to true.

A PreObsolete annotation has been added to the legacy switch and LegacyDeterministicGuid class, documenting that in v11 the new algorithm becomes the default and in v12 both will be removed.

Technically using PreObsolete is slightly awkward because we will never be turning this into a real obsolete but it seemed still be the most pragmatic choice because we have explicit processes that are supposed to look for PreObsolete attributes.

[ramonsmits]

Host ID generation (HostingComponent.Settings.GenerateHostId) now defaults to the legacy MD5 algorithm for backward compatibility. Endpoint host IDs will remain unchanged unless the AppContext switch NServiceBus.Core.Hosting.UseV2DeterministicGuid is explicitly set to true.

@danielmarbach why not be more explicit with the key its implementation and intend like:

NServiceBus.Core.HostId.UseRfc9562Uuid

AppContext is here used for opt-in which makes sense especially because there are no new APIs but wonder why though. Is this because this should be considered "private" to the platform?

As HostID is not really used by customers and only by our platform the concern that a user did not even consciously opted into this new behavior is not a big risk.

[danielmarbach]

Unfortunately, this is not private to the platform. We have guidance on how to set it over the APIs too, which by default "makes it public". We have looked at this in detail, and the consequence of just flipping it means once you upgrade, you effectively have doubled your number of endpoints. That surprising element plus the support cases that might create in us first thinking something in ServiceControl might be broken is enough reason to make this a more "cautious" change.