Add opt-in startup diagnostics logging with automatic size optimization

[danielmarbach]

WriteDiagnosticsToLog() allows opting in to diagnostics logging instead of file output. When enabled, diagnostics are automatically compacted to prevent Application Insights telemetry limits from being exceeded.

Changes

1. Opt-in API: WriteDiagnosticsToLog() replaces DisableWritingDiagnosticsToLog()
2. Writer Suppression: When logging is enabled, the file writer (default or custom) is not called
3. Automatic Compaction: AssemblyScanning section is automatically compacted to prevent exceeding Application Insights telemetry limits (~30KB threshold)

Diagnostic Size

Our test diagnostic analysis revealed critical size issues:

Large endpoints (e.g., NServiceBus.Core.Tests):

• Total size: 64.5 KB (exceeds Application Insights 64KB limit!)
• AssemblyScanning: 55.4 KB (86% of payload)
• Everything else: ~9 KB (14% of payload)

Typical endpoints (acceptance tests):

• Size: 12-16 KB (safe for Application Insights)

This analysis showed that AssemblyScanning alone can exceed telemetry limits on complex endpoints, making automatic compaction essential for reliable diagnostics logging.

Large endpoints: Diagnostics can exceed 64KB (Application Insights limit), with AssemblyScanning alone consuming ~55KB

Solution: When logging to Application Insights, AssemblyScanning is automatically compacted:

• Before: Full assembly list (~55KB)
• After: AssemblyCount, SkippedFileCount, ErrorsThrownDuringScanning, Settings (~100 bytes)

Implementation: AssemblyScanningDiagnostics.CreateCompactedVersion() centralizes the compaction logic

Why

1. Predictable: Always compact when logging - no threshold guessing
2. Safe: Prevents Application Insights telemetry limits from being exceeded
3. Type-safe: Strongly-typed DTO prevents runtime type errors
4. Encapsulated: Compaction logic lives with the data it compacts
5. Backward compatible: Existing file-only behavior is unchanged (opt-in feature)