Optimize managed postprocessing and allocations

[ericstj]

Summary

• decode raw model logits instead of computing log-softmax
• use sparse Viterbi predecessor tables and pooled compact backpointers
• pool inference input and score buffers and avoid copying dense ONNX outputs
• add randomized decoder parity coverage and managed/end-to-end benchmarks

Results

• sparse Viterbi is 54% faster with 97% less managed allocation
• end-to-end managed allocation falls 52% for short text and 66% for long text
• model-dominated end-to-end latency remains effectively unchanged

Testing

• dotnet test PrivacyFilter.Net.sln -c Release --nologo

[Copilot]

Pull request overview

This PR optimizes managed postprocessing in PrivacyFilter.Net by reducing allocations and improving decoding throughput (notably Viterbi), while adding tests and benchmarks to validate parity and measure performance.

Changes:

• Switch decoding to operate directly on raw logits (removing the production log-softmax pass) and pool model score buffers.
• Replace dense Viterbi transition scanning with sparse valid-predecessor tables and pooled compact backpointers.
• Add randomized decoder parity tests and expand BenchmarkDotNet harness/bench documentation for managed postprocessing.

Show a summary per file

File	Description
tests/PrivacyFilter.Net.Tests/DecoderTests.cs	Adds randomized parity tests (log-softmax invariance) and a dense Viterbi reference to validate the sparse decoder.
src/PrivacyFilter.Net/ViterbiDecoder.cs	Implements sparse predecessor tables and pooled byte backpointers; updates Decode to accept ReadOnlySpan<float>.
src/PrivacyFilter.Net/PrivacyFilter.cs	Pools score buffers, avoids copying dense ONNX outputs when possible, and changes inference to fill a caller-provided Span<float>.
src/PrivacyFilter.Net/PrivacyFilter.Net.csproj	Exposes internals to the benchmarks project for performance testing.
bench/PrivacyFilter.Net.Benchmarks/Program.cs	Uses BenchmarkSwitcher and adds managed postprocessing benchmarks (log-softmax, dense vs sparse Viterbi, argmax).
bench/results.md	Documents new managed postprocessing benchmark results and allocation/time deltas.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/PrivacyFilter.Net/ViterbiDecoder.cs:96

• tokenCount * classCount is computed without overflow/negative validation. If tokenCount is negative or large enough to overflow, the dimension check can behave incorrectly and later allocations (e.g., new int[tokenCount]) will throw in less clear ways. Consider validating tokenCount >= 0 and using checked for the expected emission length.

        int classCount = _labels.TokenClassNames.Length;
        if (emissions.Length != tokenCount * classCount)
        {
            throw new ArgumentException("Emission dimensions do not match the label space.", nameof(emissions));
        }

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Low