Skip to content

Remove unsafe code from CborHelpers, Crc32, and BlobUtilities#126269

Closed
EgorBo wants to merge 7 commits intomainfrom
unsafe-code-removal
Closed

Remove unsafe code from CborHelpers, Crc32, and BlobUtilities#126269
EgorBo wants to merge 7 commits intomainfrom
unsafe-code-removal

Conversation

@EgorBo
Copy link
Copy Markdown
Member

@EgorBo EgorBo commented Mar 29, 2026

Note

This PR was generated with the assistance of GitHub Copilot.

Summary

Replace Unsafe.* and MemoryMarshal.* API usage with safe BinaryPrimitives equivalents in three libraries:

System.Formats.Cbor — CborHelpers.netstandard.cs

  • ReadHalfBigEndian: MemoryMarshal.Read + manual endian swap → BinaryPrimitives.ReadUInt16BigEndian
  • WriteHalfBigEndian: MemoryMarshal.Write + manual endian swap → BinaryPrimitives.WriteUInt16BigEndian
  • ReadDoubleBigEndian: MemoryMarshal.Read + ReverseEndianness → BinaryPrimitives.ReadInt64BigEndian
  • WriteDoubleBigEndian: MemoryMarshal.Write + ReverseEndianness → BinaryPrimitives.WriteInt64BigEndian

System.IO.Hashing — Crc32ParameterSet.WellKnown.cs

  • UpdateScalarArm64, UpdateScalarArm: Unsafe.ReadUnaligned → BinaryPrimitives.ReadUInt64/32LittleEndian
  • UpdateIntrinsic (SSE path): MemoryMarshal.Cast<byte, ulong/uint> → explicit for-loops with BinaryPrimitives
  • UpdateIntrinsic (ARM path): Unsafe.ReadUnaligned → BinaryPrimitives.ReadUInt64/32LittleEndian
  • Removed MemoryMarshal.GetReference + Unsafe.Add patterns
  • Simplified remainder-byte loops

System.Reflection.Metadata — BlobUtilities.cs

  • WriteUInt16/32/64 and BE variants: Unsafe.WriteUnaligned → BinaryPrimitives.Write*LittleEndian/BigEndian
  • WriteDouble (#else branch): pointer cast *(ulong*)&valueBitConverter.DoubleToInt64Bits

All changes are behavioral equivalents — no public API changes. All existing tests pass.

@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces several Unsafe.* / MemoryMarshal.*-based unaligned read/write patterns with System.Buffers.Binary.BinaryPrimitives (and BitConverter bit-conversion helpers) to keep the implementations safe while preserving behavior.

Changes:

  • System.Reflection.Metadata: replace unaligned integer writes with BinaryPrimitives.Write*Endian and remove unsafe double bit reinterpretation in WriteDouble (non-NET path).
  • System.IO.Hashing: replace Unsafe.ReadUnaligned / MemoryMarshal.Cast patterns in CRC32(C) intrinsic/scalar paths with BinaryPrimitives.Read*LittleEndian.
  • System.Formats.Cbor: simplify big-endian half/double read/write helpers using BinaryPrimitives.Read/Write*BigEndian.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/libraries/System.Reflection.Metadata/src/System/Reflection/Internal/Utilities/BlobUtilities.cs Switches unaligned integer writes to BinaryPrimitives and uses BitConverter.DoubleToInt64Bits for safe double bit extraction in non-NET builds.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.WellKnown.cs Removes Unsafe/MemoryMarshal usage in CRC32(C) update loops, using BinaryPrimitives reads instead.
src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.netstandard.cs Replaces endian-swap + MemoryMarshal patterns for half/double big-endian helpers with BinaryPrimitives calls.

@github-actions

This comment has been minimized.

@EgorBo EgorBo marked this pull request as draft March 29, 2026 00:21
- Revert Crc32ParameterSet.WellKnown.cs changes (bounds check concern)
- BlobUtilities: use BitConverter.DoubleToInt64Bits on all TFMs
- CborHelpers: remove ReadDoubleBigEndian/WriteDoubleBigEndian wrappers,
  inline BinaryPrimitives calls at call sites with #if NET polyfill.
  Remove WriteHalfBigEndian wrapper, call BinaryPrimitives directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 29, 2026 13:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.netstandard.cs:104

  • PR description indicates CborHelpers replaces Unsafe.* and MemoryMarshal.* usage with BinaryPrimitives equivalents, but CborHelpers.netstandard.cs still uses MemoryMarshal.Read/Write (e.g., ReadSingleBigEndian/WriteSingleBigEndian) and contains unsafe pointer casts for float bit conversions. If the PR goal is to fully remove these APIs/unsafe code from CborHelpers, consider updating the remaining helpers (where possible for the target TFMs) or clarifying the scope in the PR description.
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static ushort ReadHalfBigEndian(ReadOnlySpan<byte> source)
        {
            return BinaryPrimitives.ReadUInt16BigEndian(source);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static float ReadSingleBigEndian(ReadOnlySpan<byte> source)
        {
            return BitConverter.IsLittleEndian ?
                Int32BitsToSingle(BinaryPrimitives.ReverseEndianness(MemoryMarshal.Read<int>(source))) :
                MemoryMarshal.Read<float>(source);
        }

@github-actions

This comment has been minimized.

@@ -135,28 +117,6 @@ public static void WriteSingleBigEndian(Span<byte> destination, float value)
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
extension(BinaryPrimitives)
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static double ReadDoubleBigEndian(ReadOnlySpan<byte> source)
{
return BitConverter.Int64BitsToDouble(BinaryPrimitives.ReadInt64BigEndian(source);
}
}

Can we convert all polyfills in this type to extension methods like this, so that the actual source can use all modern APIs directly without unnecessary wrappers?

EnsureWriteCapacity(1 + sizeof(double));
WriteInitialByte(new CborInitialByte(CborMajorType.Simple, CborAdditionalInfo.Additional64BitData));
CborHelpers.WriteDoubleBigEndian(_buffer.AsSpan(_offset), value);
BinaryPrimitives.WriteInt64BigEndian(_buffer.AsSpan(_offset), BitConverter.DoubleToInt64Bits(value));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BinaryPrimitives.WriteInt64BigEndian(_buffer.AsSpan(_offset), BitConverter.DoubleToInt64Bits(value));
BinaryPrimitives.WriteDoubleBigEndian(_buffer.AsSpan(_offset), value);

Address jkotas feedback: convert CborHelpers Single/Double wrappers
to extension(BinaryPrimitives) polyfills so callers can use
BinaryPrimitives.Read/WriteSingle/DoubleBigEndian directly on all TFMs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@@ -27,21 +27,5 @@ public static string BuildStringFromIndefiniteLengthTextString<TState>(int lengt
[MethodImpl(MethodImplOptions.AggressiveInlining)]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we go further and delete this file completely?

Comment on lines +130 to +132
return BitConverter.IsLittleEndian ?
CborHelpers.Int32BitsToSingle(BinaryPrimitives.ReverseEndianness(MemoryMarshal.Read<int>(source))) :
MemoryMarshal.Read<float>(source);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return BitConverter.IsLittleEndian ?
CborHelpers.Int32BitsToSingle(BinaryPrimitives.ReverseEndianness(MemoryMarshal.Read<int>(source))) :
MemoryMarshal.Read<float>(source);
return CborHelpers.Int32BitsToSingle(BinaryPrimitives.ReadInt32BigEndian(source));

public static void WriteSingleBigEndian(Span<byte> destination, float value)
{
MemoryMarshal.Write(destination, ref value);
if (BitConverter.IsLittleEndian)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Copy link
Copy Markdown
Member

@jkotas jkotas Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BinaryPrimitives.WriteSingleLittleEndian(buffer.AsSpan(start), value);

MemoryMarshal.Write(destination, ref value);
}
}
internal static uint SingleToUInt32Bits(float value)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline the unsafe casts into the BinaryPrimitives extensions directly?

@github-actions

This comment has been minimized.

…ves directly

- CborHelpers: simplify ReadSingleBigEndian/WriteSingleBigEndian extensions
  to use ReadInt32BigEndian/WriteInt32BigEndian with inlined unsafe casts,
  removing MemoryMarshal.Read/Write and ReverseEndianness
- BlobUtilities: use BinaryPrimitives.WriteSingle/DoubleLittleEndian on NET,
  keep existing fallback on netstandard2.0/net462

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 29, 2026 15:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

@github-actions

This comment has been minimized.

Move all TFM-specific CborHelpers methods into one file with #if NET
guards. Delete the netcoreapp-only partial class file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
namespace System.Formats.Cbor
{
internal static partial class CborHelpers
internal static class CborHelpers
Copy link
Copy Markdown
Member

@jkotas jkotas Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not mean to add a bunch of ifdefs to the common file.

I meant to:

  • Delete CborHelpers.netcoreapp.cs
  • Use the netcoreapp APIs directly throughout the code
  • Add extensions to CborHelpers.netstandard.cs to polyfill the netcoreapp APIs

Copy link
Copy Markdown
Member Author

@EgorBo EgorBo Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I did have that in mind, in my workflow I just ask AI to address the feedback and then change it. It sometimes proactively pushes it before I change it, but I assumed it's fine since the PR is in draft 🙂 Still, I appreciate the early feedback!

@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings March 29, 2026 16:28
…n APIs on NET

- Add extension(BitConverter) polyfills for SingleToInt32Bits,
  Int32BitsToSingle, SingleToUInt32Bits, UInt32BitsToSingle on netstandard
- Update HalfHelpers and CborWriter.Simple to use BitConverter.* directly
- Keep CborHelpers wrappers for UnixEpoch, BigInteger, decimal.GetBits,
  string.Create (different signatures per TFM, not pure polyfills)
- Guard netstandard-only polyfills with #if !NET, add NET versions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.cs:11

  • using System.Buffers; and using System.Runtime.InteropServices; are not used anywhere in this file, which will produce unused-using warnings (treated as errors in this repo). Please remove the unused using directives.
    src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.cs:179
  • The BinaryPrimitivesPolyfills / BitConverterPolyfills blocks introduce new unsafe code and rely on extension(Type) syntax. This undermines the PR goal of removing unsafe code, and the extension-member syntax is not used elsewhere in the repo (making it risky/unclear for multi-target builds). Since BinaryPrimitives.Read/WriteSingle/Double*Endian already exist in System.Memory (used in netstandard builds), please remove these polyfills and call the existing BinaryPrimitives APIs directly; for bit conversions, keep the previous internal helper methods instead of extending BitConverter.

Comment on lines 39 to 41
result = HalfHelpers.FloatToHalf(value);
return float.IsNaN(value) || CborHelpers.SingleToInt32Bits(HalfHelpers.HalfToFloat(result)) == CborHelpers.SingleToInt32Bits(value);
return float.IsNaN(value) || BitConverter.SingleToInt32Bits(HalfHelpers.HalfToFloat(result)) == BitConverter.SingleToInt32Bits(value);
}
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On non-.NETCoreApp targets (netstandard2.0 / .NET Framework), BitConverter.SingleToInt32Bits is not available (the test helper for netstandard uses an unsafe bitcast for this reason). This change will break compilation unless you provide a valid polyfill; consider restoring the previous internal helper (e.g., CborHelpers.SingleToInt32Bits) or comparing bit patterns via BinaryPrimitives-based conversions instead.

Copilot uses AI. Check for mistakes.
Comment on lines 56 to 66
@@ -62,7 +62,7 @@ public static float HalfToFloat(ushort value)
return CreateSingle(sign, (byte)(exp + 0x70), sig << 13);

static float CreateSingle(bool sign, byte exp, uint sig)
=> CborHelpers.Int32BitsToSingle((int)(((sign ? 1U : 0U) << FloatSignShift) + ((uint)exp << FloatExponentShift) + sig));
=> BitConverter.Int32BitsToSingle((int)(((sign ? 1U : 0U) << FloatSignShift) + ((uint)exp << FloatExponentShift) + sig));
}
Copy link

Copilot AI Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BitConverter.UInt32BitsToSingle, BitConverter.Int32BitsToSingle, and BitConverter.SingleToUInt32Bits are not available on the netstandard2.0 / .NET Framework TFMs that this file targets, so these replacements will fail to compile without a robust polyfill. Please keep using the existing internal conversion helpers (previously on CborHelpers) or implement local bit-conversion helpers for these TFMs.

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🤖 Copilot Code Review — PR #126269

Note

This review was generated by GitHub Copilot and has not been manually verified.

Holistic Assessment

Motivation: Well-justified. Replacing manual Unsafe.WriteUnaligned / MemoryMarshal.Read/Write + endian-swap patterns with BinaryPrimitives calls is the right direction. The reduce-unsafe label is appropriate. The Crc32 revert after stephentoub/jkotas feedback about bounds-check overhead was a good call.

Approach: Clean. The C# 14 extension(BinaryPrimitives) / extension(BitConverter) pattern for netstandard polyfills is exactly what jkotas requested — callers use the modern APIs directly (BinaryPrimitives.ReadDoubleBigEndian, BitConverter.SingleToInt32Bits) and the polyfills transparently fill the gap for older TFMs. Merging the two partial class files into a single CborHelpers.cs with #if NET / #else blocks is a net simplification.

Summary: ✅ LGTM. The changes are semantically equivalent, correctly constrained to internal helpers, and address all maintainer review feedback. No blocking issues found. Two minor observations below.


Detailed Findings

✅ Correctness — All BinaryPrimitives replacements are semantically equivalent

Verified each transformation:

  • CborHelpers: MemoryMarshal.Read<ushort> + ReverseEndiannessBinaryPrimitives.ReadUInt16BigEndian — equivalent. All four Read/Write Half/Single/Double helpers are correct.
  • BlobUtilities: Unsafe.WriteUnaligned(ref buffer[start], endian-swapped-value)BinaryPrimitives.Write*LittleEndian/BigEndian(buffer.AsSpan(start), value) — equivalent and strictly safer (the old Unsafe.WriteUnaligned could silently write past array bounds if start was near the end).
  • BlobUtilities WriteDouble #else: *(ulong*)&valueunchecked((ulong)BitConverter.DoubleToInt64Bits(value)) — bit-preserving. DoubleToInt64Bits is available on netstandard2.0.
  • HalfHelpers.netstandard.cs: CborHelpers.UInt32BitsToSingle(...)BitConverter.UInt32BitsToSingle(...) — resolves to the polyfill extension; correct.
  • CborWriter.Simple.netstandard.cs: CborHelpers.SingleToInt32Bits(...)BitConverter.SingleToInt32Bits(...) — resolves to the polyfill extension; correct.

✅ Extension polyfill design — Correct and consistent with maintainer guidance

The BinaryPrimitivesPolyfills and BitConverterPolyfills classes using extension(BinaryPrimitives) / extension(BitConverter) are properly guarded by #if !NET, so they only compile for netstandard2.0 / .NET Framework where the real APIs are missing. On modern .NET, the actual BCL methods are used directly with no wrapper overhead.

The remaining unsafe pointer casts in the polyfills (*(float*)&intValue, *(int*)&value) are necessary for netstandard2.0 where BitConverter.SingleToInt32Bits / Int32BitsToSingle don't exist. This is the right tradeoff — unsafe code is isolated to polyfills that only compile on older TFMs.

✅ File consolidation — CborHelpers merge is correct

Deleting CborHelpers.netcoreapp.cs and merging into CborHelpers.cs is clean. The .csproj correctly moves the <Compile> include from the TFM-conditional ItemGroup to the unconditional one. The class is no longer partial since there's only one file, which is correct.

✅ No public API surface changes

All changed types are internal. No ref/ assembly changes. No new public members.

💡 Unused using directives — Minor cleanup

using System.Runtime.InteropServices; appears unused in both CborHelpers.cs (line 11) and BlobUtilities.cs (line 8) now that all MemoryMarshal / Unsafe.WriteUnaligned calls have been removed. CI/analyzers should catch this, but noting it in case they don't.

Generated by Code Review for issue #126269 ·

@EgorBo
Copy link
Copy Markdown
Member Author

EgorBo commented Mar 29, 2026

I'll split this PR into smaller ones since it now does more than I initially wanted (to remove unsafe) also it is getting harder to review. Also, I should probably rely less on AI on addressing feedback 😥

@EgorBo EgorBo closed this Mar 29, 2026
EgorBo added a commit that referenced this pull request Mar 30, 2026
> [!NOTE]
> This PR description was AI/Copilot-generated.

Extracted from #126269

I wonder if we should move all Polyfils for NS/NETF to just one common
place for all libraries. Currently, libs just define their own in-place.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants