[3.15] gh-152235: Defer GC tracking in more set operations (gh-152273)

[3.15] gh-152235: Defer GC tracking in more set operations (gh-152273) #121703