[3.14] gh-152235: Defer GC tracking in more set operations (gh-152273)

[3.14] gh-152235: Defer GC tracking in more set operations (gh-152273) #121705