Making a queue private does not remove it from benchmarks

[Didayolo]

See #1012

• Make a queue public
• Another user add the queue to its benchmark as the queue is public
• Make the queue private
• The queue should be back to default in the benchmark

[Didayolo]

Tracking in #1062