Question about SwinUNETR

[chaoscls]

https://github.com/Project-MONAI/MONAI/blob/ab800d8413df5680161ea00fb3d6c1a7aa8dd04b/monai/networks/nets/swin_unetr.py#L304
Why isn't a encoder layer after hidden_states_out[3] like the other hidden outputs?

[tangy5]

Hi @chaoscls , thanks for the interests of the work. The hidden out 3 is a design option, the goal is to keep 4 times of downsampling to avoid the model to be too large and complex. You could freely reformat the hidden to 5 times downsamples and connect all hidden layers to skip connection and outputs. That will increase the # of parameter by a bit. From our observation, the difference is minimal. Thank you.

[denproc]

Hi @chaoscls and @tangy5,

I have been looking into the SwinUNETR implementation and ran into the same confusion regarding the missing encoder block.
Assuming the standard 4 downsampling stages, the absence of a residual block before concatenation for hidden_states_out[3] makes it inconsistent with all other hidden outputs and with the architecture introduced in the paper. It is the only stage whose skip connection bypasses the ResBlock entirely (see the screenshot attached, where missing block marked with red X).

While the use_v2 option adds additional blocks within the SwinTransformer encoder stages, these operate on the features before the Swin stages rather than on the skip connection paths. So hidden_states_out[3] remains the only branch that differs from both the original SwinUNETR and SwinUNETRv2 architectures as described in the papers.

I'm more than happy to help make a PR. Please, let me know if you'd like me to go ahead with the fix.
Thank you!