UnetBasicBlock does not use bias for convolutions

[function2-llx]

I find that the implementation of UnetBasicBlock does not use bias for its convolutions. E.g., it uses the default value for bias (by not specifying it) :

MONAI/monai/networks/blocks/dynunet_block.py

Lines 142 to 152 in 00a6434

	self.conv1 = get_conv_layer(
	spatial_dims,
	in_channels,
	out_channels,
	kernel_size=kernel_size,
	stride=stride,
	dropout=dropout,
	act=None,
	norm=None,
	conv_only=False,
	)

while the default value of bias is False according to the definition of the get_conv_layer function.

MONAI/monai/networks/blocks/dynunet_block.py

Lines 267 to 277 in 00a6434

	def get_conv_layer(
	spatial_dims: int,
	in_channels: int,
	out_channels: int,
	kernel_size: Union[Sequence[int], int] = 3,
	stride: Union[Sequence[int], int] = 1,
	act: Optional[Union[Tuple, str]] = Act.PRELU,
	norm: Optional[Union[Tuple, str]] = Norm.INSTANCE,
	dropout: Optional[Union[Tuple, str, float]] = None,
	bias: bool = False,
	conv_only: bool = True,

However, the nnU-Net (the model that DynUNet is based on) uses bias by default: https://github.com/MIC-DKFZ/nnUNet/blob/aa53b3b87130ad78f0a28e6169a83215d708d659/nnunet/network_architecture/generic_UNet.py#L112-L113

I'm also not able to find anything mentioning this behavior in the paper Optimized U-Net for Brain Tumor Segmentation (by searching text bias).

On the other hand, however, the implementation of most of the convolution-based models (e.g., ResNet, UperNet) do not use bias.

Could anyone please give any explanation on this? Thanks!

[function2-llx]

Find some plausible explanation from the Internet: there's usually a normalization layer after the convolution, which has a bias term.

[wyli]

yes, and the learnable parameters may or may not be enabled by default (for example in batchnorm vs instancenorm), please see #5222 pytorch/pytorch#22755

[function2-llx]

Thanks for the additional information. I never knew of such an issue with InstanceNorm, though I've used it for a long time.