the difference between DETRIS-B and DETRIS-B (Default Setting)

[feng-zitong]

Hi, thanks for the great work on this project!
What's the difference between DETRIS-B and DETRIS-B (Default Setting) in Table 2 of the paper?
Is it due to the different number of adapter layers, or something else?
From the code, I see that the Dense Aligner is applied at layers [1, 3, 5, 7, 9, 11], and the Text Adapter is also applied at [1, 3, 5, 7, 9, 11].

[Kimsure]

no difference. Tab.2 adjust the parameters (decrease dense aligner numbers) for a fair comparison, where the default setting is to remain the same setting in Tab.1

[kkakkkka]

Hello,

Yes, you are exactly right. The difference lies in the number and configuration of the adapter layers.

For the implementation in our code (and the main result in the paper) adopts an Interval (Sparse) Insertion Strategy. As you observed in config/refcoco/DETRIS_base.yaml, the adapters are only applied to the odd-indexed layers:

visual_adapter_layer: [1, 3, 5, 7, 9, 11]
txtual_adapter_layer: [1, 3, 5, 7, 9, 11]

This reduces the number of trainable parameters by half, while maintaining comparable performance.

For the DETRIS-L (Large) model, the configuration in config/refcoco/DETRIS_large.yaml reverts to a denser strategy, because the larger model capacity with more training data benefits more from the dense adaptation in our architecture.

Best regards.