add UMIC metric to the benchmark #3
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi there,
This PR is for UMIC score. The difference between this PR and the UMIC author are:
faster_rcnn_R_101_C4_3x.yamlwhich produce the visual feature with size of (B, N, 2048) and my work usefaster_rcnn_R_101_FPN_3x.yamlwhich produce the visual feature with size of (B, N, 1024). But the idea is the same, both methods want to represent image with regional image feature. Potential for future work to replicate their work withfaster_rcnn_R_101_C4_3x.yamlconfig.the files
models/uniter/model.pyandmodels/uniter/layer.pyis originated from UNITER repository. I only replace their choice ofFusedLayerNormfrom apex because it was kinda unreasonable, nobody wants to install the whole apex for just a norm layer which can easily imported from torchLayerNormLastly, the instruction on how to install detectron2 is already included in the umic_score.py file, I didnt modify your
requirements.txtto include my detectron2 version because it was a quite tricky process to get it running on my environment. so I recommend to install it separately.Cheers,
Hien Bui