MaskFormerFusionHead - semantic_postprocess is missing

[susaha]

The semantic_postprocess function has not been implemented yet in the MaskFormerFusionHead class. Could you please provide the code or share a link to a reference code?

[susaha]

I have implemented the function, and the visualizations look reasonable to me, please cross check once:
def semantic_postprocess(self, mask_cls, mask_pred, meta):
"""
# K: num classes, N: num queries, H: hieght, W: width
num_query, num_classes = mask_cls.shape # [N, K]
num_classes = num_classes - 1
mask_cls = mask_cls[:, :num_classes]
_, imgH, imgW = mask_pred.shape # [N, H, W]
mask_cls = F.softmax(mask_cls, dim=-1) # convert logits to softmax prob., softamx over K classes
mask_cls = mask_cls.permute(1,0) # [N, K] -> [K, N] : permute matrix for matrix multiplication
mask_pred = mask_pred.sigmoid() # converts logits to score between 0 and 1
mask_pred = mask_pred.view(num_query,-1) # [N, H, W] -> [N, HW] : change view for matrix multiplication
semantic_seg = torch.matmul(mask_cls, mask_pred) # [K, N] x [N, HW] --> [K, HW]
semantic_seg = semantic_seg.view(num_classes, imgH, imgW) # [K, HW] --> [K, H, W]
############
rows, cols = 2, 2
gridspec_kw = {'hspace': 0.1, 'wspace': 0, 'top': 0.95, 'bottom': 0, 'right': 1, 'left': 0}
fig, axs = plt.subplots(rows, cols, figsize=(3 * cols, 3 * rows), gridspec_kw=gridspec_kw)
semseg = semantic_seg.argmax(0)
subplotimg(axs[0][0], semseg, 'SemGT', cmap='cityscapes')
for ax in axs.flat:
ax.axis('off')
out_dir = '....'
outfile = os.path.join(out_dir, meta['ori_filename'])
plt.savefig(outfile)
plt.close()
##########
return semantic_seg

maskFormer prediction visualization:

[ZwwWayne]

It seems correct and the results seem good! Feel free to create a PR to support it. We also recommend you MMSegementation, which will support semantic segmentation of maskformer if you are interested.