Utilities for object detection

[old-school-kid]

Some ready to use utilities that could be useful during an object detection application:

• Label Encoder to transform raw labels into targets for training
• Anchor Box generation
• Conversion of anchor box from [xmin, ymin, xmax, ymax] to [x, y, width, height] and vice versa
• Visualization of Anchor Box
• Anchor optimization for custom data set using MetaAnchor (citations 100) or KNN means

Not including other utilities like non max suppression and generating bounding box since they are available under tf.image.

[bhack]

We could also double check what useful bbox ops could be copied/refactored from:

https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/box_ops.py

[old-school-kid]

We could also double check what useful bbox ops could be copied/refactored from:

https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/box_ops.py

Thank you @bhack . Skimmed through the code and the first three tasks can be refactored from the code. Moreover top_k_boxes and bbox_matching from the code could prove to be useful too.

[bhack]

Others are in:

https://github.com/tensorflow/models/tree/master/official/vision/utils/object_detection

[old-school-kid]

Others are in:

https://github.com/tensorflow/models/tree/master/official/vision/utils/object_detection

Ah, visualzation also checks out. Do we have any anchor box optiization tools already available too?

[bhack]

For the generation?
You could see:
https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor_generator.py
https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor.py

[innat]

Weighted box fusion, an alternate of non-max impression.

• paper
• official code
• tf code

[old-school-kid]

For the generation? You could see: https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor_generator.py https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor.py

I was talking about optimization of the hyperparameters that doesnt come integrated with tensorflow object detection API. This repo uses K means whereas MetaAnchor is SOTA afaik.

[bhack]

Ah ok.. with enough resources we could review also this. I suppose with that current PR/maintainer ratio it will be still not a problem.
When this repo will have more popularity probably and PR are going to have lags in the review probably we could try to figure out some priorities.

In that case we could evaluate how much coverage we want to have to all the features related to anchor based models:
Bridging the Gap Between Anchor-based and Anchor-free Detection via
Adaptive Training Sample Selection(CVPR 2020 - citations >300)

[LukeWood]

Some ready to use utilities that could be useful during an object detection application:

• Label Encoder to transform raw labels into targets for training
• Anchor Box generation
• Conversion of anchor box from [xmin, ymin, xmax, ymax] to [x, y, width, height] and vice versa
• Anchor optimization for custom data set using MetaAnchor (citations 100) or KNN means
• Visualization of Anchor Box

Not including other utilities like non max suppression and generating bounding box since they are available under tf.image.

We could also consider adding some sort of a non max suppression layer. The object detection API will definitely take a lot of design.

[old-school-kid]

The object detection API will definitely take a lot of design.

Yeah, agreed. Should we open a discussion for this too?

We could also consider adding some sort of a non max suppression layer.

Yeah, @innat provided a nice such example above.

[LukeWood]

I’m also considering offering a bounding box extension type:

https://www.tensorflow.org/guide/extension_type

I’m not sure how useful this would be though compared to just using -1 padded tensors or ragged tensors.

[LukeWood]

The object detection API will definitely take a lot of design.

Yeah, agreed. Should we open a discussion for this too?

We could also consider adding some sort of a non max suppression layer.

Yeah, @innat provided a nice such example above.

Are non-admin users able to open discussions right now?

[bhack]

Yes but admin can convert also an issue to a discussion:
https://docs.github.com/en/discussions/managing-discussions-for-your-community/moderating-discussions#converting-an-issue-to-a-discussion

[old-school-kid]

Are we adding losses again over here like GIoU and family of IoU loss here? Since I see issues like SmoothL1boundingBoxLoss and IoU loss and IoU is already available as metric in core keras.

[innat]

Here is a post-processing technique for object detection (and instance segmentation). Officially it supports PyTorch ecosystem.

• SAHI: Slicing Aided Hyper Inference

(Issue: slow process - need to vectorize the process.)

[LukeWood]

Here is a post-processing technique for object detection (and instance segmentation). Officially it supports PyTorch ecosystem.

• SAHI: Slicing Aided Hyper Inference

(Issue: slow process - need to vectorize the process.)

We should open a bug for this. This is really useful!

[innat]

@LukeWood did you mean to open a new issue?

[innat]

Drawing bounding box on a batch of images.

https://www.tensorflow.org/io/api_docs/python/tfio/experimental/image/draw_bounding_boxes

[bhack]

Drawing bounding box on a batch of images.

https://www.tensorflow.org/io/api_docs/python/tfio/experimental/image/draw_bounding_boxes

This was implemented as a native op. How we re-implement this in an efficient way in a python only repository using only compositional TF ops

/cc @yongtang

[innat]

There is another method: https://www.tensorflow.org/api_docs/python/tf/image/draw_bounding_boxes
cc. @bhack @yongtang

[innat]

The YOLO-V4 introduces many tactics to improve OD model performance. A medium blog author (Jonathan Hui) made a nice summary of it. It's a good read: https://medium.com/p/c9901eaa8e61

• Bag of Freebies (BoF) for backbone
  • CutMix and Mosaic data augmentation,
  • DropBlock regularization, and
  • Class label smoothing
• Bag of Specials (BoS) for backbone
  • Mish activation,
  • Cross-stage partial connections (CSP), and
  • Multi-input weighted residual connections (MiWRC)
• Bag of Freebies (BoF) for detector
• Bag of Specials (BoS) for detector

[jbischof]

We still need anchor optimization for sure, but closing for now in favor of newer discussions.