Utilities for object detection

[old-school-kid]

Some ready to use utilities that could be useful during an object detection application:

• Label Encoder to transform raw labels into targets for training
• Anchor Box generation
• Conversion of anchor box from [xmin, ymin, xmax, ymax] to [x, y, width, height] and vice versa
• Visualization of Anchor Box
• Anchor optimization for custom data set using MetaAnchor (citations 100) or KNN means

Not including other utilities like non max suppression and generating bounding box since they are available under tf.image.

[bhack]

We could also double check what useful bbox ops could be copied/refactored from:

https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/box_ops.py

[old-school-kid]

We could also double check what useful bbox ops could be copied/refactored from:

https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/box_ops.py

Thank you @bhack . Skimmed through the code and the first three tasks can be refactored from the code. Moreover top_k_boxes and bbox_matching from the code could prove to be useful too.

[bhack]

Others are in:

https://github.com/tensorflow/models/tree/master/official/vision/utils/object_detection

[old-school-kid]

Others are in:

https://github.com/tensorflow/models/tree/master/official/vision/utils/object_detection

Ah, visualzation also checks out. Do we have any anchor box optiization tools already available too?

[bhack]

For the generation?
You could see:
https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor_generator.py
https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor.py

[innat]

Weighted box fusion, an alternate of non-max impression.

• paper
• official code
• tf code

[old-school-kid]

For the generation? You could see: https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor_generator.py https://github.com/tensorflow/models/blob/master/official/vision/beta/ops/anchor.py

I was talking about optimization of the hyperparameters that doesnt come integrated with tensorflow object detection API. This repo uses K means whereas MetaAnchor is SOTA afaik.

[bhack]

Ah ok.. with enough resources we could review also this. I suppose with that current PR/maintainer ratio it will be still not a problem.
When this repo will have more popularity probably and PR are going to have lags in the review probably we could try to figure out some priorities.

In that case we could evaluate how much coverage we want to have to all the features related to anchor based models:
Bridging the Gap Between Anchor-based and Anchor-free Detection via
Adaptive Training Sample Selection(CVPR 2020 - citations >300)

[LukeWood]

Some ready to use utilities that could be useful during an object detection application:

• Label Encoder to transform raw labels into targets for training
• Anchor Box generation
• Conversion of anchor box from [xmin, ymin, xmax, ymax] to [x, y, width, height] and vice versa
• Anchor optimization for custom data set using MetaAnchor (citations 100) or KNN means
• Visualization of Anchor Box

Not including other utilities like non max suppression and generating bounding box since they are available under tf.image.

We could also consider adding some sort of a non max suppression layer. The object detection API will definitely take a lot of design.

[old-school-kid]

The object detection API will definitely take a lot of design.

Yeah, agreed. Should we open a discussion for this too?

We could also consider adding some sort of a non max suppression layer.

Yeah, @innat provided a nice such example above.

[LukeWood]

I’m also considering offering a bounding box extension type:

https://www.tensorflow.org/guide/extension_type

I’m not sure how useful this would be though compared to just using -1 padded tensors or ragged tensors.

[LukeWood]

The object detection API will definitely take a lot of design.

Yeah, agreed. Should we open a discussion for this too?

We could also consider adding some sort of a non max suppression layer.

Yeah, @innat provided a nice such example above.

Are non-admin users able to open discussions right now?

[bhack]

Yes but admin can convert also an issue to a discussion:
https://docs.github.com/en/discussions/managing-discussions-for-your-community/moderating-discussions#converting-an-issue-to-a-discussion

[innat]

@LukeWood could you please convert this issue to the discussion too? I think it's better suited there.