Bucketing module after Hybridize ?

[sravanbabuiitm]

How does bucketing via diff padding work seamlessly post hybridize ? In symbolic mxnet, we create separate copies of the model for each bucket size & with gluon, i think we avoid that coz of imperative style. I have noticed models working seamlessly with hybridize as well, without us needing to do anything different as in gluon. How does this work ? what are the internal changes/modifications that are enabling this ?

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Bug

[vdantu]

@sravanbabuiitm : Thats a good question. I would recommend you to post this question on discuss.mxnet.io as well.

@roywei : Would you be able to answer this?

@mxnet-label-bot add [question, gluon]

[szha]

Gluon's hybridize for HybridBlock is backed by CachedOp, a special type of operator that caches the computation graph. It has internal buffers for data and metadata such as shape and type, and is intelligent enough to reallocate bigger buffer if requested space is larger than the previously allocated space. If the model can be expressed in hybridized gluon model, then the space is managed by the cached op.

On the other hand, regular Block can be partially hybridized, which has the effect of recursively hybridizing all HybridBlock children. In this case, the memory in HybridBlock children is handled by CachedOp, and the rest runs in imperative mode.

[sravanbabuiitm]

Thank you!
Does this mean that even without calling hybridize, the children of regular block have this CachedOp ?

And by using hybridize the entire model gets managed this way ?

[szha]

Regular block doesn't have CachedOp, only HybridBlock does. Hybridize call on a regular block recursively calls hybridize on all children, which has the effect of hybridizing all HybridBlock children