How to implement ODISe's prompt embedding technique in this code?

[Bavesh-B]

ODISe uses a CLIP image embedding-based prompt embedding technique. How do I implement that in this code as the conditions are of a specific shape and type in "encoder_hidden_states". It would be nice if I could get some help as i'm not entirely sure on how to implement it.

[stefan-baumann]

We provide a separate branch that integrates our weights with the codebase from GeoAware-SC: https://github.com/CompVis/cleandift/tree/geoaware-sc-eval
This should also include this kind of CLIP conditioning, as their code is based on ODISE. If you're looking to integrate it with our main codebase instead, let us know and we'll see if we can offer some guidance :)

[Bavesh-B]

Thanks for the reply! I’m looking to integrate it with the main codebase as I want to use it for some other task. Is there any more information that I need to provide?