[Modular] Add Custom Blocks guide to doc #12339
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
looking great! thanks @DN6
Does not need to be in this PR but relevant on the topic:
for the remote custom code, I think maybe it's easier to work without the cache system. e.g. if I want to make my own version of florence node, I would probably want to download the folder locally first (with local_dir_use_symlinks=False) so I can go in and edit the code
let me know what you think about what should be the best process for this?
|
|
||
| In this example, we will create a custom block that uses the Florence 2 model to process an input image and generate a mask for inpainting | ||
|
|
||
| ```py |
There was a problem hiding this comment.
It'd be nice to break up this big code example into several smaller chunks that explain whats happening. Maybe break it up into:
- inputs
- outputs
- computation logic
which also nicely mirrors https://huggingface.co/docs/diffusers/main/en/modular_diffusers/pipeline_block#inputs-and-outputs
There was a problem hiding this comment.
Would explanations using inline comments in the code work? Reason I wanted to keep it as a single block was to make it easier to just copy and use as a template
There was a problem hiding this comment.
oh i see! in that case, feel free to keep it as a single block but i would at least still add some text before it that very briefly explains whats going on. maybe even something like the below should be fine and you won't even need to add inline comments :)
The example below creates a custom block with the Florence 2 model. It accepts several inputs like image, annotation task type, prompt, output type and and outputs a mask image.
| 1. From the CLI | ||
|
|
||
| ```shell | ||
| # In the folder with the `block.py` file, run: | ||
| diffusers-cli custom_block | ||
| ``` | ||
|
|
||
| Then upload the block to the Hub: | ||
|
|
||
| ```shell | ||
| hf upload <your repo id> . . | ||
| ``` | ||
|
|
||
| 2. From Python | ||
|
|
||
| ```py | ||
| from block import Florence2ImageAnnotatorBlock | ||
| block = Florence2ImageAnnotatorBlock() | ||
| block.push_to_hub("<your repo id>") | ||
| ``` |
There was a problem hiding this comment.
Maybe wrap this in <hfoptions>:
<hfoptions id="share">
<hfoption id="hf CLI">
...
</hfoption>
<hfoption id="push_to_hub">
...
</hfoption>
</hfoptions>
@yiyixuxu this is a good point. I can add a section for this. |
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
Co-authored-by: Steven Liu <[email protected]>
4 participants
What does this PR do?
Simple guide for creating custom blocks. Will try to add some more advanced examples in a follow up.
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.