this student made a tensorRT GUI

[gsgoldma]

https://github.com/ddPn08/Lsmith

maybe some intrepid soul knows how to implement this for the humble auto111 webui

[specblades]

We need it

[Albies42]

Holy shit I need this in automatic1111 right now!

[ghost]

I tried to install the gui but all I got was errors on top of errors. My system is pretty standard with Win10 and the overall setup. So far I wasnt able to get it running locally

[ghost]

after changing my workflow with controlnet I will give this a try for prototyping shapes and scenes. I wished all the prompt tweaks and features of A1111 would be there : - (

[ClashSAN]

https://github.com/jonigata/lsmithbridge

Desc:
stable-diffusion-webui extension to bypass txt2img generate to Lsmith running locally. The bypass destination is a URL in the UI, so if you have a public server, you can use it.

[gsgoldma]

I've been trying to get a server running for this for a day but I keep getting errors like this

(base) C:\Users\Gregory\Desktop>python server.py

• Serving Flask app "server" (lazy loading)
• Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
• Debug mode: off
• Running on http://localhost:8000/ (Press CTRL+C to quit)
  127.0.0.1 - - [17/Feb/2023 09:51:50] "←[37mPOST /api/images/generate HTTP/1.1←[0m" 200 -
  127.0.0.1 - - [17/Feb/2023 09:52:47] "←[37mPOST /api/images/generate HTTP/1.1←[0m" 200 -
  127.0.0.1 - - [17/Feb/2023 10:03:49] "←[33mGET / HTTP/1.1←[0m" 404 -
  127.0.0.1 - - [17/Feb/2023 10:03:49] "←[33mGET /favicon.ico HTTP/1.1←[0m" 404 -
  127.0.0.1 - - [17/Feb/2023 10:06:50] "←[37mPOST /api/images/generate HTTP/1.1←[0m" 200 -
  127.0.0.1 - - [17/Feb/2023 10:07:05] "←[37mPOST /api/images/generate HTTP/1.1←[0m" 200 -

maybe my server code is wrong?

from flask import Flask, jsonify, request

app = Flask(name)

@app.route('/api/images/generate', methods=['POST'])
def generate():
if request.method == 'POST':
# assume the request body is JSON
data = request.get_json()
# do some processing with the data here
result = {'success': True}
return jsonify(result)
else:
result = {'success': False}
return jsonify(result)

if name == 'main':
app.run(host='localhost', port=8000)

[78Alpha]

I think this is the 5th tensor RT UI/toolset. All of them suffer from problems making models on Windows or 8 GB cards.

I'll certainly give it a try but I'm expecting it to fail to compile the model like every other one.

[BetaDoggo]

That's because it's impossible to convert SD models to tensorRT with only 8gb of vram.

[78Alpha]

That's because it's impossible to convert SD models to tensorRT with only 8gb of vram.

It is possible to make a model on 8 GB. I got just the official TRT demo to run. It choked on anything that wasn't the base v1.5 model though.

I must add that the images generated by TRT looked horrible. Maybe I'm just not used to seeing the bare 1.5 model anymore.

[minienglish1]

Forgive my ignorance, but what exactly does tensorRT do differently that the current AUTOMATIC1111/stable-diffusion-webui?

[ghost]

Its 20 to 30% faster because it changes the models structure to an optimized state. Thats why its not that easy to integrate it. Also, every card / series needs to accelerate their own models. An accelerated model from a 3060 probably wont run on the 400x series.

[ClashSAN]

20 to 30 % vs --xformers ?
how much vram consumption at minimum?

[minienglish1]

Its 20 to 30% faster because it changes the models structure to an optimized state. Thats why its not that easy to integrate it. Also, every card / series needs to accelerate their own models. An accelerated model from a 3060 probably wont run on the 400x series.

Thanks. That's what I thought. Also looks like you need to convert the models before using them.

[78Alpha]

20 to 30 % vs --xformers ? how much vram consumption at minimum?

For generating the model, 8 GB minimum. You have to generate the model on the card you want to run. They say you can compile on 1 from a family (3090 for example) and run on lower end cards, but everyone has been pulling issues of Polygraphy not being able to do anything with the precompiled models because they were made on a card different from theirs.

The optimizations are also memory dependent. So an 8 GB card will do worse than a 40 GB card even with the same core and clocks, at least, from what they have said.

[KNARZ0R]

anyway I think even 20% faster is worth the effort to integrate this, because on the long run this is a good benefit.
so please dear python magicians, do the magic :D

[Kotori05]

maybe an amateur question, SDwebui did not use tensorcore for running now?

[78Alpha]

maybe an amateur question, SDwebui did not use tensorcore for running now?

It used them before. tensorRT optimizes models heavily (Nvidia only), so it's best to say it uses them "better".

[hnsywangxin]

how to use tensorrt to accelerate controlnet with stable diffusion, refer to this repo:https://github.com/hnsywangxin/controlnet_stable_tensorrt