inference speed is very low #91
Description
Sry I found that the inference speed on tesla v100 card is very slow (it takes 30 seconds to generate text responses and masks), would it be much faster if using graphics card that flash attention?
{{ message }}