Skip to content

Conversation

ziyuan-linn
Copy link
Member

  • add await tf.ready() to handpose
  • add await tf.ready() to pose-detection
  • add tf.setBackend("webgl") to neuralNetwork

Hi @shiffman, I think the problem here is caused by tf.js using webgpu, instead of webgl, as the default backend on some browsers and machines. (I tried running the neural network example in both Chrome and Firefox and the error only occurred in Chrome. Firefox doesn't support webgpu yet.)

I am not exactly sure how the dependencies from pose-detection trigger this error, but I found a solution. We have to call await tf.ready();, more info here, before calling any other tensorflow.js functions because unlike webgl, setting webgpu backend is an asynchronous task.

I have added await tf.ready(); to all other models; however, using the webgpu backend for neuralNetwork caused another problem, which seems like a much deeper issue to resolve:
error
I think the best solution, for now, is to force neuralNetwork to use the original WebGL backend by calling tf.setBackend("webgl");

Just some other random thoughts:

I was switching between the webgl an webgpu backends and found that the frame rate varied depending on the model and the computer being used. I think it might be helpful to add an option argument to tf.js-based models, allowing more advanced users to manually set the backend and fine-tune their projects.

let options = {
  backend: "webgl"
}
let handpose = ml5.handpose(options);

@ziyuan-linn ziyuan-linn requested review from shiffman and gohai August 14, 2023 21:10
@ziyuan-linn ziyuan-linn mentioned this pull request Aug 14, 2023
@shiffman
Copy link
Member

Yes, I agree that an option for setting the back end in ml5 would be great! (though this is something we should try to elegantly handle behind the scenes as much as possible so beginners never have to consider it!).

Merging this!

@shiffman shiffman merged commit 42375ed into main Aug 14, 2023
@ziyuan-linn ziyuan-linn deleted the fix-webgpu-not-initialized branch August 21, 2023 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants