Rotation and stretch issues while loading the video using createVideoTexture #4278

Anburaj · 2023-05-12T04:08:32Z

Anburaj
May 12, 2023

I am trying to load the video using createVideoTexture. Using the below code, I can able to load the video successfully. But video rotation is upside down and it is stretched to fit the entire texture size.

<!doctype html>
<html lang="en">
  <head>
    <title>&lt;model-viewer&gt; template</title>
    <meta charset="utf-8">
    <meta name="description" content="&lt;model-viewer&gt; template">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <link type="text/css" href="./styles.css" rel="stylesheet"/>
    <script async src="https://ga.jspm.io/npm:[email protected]/dist/es-module-shims.js">
      
    </script>

<script type="importmap-shim">
{
  "imports": {
    "three": "https://ajax.googleapis.com/ajax/libs/model-viewer/3.0.1/model-viewer.min.js"
  }
}
</script>
<script type="module" 
src="https://ajax.googleapis.com/ajax/libs/model-viewer/3.0.1/model-viewer.min.js">
</script>
  </head>
  <body>
    <model-viewer id="animated" 
    camera-controls 
    touch-action="pan-y" 
    src="sample.glb" 
    ar >
    </model-viewer>
    <script type="module">
    const modelViewerAnimated = document.querySelector("model-viewer#animated");
    let videoTexture = null;
    customElements.whenDefined('model-viewer').then(() => {
      videoTexture = modelViewerAnimated.createVideoTexture("bird.mp4");
    });
    modelViewerAnimated.addEventListener("load", async () => {
    
      const material = modelViewerAnimated.model.materials[8];
      const {baseColorTexture} = material.pbrMetallicRoughness;
     baseColorTexture.setTexture(videoTexture);
    })
    </script>
  </body>
</html>

Screen shot of the video texture

Help me to find out the solution for rotation and stretch problems.

Thanks

Answered by Anburaj

May 30, 2023

It is working fine if you unmute it on the pointer event listener.

View full answer

elalish · 2023-05-12T17:23:03Z

elalish
May 12, 2023

Indeed, just like an image, the video's dimensions are mapped from 0 to 1 in UV coordinate space. In general the way to fix this is to change your model's UV coordinates. However, we just added a feature to make this much easier: see setRotation and setScale in the Sampler interface of our materials API. Since this hasn't been released yet, you'll need to test with our pre-release version: https://modelviewer.dev/docs/faq.html#entrydocs-general-questions-api

0 replies

Anburaj · 2023-05-12T18:25:43Z

Anburaj
May 12, 2023
Author

I have changed my js link from released version to pre-release version (https://modelviewer.dev/node_modules/@google/model-viewer/dist/model-viewer.min.js).
If I use the setRotation method, I am getting the following error and warnings.

Also, Can you provide sample snippet code for using the setScale method.

1 reply

elalish May 12, 2023

Here's the example, but I just noticed there's a path problem on the GLB 🙃 :https://modelviewer.dev/examples/scenegraph/#transformTexturesExample

Don't worry about the warnings - the other error looks unrelated to setRotation, but if you can't figure it out from our example, then open an issue with a minimal repro. Thanks!

Anburaj · 2023-05-18T06:56:55Z

Anburaj
May 18, 2023
Author

I can able to rotate the video using setRotation method. It is working fine now. But I can't hear the audio from the video. I try to unmute the video using the below line but no luck.
baseColorTexture.texture.source.element.muted = false;

1 reply

elalish May 18, 2023

Hmm, I don't know about hearing the audio. Have you tried it when it's just a video element on the page? I know various browsers have special restrictions on playing audio, so it can be a little tricky.

Anburaj · 2023-05-22T07:19:44Z

Anburaj
May 22, 2023
Author

If I try with just a video element, video is playing fine with audio.
Audio is not working if i use the same video as a video texture and play it.

5 replies

elalish May 23, 2023

No, a repro link is to the actual served webpage where I can see the problem - I'm not going to clone your git repo.

Anburaj May 24, 2023
Author

It is in R&D stage, so i haven't upload to live server. For your review, uploaded temporarily. Website link for you reference
In this example, I am loading the Laptop glb model and on the laptop screen I placed the video texture. Video is playing fine but audio is muted. I tried to unmute it but no luck.
Please tell me the way to play the video with sound.

elalish May 24, 2023

This works to play audio: videoTexture.source.element.muted=false;, however if I try to do it before the user has interacted, I get Unmuting failed and the element was paused instead because the user didn't interact with the document before.

I'd guess if you put it into a pointer-event listener it should work; it just can't unmute automatically (and as a user, I'm so glad!).

Anburaj May 30, 2023
Author

It is working fine if you unmute it on the pointer event listener.

Answer selected by elalish

Rotation and stretch issues while loading the video using createVideoTexture #4278

Uh oh!

Anburaj May 12, 2023

Replies: 4 comments · 7 replies

Uh oh!

elalish May 12, 2023

Uh oh!

Anburaj May 12, 2023 Author

Uh oh!

elalish May 12, 2023

Uh oh!

Anburaj May 18, 2023 Author

Uh oh!

elalish May 18, 2023

Uh oh!

Anburaj May 22, 2023 Author

Uh oh!

elalish May 23, 2023

Uh oh!

Anburaj May 24, 2023 Author

Uh oh!

elalish May 24, 2023

Uh oh!

Anburaj May 30, 2023 Author

Anburaj
May 12, 2023

Replies: 4 comments 7 replies

elalish
May 12, 2023

Anburaj
May 12, 2023
Author

Anburaj
May 18, 2023
Author

Anburaj
May 22, 2023
Author

Anburaj May 24, 2023
Author

Anburaj May 30, 2023
Author