how does drawElement interact with devicePixelRatio and/or how can one be specified?

[greggman]

the readme on this issue says:

drawElement(element ...) takes the CTM (current transform matrix) of the canvas into consideration. The image drawn into the canvas is sized to element's border box size;

But that is not enough into to know how to render when wanting to take devicePixelRatio into account.

Assume a devicePixelRatio of 3.5 from a Samsung S20 Ultra. Given an element with a border box size of 400x300 CSS pixels, to match the HTML I'd want that rendered to 1400x1050 in the canvas. It doesn't seem like there is a way to know apriori what size the developer wants. Should a devicePixelRatio option be add to the options? Or maybe there's some other solution like making a container element that is 400x300 but setting it's zoom to 350%. Just throwing out ideas.

[mrdoob]

+1

Developers will want to be able to control the resolution of the texture being created.

Without this control it's easy to imagine a developer getting reports of, say... a form being too blurry on some user's device. Maybe the developer used a 4K display with a DPI of 3 to develop the site, but the user is running it on a small phone with a DPI of 1. How could the developer address the issue?

For quality, memory and performance reasons; they will also need to control the size for textures being used on 3d objects not too close (or too close) to the camera.

[chrishtr]

The prototype and spec proposal already takes into account devicePixelRatio; the explainer was out of date. I updated it to read "The image drawn into the canvas is sized to element's devicePixelContentBox". In other words, the image resolution matches the display's resolution.

Regarding:

Maybe the developer used a 4K display with a DPI of 3 to develop the site, but the user is running it on a small phone with a DPI of 1. How could the developer address the issue?

With the above wording (which is reflected in the prototype), it should Just Work in this scenario.

For quality, memory and performance reasons; they will also need to control the size for textures being used on 3d objects not too close (or too close) to the camera.

To address, this we could add optional width/height parameters to texElement2D, the effect of which would be to scale the image down to the desired width and height resolution when rastering.

[greggman]

"The image drawn into the canvas is sized to element's devicePixelContentBox"

That seems like it would be very non-portable.

I write this

<canvas width="400" height="300">
  <pre style="
      width: 100%; height: 100%;
      color: yellow; background-color: red;
      font-size: 100pt;
   ">Hello
World</pre>
</canvas>

I then do this

const canvas = device.querySelector('canvas');
const elem = canvas.querySelector('div');
const ctx = canvas.getContext('2d');
ctx.drawElement(elem, 0, 0);

On a 1.0 dpr machine it draws a 400x300 image into my 400x300 canvas. On a 2.0 dpr machine it draws the top left 400x300 of an 800x600 image into my 400x300 canvas. So I thought I'd get

but instead I get this

[chrishtr]

On a 1.0 dpr machine it draws a 400x300 image into my 400x300 canvas. On a 2.0 dpr machine it draws the top left 400x300 of an 800x600 image into my 400x300 canvas.

That's because you need to scale the backing of your canvas to match the device pixel size of the canvas element. See instructions here. Example implementation here.

[greggman]

My point was I don't think that's a valid requirement. The API should provide consistent results regardless of the user's devicePixelRatio. The app should opt into dealing with devicePixelRatio issues. This is how all the rest of the 2D, WebGL, and WebGPU APIs work

[chrishtr]

My point was I don't think that's a valid requirement. The API should provide consistent results regardless of the user's devicePixelRatio.

It will, if you set up the canvas correctly. If we were inventing the canvas element today, we'd set the backing sizinig automatically to account for devicePixelRatio, but we can't just change that or it will break content. Developers already have to account for it. And if they don't, content will look blurry already.

In the example you gave, it's a developer error. I don't think it's a good idea for the browser to guess that a developer failed to resize the canvas backing properly to match the pixel density on their screen.

[greggman]

It will, if you set up the canvas correctly. If we were inventing the canvas element today, we'd set the backing sizinig automatically to account for devicePixelRatio, but we can't just change that or it will break content. Developers already have to account for it. And if they don't, content will look blurry already.

No, we would not. Apple did that in 2013, shipped it, had to revert because it broke so many sites. The Web can not be a "follow these 12 steps and your app will be portable". It needs to write once and it works.

[chrishtr]

Discussed offline with @greggman.

Options for default sizing:

1. Render sized to devicePixelContentBox

• Pro: this is the same resolution and sizing of DOM content, so (a) content drawn into the canvas on first render will match the screen pixel density & match the quality of surrounding DOM content, and (b) it's the sizing that layout of the DOM subtree optimizes and pixel-snaps for. This will yield the highest quality output as a result.
• Con: developer needs to know this sizing and account for it when positioning content. (However, developers already need to account for devicePixelContentBox if they want their canvas to look good on all screens.)

2. Render as if devicePixelRatio was 1.

• Pro: this is (a) easier to understand for developers who know the CSS pixel size of the canvas, and (b) renders correctly with the same default sizing as a canvas rendering context for existing <canvas> elements.
• Con: sizing is not optimized to look good at device pixel resolution; forces complexities in the layout engine for different rounding (or worse quality output).

Options for default canvas rendering context sizing when layoutsubtree is specified:

a. Size to devicePixelContentBox

• Pro: sizing of the back buffer automatically matches that of drawElement DOM source without any special effort
• Con: differs from the usual sizing of CSS pixels for canvas rendering contexts. Won't automatically update if devicePixelContentBox changes. Requires developers to account for it when positioning content. (However, developers already need to account for devicePixelContentBox if they want their canvas to look good on all screens.)

b. Size to CSS pixels

• Pro: matches existing <canvas> defaults
• May require extra effort for drawElement users if (1) is chosen.

I currently think that (1) + (b) is a good choice to consider, because it's a bit better quality out of the box (though it doesn't save the developer from having to observe devicePixelContentBox for optimal results after changes to resolution). At the moment, the implementation does (1) + (b). I'd like to avoid (2) because it will result in worse output and complexity in the layout engine we want to avoid.

[greggman]

I think you could get (2) with near zero overhead by internally setting the zoom level to 1 / devicePixelRatio you'd get what's expected. The browser would generate a texture for the elements where width = canvas.width, height = canvas.height, devicePixelRatio = 1. Drawing it would not require any re-layout nor anything special. Developers would get something that matches all the other APIs.

If the user wanted to opt into a higher res result they could set some attribute on the canvas like canvas.childElementsDevicePixelRatio = devicePixelRatio but childElementsDevicePixelRatio would default to 1. Regardless of what they set it to, they'd get the same (or similar) perf.

If that didn't make any sense, imagine this setup

<canvas width="400" height="300">
  <internal-invisible-element
     style="
        zoom: canvas.childElementsDevicePixelRatio / devicePixelRatio;
        width: canvas.width;    // needed to make the children the correct size
        height: canvas.height;  // needed to make the children the correct size
     "
  >
     ...child-elements-go-here...
   </internal-invisible-element>
</canvas>

Note: In all cases, the canvas would stay 400x300. The only change is what's rendered. if you developer wanted a canvas to match dpr for display they'd do what they have for the last 12 years and set the size of the canvas to desiredSize * devicePixelRatio

I still don't think (1) is viable.

you do this

   ctx.arc(100, 100, 50, 0, Math.PI);
   ctx.fill();
   ctx.fillRect(160, 50, 20, 100);
   ctx.drawElement(someElementYouExpectToBe100x100, 190, 50)

The first 3 commands always work the on all devices at all zoom levels. The last command does not. It effectively draws a random size element based on the user's devicePixelRatio and their current zoom level.

Adding some transform to try to fix that is not enough because you don't know what the user is using the canvas for. They asked for 400x300 canvas (or some other specific size). They might be using it to generate a social media badge or avatar or thumbnail that requires a size, something that they specifically want 400x300. The browser should not be effectively given them a random sized canvas different than what they asked for. Rather, if they want a dpr sized canvas they should ask for it.

Once drawElement is added people are going to start using it as a new draw primitive. Not just was a way to show html elements for UI. For example to draw fancy text like

Is far easier to do in with HTML and CSS than individual canvas draw commands.

Similarly for a formatted colored text like