Curiosity: Why does the properties panel only show the text in the taskbar preview?

[DavidBerdik]

Hello ExplorerPatcher Community,

This is not a serious topic, but rather, just a small question of curiosity that I would like to share with the hope that through an answer, I will learn something about how this all works.

I have noticed that if you look at the preview thumbnail for the ExplorerPatcher properties window in the taskbar, the preview only shows the text as though there is no background window. As I am sure we all know, this is not the usual behavior for taskbar previews. I was wondering if anyone has any idea as to what could be going on here? Why is the background window missing from the preview? I assume it probably has something to do with how ExplorerPatcher is spawning the window in explorer.exe?

For anyone who does not know what I am talking about, here is a screenshot showing it:

[valinet]

Hi

Well, let's get technical. The background color of the window is actually black. The memory area where I draw the stuff on the window is initialized to zeros. The size of this memory area is window width*window height*number of channels per-pixel. Each pixel has 4 channels, red, green, blue and alpha which all get zeroed out initially. Then, I draw the stuff over there, the text shown basically.

So normally, the window should be invisible if you account for the alpha channel which is 0, or all black if you do not account that, and only the text be drawn in black with an alpha of 1, which means opaque, but out of the box, alpha is disregarded and everything is opaque anyway. So you would see a black window with black text, so basically a black window.

The things displayed on the screen in modern Windows versions (since Windows Vista) are not rendered directly by the applications. Instead, all applications render their content to off-screen surfaces. There is a program, the compositor (dwm.exe aka Desktop Window Manager) that takes all these off-screen surfaces, positions and overlaps these bitmaps according to how the windows are placed on the screen and renders them on screen. Of course, it's a bit of an overhead, but using this has many many many advantages:

• It enables easy possibility of complex desktop effects: the compositor can apply mathematical transformations on these bitmaps and then render those on the screen; one such example was Win+Tab aka Flip 3D from Windows Vista or Windows 7. The effect is transparent to the applications: they continue drawing themselves in the off screen bitmap and it is just the compositor that takes the data from there, creates the scene and presents it to the user. It virtually enables to present any kind of visual to the user based on what the applications are doing, such examples being Aero Peek etc.
• It allows to provide better experiences when applications hang: remember when, in Windows XP, when an application hang and you moved a window on top of it, that window left trails on the hung application? Well, that's because, without a compositor, applications draw directly on the screen surface that is presented to the user. When you move a window, the foreground application is sent a notification to draw itself at the new position, while background applications are also sent notification to draw themselves in the areas that are now not covered by the moved window. If such a background application would hang, or become slow to respond and act on these notifications, what you saw was the previous, unupdated visual, where the window you moved was still there because the hung app just failed to update that area it is responsible for in time. When the desktop is composed, this problem is gone by design. So, now, advanced experiences can be enabled when a window hangs. What Windows does is that the shell requests the compositor not to draw the hung application anymore. In its place, the shell creates a "ghost" window that displays an image of the hung application's last known image, shaded with a whitish overlay and with a "(Not Responding)" suffix in the title bar to indicate the user that the app is hung and allowing the user to still close or minimize the app.
• It enables easy implementation of some advanced functionalities. For example, virtual desktops. The way those are implemented is that, when you switch to a new desktop, all windows on other desktops are marked as "cloaked". Cloaked windows are windows that are not to be rendered by the compositor. So, the windows continue to render their image in the background, it is just not displayed to the user.
• It offers some pretty powerful APIs to applications. For example, Windows' compositor offers a "thumbnails" API. Basically, applications can offer the compositor a list of regions in their windows and request the compositor to draw the visual of another window over there. And the compositor does just that, when the final image is to be displayed. That's how thumbnails in Alt+Tab, Win+Tab, and taskbar previews work: the application does not render in 2 places. It renders into its off screen buffer. Before the compositor shows the final image on screen, when it composes it, it just draws the application's buffer onto the screen in 2 places: where the window is located, and scaled where the taskbar thumbnail was requested by the shell, for example.
• It enables easy scaling of DPI unaware applications. For example, apps that do not render themselves correctly on high DPI screens can just draw regularly in their off screen bitmap, as if they were run on 100% scaling. When the compositor has to draw the final image, it can just use some algorithm to stretch the image and enlarge it to 125%, 150% or whatever. It’s blurry, but at least it’s readable. Without a compositor, anything similar is basically impossible. Before Windows Vista, “scaling” was done global on the whole OS and wasn’t really scaling. Instead, the system communicated bigger font sizes and icon sizes to applications. Some were aware to those, some just were not and still rendered their tiny 16x16px icons on giant screens and there was not really anyway to fix this other than this 2-step approach with a compositor, other than updating all apps to be aware of high DPI screens
• Lastly, it can add extra stuff to the window. Because there is another step before the window is displayed on the screen, there is a last chance, for the compositor, to draw stuff on each window before it is presented to the screen.

Let's concentrate on the last point. Windows' compositor implements numerous effects, like Aero Glass in Windows 7 and below, and some exotic stuff, like Acrylic and Mica brushes in Windows 10 and 11, respectively, to name a few. How do those work? Well, when I draw the Properties GUI, as I said, by default, the window would be black. But I also request the compositor to use the Mica brush over this window when presenting it on the screen. A brush is basically an image. The Mica brush is a blurry image based on the contents of the desktop wallpaper. Requesting the use of this brush over my window has the following effects in the compositor:

• When rendering the window on the screen, the compositor looks at the pixels in the off-screen buffer where I draw. It blends the contents of my window with the contents of the brush based on the alpha values of the pixels in my off screen buffer. Since the window background has all zero alpha (0%), then it uses 100% of the information in the Mica brush on those areas. Basically, the final image contains the Mica brush over all my window's background.
• Remember I said the compositor offers an API via which applications can request it to draw representations of some other windows onto regions from the application? When displaying taskbar thumbnails, explorer.exe requests dwm.exe that it draws a representation of my window in that area where the thumbnail is to be shown. The compositor has the control of what to draw there, when it composes the image; explorer.exe never sees the actual data, as it does not hold it - it is just an image that is composed and presented only by dwm.exe. So, the compositor has to draw an image of my window there. Since I told it to draw the Mica brush over my transparent pixels, it knows it is safe to do whatever it wants with those pixels. So, when drawing the taskbar thumbnail, instead of drawing Mica there, it decides to draw with the Acrylic brush. Why? Because the windows where the thumbnails are to be drawn were also requested to be drawn using the acrylic brush by explorer.exe. So it decides to blend whatever it has to draw for explorer.exe with my visual. Thus, that preview window results. Why it decides so? Idk, it just does, this is how Microsoft coded an envisioned it. Basically, the way I think it works is this: the information for each window held by the compositor tells whether a window has to be drawn using some brush (any brush, whatever it might be). It also holds a list of regions and their associated brush. When it composes the desktop image, it passes through the list of windows and where they should be drawn. When it gets to ExplorerPatcher, it sees that it has to be drawn with a brush. When it has to draw that thumbnail, since it has to be drawn on some region of explorer associated with an acrylic brush, it uses that. Basically, if a region has an associated brush, that is used; if not, it uses the window's brush, or if that doesn't have one, then yeah, it simply uses no brush. I am sure a lot of design decisions like these boil down to how efficient and costly doing it one way or the other is, how they structured their internal data representation, in what form they store the data etc. That's ALWAYS challenging. I haven't worked on projects the scale of dwm, but even the tiniest things you eventually come across the need of managing data, structuring it in memory and the decisions you make really have an impact on what capabilities you are ultimately able to offer more easily or not. It can be a VERY complicated problem.

There isn't much control about this at the level of my application level. When I request the compositor to draw using the Mica brush over parts of my window, I gave it the freedom to alter my visual with that however it sees fit. I mean, it is already free to do whatever it wants, since it is the only thing that draws directly in the screen buffer anyway, but by that, it knows that doing similar but not exactly the same stuff I requested can’t break my visual of the app.

Similarly, when explorer.exe requests that the compositor draws a visual representation (thumbnail) of my app onto that preview window, it has no control over how the compositor decides to draw it.

So, the compositor is to "blame" here, as it has free hand to do represent certain areas of my app in certain ways, and it chooses to so when it comes to visual representations offered to other apps via the thumbnails API. Yeah, it looks weird, but that's simply because of the way it works. Of course, explorer.exe could "screenshot" the windows themselves and display them there. Or actually, you see, that would not work: if explorer.exe would copy the contents of my window into a buffer of its own, it would actually copy from my off-screen buffer where I draw, which, remember, has the background set to all zeros, so black. So, with that image, you don't see anything !!! Without a compositor to do that crucial step at the end, as it stands now, my application would present all black to the user. Ofc, the easy fix to that is to fill the background with some color in that case. Which is what I do when themes are disabled, for example, in which case I do not request the compositor to use the Mica brush, but instead draw the background of the application in the system color for that. Note that disabling themes (or visual styles) or the compositor in Windows 8 and up is totally unsupported and merely a hack, but it can be done. People disable themes even today because the windows are then drawn using the Windows 95-era UI.

So, screenshot of my application directly would not work. You still have to get that through the compositor to get something useful for some apps, so explorer.exe kind of has to go through this route. What it could do with the way things stand now is not to request its window preview to be drawn with acrylic by the compositor. dwm draws the thumbnails without replacing the effect requested by the represented app when the window it draws onto did not request an effect itself. Or just alter dwm.exe and have it not replace the effect when drawing over an area that should have some other effect. Idk, multiple fixes possible, but all are out the scope of my application.

So yeah, that's the full technical explanation. I know, it is rather long, but hey, modern computing is always overly complicated. All the nice effects do not come for free and aren't cheap. That's why the Properties GUI does not have effects when hovering the mouse over its elements. As I said, that's not for free, and I simply haven't bothered implementing those. The interface is 100% drawn manually, no GUI widgets toolkit is used, so mouse hovering over stuff has to be implemented manually as well.

So yeah, it's not explorer.exe to blame per se, or the fact that I run EP in explorer.exe. The GUI actually runs out-of-process, in a rundll32.exe instance that I spawn, so that restarting Explorer does not close the Properties window. And you can get the same effect from the compositor from any window. Just create a window and request Windows to draw its background using the Mica brush:

BOOL value = 1;
DwmSetWindowAttribute(hWnd, 1029, &value, sizeof(BOOL));

You will see, you still get a taskbar thumbnail of full acrylic, even though the window on screen would be full Mica. And you similarly would seem to get no window thumbnail of my other window if you create another window and register a thumbnail of the other window with dwm.exe and request dwm.exe that it draws the background of this window using the acrylic brush:

ACCENTPOLICY policy;
policy.nAccentState = 4;
policy.nFlags = 0;
policy.nColor = (Opacity << 24) | (Color & 0xFFFFFF);
policy.nFlags = 0;
WINCOMPATTRDATA data = { 19, &policy, sizeof(ACCENTPOLICY) };
SetWindowCompositionAttribute(hWnd, &data);

And that's because it has this behavior where it actually uses the last effect in the chain when drawing a thumbnail, for whatever reason.

The thumbnails API is public, so you can look up on MSDN how to use that. The snippets I published for enabling Mica or Acylic brushes are not documented. Oh yeah, forgot to mention that, it's not officially supported, applications are not supposed to use those. But I mean, what is documented these days, EP is a giant pile of hacks and uses of undocumented functionality from Windows.

[DavidBerdik]

Wow! That was a really nice explanation! It took three reads to get a full grasp on it all, but it makes sense now. Thank you!