No Content-Encoding added when loading .gz files

[CrumbSandwich]

I've been trying for days to get it to load and process gz-compressed files, but it just doesn't work because of the missing content encoding header.
(ChatGPT is useless too; it just produces garbage in that case.)

I've read here (me-no-dev#556) that the Content header is supposedly set automatically, but that's somehow not true.
My gz files don't receive this header from the web server.

I use a static route

 const char Startpage[] = "Dashboard.html";
 server.serveStatic("/", SD, "/").setDefaultFile(Startpage);

This doesn't work with gz-compressed files.

Then i tried several methods like this:

server.on("/js/compressed/", HTTP_GET, [](AsyncWebServerRequest *request){
  String path = request->url(); 
  String filePath = path;
  String mimeType = "application/javascript"; 
  AsyncWebServerResponse *response = request->beginResponse(SD, filePath, mimeType);
  response->addHeader("Content-Encoding", "gzip");
  request->send(response);
});

But this also doesn't work.

The browser loads it as a normal file, and there's no content encoding in the response.

See here:

await fetch("192.168.xxx.xxx/js/compressed/MyFile.js.gz", {
    "credentials": "omit",
    "headers": {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:138.0) Gecko/20100101 Firefox/138.0",
        "Accept": "*/*",
        "Accept-Language": "de,en-US;q=0.7,en;q=0.3",
        "Pragma": "no-cache",
        "Cache-Control": "no-cache"
    },
    "method": "GET",
    "mode": "cors"
});

Please tell me what else I should do. :')

Edit: I'm using the actual 3.7.7 library.

[mathieucarbou]

Did you read the doc also ? Or look at other existing projects ?
gzip serving works and there are several ways to achieve it.

[CrumbSandwich]

I would if I had found docs that covered Gzip and provided an example with multiple static routes.
Maybe I'm blind.
Do you have a link?

I just found out that if I specify a file BEFORE the static route, it works for that one file:

server.on("/js/compressed/MyFile.js.gz", HTTP_GET, [](AsyncWebServerRequest *request){
Debugging(F("MyFile.js.gz requested"));
String path = request->url();
String filePath = path;
String mimeType = "application/javascript";
AsyncWebServerResponse *response = request->beginResponse(SD, filePath, mimeType);
response->addHeader("Content-Encoding", "gzip");
request->send(response);
});

But I want to intercept all files in the directory "/js/compressed/" and not specify each individual file.

[CrumbSandwich]

Guys, i got it.

Here's my explanation with a static route example:

server.serveStatic("/FOLDER1/", SD, "/FOLDER1/")

Lets say we have a static route for directory called X:/FOLDER1/.
All directories INSIDE this folder cannot be intercepted with server.onNotFound() when you have a valid file request.

And therefore pointless for gzip-compressed content if the gzip-compressed file is somewhere INSIDE /FOLDER1/, because this specific header isn't set for decompression:
->addHeader("Content-Encoding", "gzip");

I don't know why but this header won't be added somehow.
It only works, if i add the header manually.

So i moved the static route to another directory and my compressed file in another test directory:

const char Startpage[] = "Dashboard.html";
server.serveStatic("/OS/", SD, "/OS/").setDefaultFile(Startpage);

All not compressed files:
X:/OS/Dashboard.html

All compressed files:
X:/compressed/fileXYZ.js.gz

Everything in Folder /OS/ has a static route.
Everything outside will be intercepted at "server.onNotFound()"

It works. ! :D
This simple explanation would have saved many hours of work.

Now i check in server.onNotFound() , if the requested file exist and if it ends with .gz and add a specific header.
Here's my working code:

  server.onNotFound([](AsyncWebServerRequest *request) {
    String path = request->url();

      if (path == F("/")) path = F("/OS/Dashboard.html");

      if (SD.exists(path)) {

        if (path.endsWith(F(".gz"))) {
          String contentType;
          if (path.endsWith(F(".html.gz"))) contentType = F("text/html");
          else if (path.endsWith(F(".css.gz"))) contentType = F("text/css");
          else if (path.endsWith(F(".js.gz"))) contentType = F("application/javascript");
          else if (path.endsWith(F(".png.gz"))) contentType = F("image/png");
          else contentType = F("application/octet-stream");

          AsyncWebServerResponse *res = request->beginResponse(SD, path, contentType);
          res->addHeader(F("Content-Encoding"), F("gzip"));
          request->send(res);

        } else {
          request->send(SD, path); 
        }
      } else {
        request->send(404, F("text/plain"), F("File not found"));
      }

  });

Maybe not perfect, but I'm tired now and I'm calling it a day. :>

[mathieucarbou]

All directories INSIDE this folder cannot be intercepted with server.onNotFound() when you have a valid file request.

That's normal.

And therefore pointless for gzip-compressed content if the gzip-compressed file is somewhere INSIDE /FOLDER1/, because this specific header isn't set for decompression:

I do not understand...

serveStatic is supposed to look for the existence of a .gz file matching the requested file.

Example: if http://192.168.4.1/static/foo.html is requested, then foo.html.gz should be tried first.
That's what _searchFile line 139 of WebHandlers.cpp does.

Fallback is to find the requested filename.

[CrumbSandwich]

I do not understand...

serveStatic is supposed to look for the existence of a .gz file matching the requested file.

Example: if http://192.168.4.1/static/foo.html is requested, then foo.html.gz should be tried first. That's what _searchFile line 139 of WebHandlers.cpp does.

Fallback is to find the requested filename.

But that seems to be the issue and i don't know why.
If i have a static route and i request a gz-file, i get the Uncaught SyntaxError: illegal character U+001F because the Content-Encoding header is missing in the response.

For example, i have a static route to the root directory:
server.serveStatic("/", SD, "/");

In the root directory i have X:/js/Script.js.gz

So i call /js/Script.js.gz and get the status 200 but the browser seems to interpret the compressed file as normal text and doesn't receive any information that it is gzip-compressed file.

Do you have any idea?

[mathieucarbou]

You need to call /js/Script.js without gz at the end.