Anticipating Stephan's inevitable comment, yes I will try and figure out what might go where, but I'm curious to know if there is even any interest.

FWIW, i'd be interested in seeing it, especially if someone else did the work of implementing it ;).

https://caniuse.com/brotli

For what it's worth, as a very simple proof of concept this functions just fine. The below, horrible, patchfile simply substitutes brotli for all the gzip logic.

On my server this works flawlessly when the appropriate .br files are present for static content.

I haven't tried it with a browser that doesn't accept brotli content encoding, partly as I don't have one. I suspect curl could be used to trial.

I have some small concerns that simply testing for "br" in the accepted encoding headers might falsely trigger if, say "umbrella" was ever an accepted content type. I'm sure this could be improved on.

Note the much more tricky problem of checking brotli, and then falling back to checking for gzip and serving that instead is not attempted with this patch. It offers brotli compression only, and also doesn't rename the internal variables in althttpd.

As a proof of concept it works well though.

Note that the html files in the top level directory of the sqlite documentation download zipfile offered compress using gzip to 3.8MB, whereas they brotli encode to 2.7MB - this is why I am so interested in this as an option.

The compression does take significantly longer, but this is entirely acceptable in the context of precompression for a static website.

For a short period (probably a week or so only) and for interest, a server running this code and serving the sqlite docs can be found here.

One can, of course, check with the dev tools to ensure that compressed content is being returned. However, all of the uncompressed files have been zeroed out for this test.

Index: althttpd.c
==================================================================
--- althttpd.c
+++ althttpd.c
@@ -95,12 +95,12 @@
 **   (10) If compiled with -DENABLE_TLS and linked against OpenSSL and
 **        launched with a --cert option to identify a certificate file, then
 **        TLS is used to encrypt the connection.
 **
 **   (11) When requesting static file NAME and if the HTTP request includes
-**        "Accept-Encoding: gzip" and if file "NAME.gz" exists on disk, then
-**        return NAME.gz with an "Content-encoding: gzip" header
+**        "Accept-Encoding: brotli" and if file "NAME.br" exists on disk, then
+**        return NAME.br with an "Content-encoding: br" header
 **
 ** Command-line Options:
 **
 **  --root DIR       Defines the directory that contains the various
 **                   $HOST.website subdirectories, each containing web content
@@ -2044,10 +2044,11 @@
   { "au",         2, 0x00, "audio/ulaw"                       },
   { "avi",        3, 0x00, "video/x-msvideo"                  },
   { "bat",        3, 0x00, "application/x-msdos-program"      },
   { "bcpio",      5, 0x00, "application/x-bcpio"              },
   { "bin",        3, 0x00, "application/octet-stream"         },
+  { "br",         2, 0x00, "application/x-brotli"             },
   { "c",          1, 0x00, "text/plain"                       },
   { "cc",         2, 0x00, "text/plain"                       },
   { "ccad",       4, 0x00, "application/clariscad"            },
   { "cdf",        3, 0x00, "application/x-netcdf"             },
   { "class",      5, 0x00, "application/octet-stream"         },
@@ -2520,20 +2521,20 @@
     MakeLogEntry(0, 470);  /* LOG: ETag Cache Hit */
     return 1;
   }
   if( rangeEnd<=0
    && zAcceptEncoding
-   && strstr(zAcceptEncoding,"gzip")!=0
+   && strstr(zAcceptEncoding,"br")!=0
   ){
     szFilename = strlen(zFile);
     if( szFilename < sizeof(zGzFilename)-10 ){
       memcpy(zGzFilename, zFile, szFilename);
-      memcpy(zGzFilename + szFilename, ".gz", 4);
+      memcpy(zGzFilename + szFilename, ".br", 4);
       if( access(zGzFilename, R_OK)==0 ){
         memset(&statbuf, 0, sizeof(statbuf));
         if( stat(zGzFilename, &statbuf)==0 ){
-          zEncoding = "gzip";
+          zEncoding = "br";
           zFile = zGzFilename;
           pStat = &statbuf;
         }
       }
     }

For what it's worth, as a very simple proof of concept this functions just fine.

To be accepted in this tree it would need to support both gz and brotli. We have at least one frequent fossil user who insists on sticking with MSIE, which doesn't support brotli.

It would probably need to look like:

• If AcceptEncoding supports brotli:
  • If filename.br exists, serve that, else...
• If AcceptEncoding supports gz:
  • If filename.gz exists, serve that, else...
• Server filename as-is

IMO brotli should be checked first because (A) it's supported by all modern browsers and (B) it compresses better.

If this were added, i would definitely switch to it.

Agreed.

As previously mentioned, I'm no computer coder, so I'd be delighted if anyone wants to take a punt at this.

Having said that, I like a challenge, so I may plug away steadily and see how things progress.

I seem to have managed to compile althttpd with both the brotli changes, my hsts patch and my openssl changes all working OK, so the test server linked to above should actually remain live for the foreseeable future, as it is compatible with all of the use cases I have.

Notably, fossil instances served by it continue to deliver gzip-ed content just fine (they use their own content headers and compression, so this change to althttpd makes no difference).

It changes things only for static content served by the althttpd server directly.

(I realise you know this, Stephan, but possibly useful for anyone else considering the move).

This means that even with MSIE, Fossil-served gzipped-content should still function normally even with this very limited patch (checked and works as expected).

That test server should now serve an explanatory file if visited using a browser that is not compatible with brotli encoding, pro tem (which it indeed does when visited from MSIE).

Notably, fossil instances served by it continue to deliver gzip-ed content just fine (they use their own content headers and compression, so this change to althttpd makes no difference).

True - fossil is hard-coded to gzip, and its compression is independent of althttpd's.

Until brotli is as widespread as zlib, it's unlikely to get any support in fossil (which strives to be as dependency-free as feasible, making exceptions only for the things which are (A) ubiquitous (zlib and libssl) and (B) difficult to create standalone implementations of (also zlib and libssl).

It changes things only for static content served by the althttpd server directly.

Right, so fossil's not even a concern (despite my comment earlier suggesting the contrary).

Exactly. hence making the patch available.

Even while I'm waiting (hoping for someone to take this off my hands... 🤣) for a definitive proper patch, it's far better to serve uncompressed content to the odd MSIE or cli user and brotli to everyone else, than gzip to everyone else including MSIE.

It really does (look as though) it will make a massive difference to my bandwidth consumption.

There is no way I'm going anywhere near fossil's internals, understandably.

For what it's worth, as a very simple proof of concept this functions just fine.

If you are able, please try /timeline?c=broti-compression and let us know if that works for you. It seems to work in my very basic tests.

Edit: that's now running on my personal site. Not all files have brotli-compressed versions because i need to change my build process to account for that, but all generated files do (which includes all CSS, JS, and HTML files). i've hand-picked certain dirs to delete the ".br" files from and am seeing that it picks up the ".gz" files instead. i.e. it "works for me.

Edit: that's now running on my personal site.

It's also running on https://wasm-testing.sqlite.org/ and seems to be working well there.

sqlite3.wasm drops from 841k (uncompressed) to 339k with brotli and 391k with gzip.

That looks great Stephan. I'll try that when I get a chance, and feedback.

I've also discovered that chrome devtools lets you restrict the encodings that are accepted, which is great for testing.

Edit:

Have deployed it.

Great. Works as expected.

If a X.gz file and an X.br file are both present, the brotli file is served preferentially, the gzip file as a backup if gzip but not br are declared, and the uncompressed file if neither accept-encoding string is declared.

The index.html of the test site I link to above has different content depending on whether you are viewing the brotli version (normal sqlite content) gzip (mentions you are viewing gzip) or uncompressed (just has information about getting a brotli capable browser, no mention of gzip) which acts as a really simple took for seeing which version you are being served.

Very happy with this, thank you for making it into a properly rounded patch.

I'm likely to run with this going forwards now. I'll report back if any issues.

Cheers!