Thanks for all the work on this Stephan. I will try to do some testing locally and provide feedback. Hopefully today.

I updated my local APKBUILD for the Alpine package to the latest trunk version and tested locally with the built in test certificate and a self signed certificate. All seems good!

Stupid question, if I wanted to provide both an http and https version of the same site, I assume I'd have to run two instances of althttpd?

Stupid question, if I wanted to provide both an http and https version of the same site, I assume I'd have to run two instances of althttpd?

Not a stupid question at all: yes. althttpd's internal state only accounts for a single type of connection (probably because that all it historically had). It "could" be refactored to support both in the same instance, i suppose, but i'm not currently volunteering to do so ;). Most simply, it could probably just fork a second copy of itself for the other port if told to - that probably wouldn't require more than a dozen lines of code plus another CLI flag or two to enable passing on of two port numbers.

I think that should be -DENABLE_TLS

Fixed, thank you.

Management summary "works for me!"

Follow-up: "still works for me!" Althttp(s)d in TLS mode (with letsencrypt certs) has been in non-stop use for all of my own Fossil repos the past six days on an Ubuntu 21.10 server with no visible defects or strangeness on the server or the clients (Firefox and Chromium on Linux). Roughly half a dozen fossil /chat pollers are in constant use, so it's been running 24/7 during that time.

The one open point is whether or not SCGI still behaves the same as it does with the non-TLS-aware trunk.

I made some minor changes to the TLS branch:

• The --tls-cert-file option was renamed to just --cert

• Added a new --pkey option that lets you specify the private key separately (if desired). If --pkey is omitted, it still expects to find the private key appended to the --cert file.

The branch has now been merge onto trunk. I'm an now running these websites using TLS-enabled althttpsd, and without stunnel4:

• https://www3.fossil-scm.org/
• https://www3.sqlite.org/
• https://www3.sqlite.org/althttpd

I'm using xinetd with the following "https" config file:

service https
{
  port = 443
  flags = IPv4
  socket_type = stream
  wait = no
  user = root
  server = /usr/bin/althttpsd
  server_args = -logfile /logs/http.log -root /home/www -user www -cert /etc/letsencrypt/live/www3.sqlite.org/fullchain.pem -pkey /etc/letsencrypt/live/www3.sqlite.org/privkey.pem
}

I made some minor changes to the TLS branch:

All good and running on the new VPS.

One question about these parts:

• /file?ci=tip&name=althttpd.c&ln=2166
• /file?ci=tip&name=althttpd.c&ln=3081

It wasn't clear to me whether it was semantically legal to re-use a connection when TLS is involved, thus those new if() blocks. That was a case of "better safe than sorry." If it is indeed legal to have that connection loop in TLS mode then those two if()s can be removed.

I'm using xinetd with the following "https" config file:

i'm currently running it standalone but will add that xinetd snippet to the setup docs.

I think it should be ok to send multiple HTTP requests over the same TLS connection. In fact, it should help it to run faster. I'll experiment with taking out those restrictions, when I get a chance.

I think it should be ok to send multiple HTTP requests over the same TLS connection. In fact, it should help it to run faster.

i tried a build of that a few hours ago, simply removing the two relevant "if" conditions, and it broke miserably with firefox reporting SSL-related connection errors. A cursory glance suggested that all connections after the first for a given althttpsd instance were affected but i didn't investigate it in detail because it interrupted an in-progress /chat on that server.

will add that xinetd snippet to the setup docs.

Oddly, with HTTPS via xinetd the sites work fine when browsing and /chat but fossil sync fails with "server did not reply". Using --httptrace doesn't say anything useful:

$ f pull -httptrace
Pull from https://stephan@fossil.wanderinghorse.net/home
server did not reply
Pull done, wire bytes sent: 448  received: 0  ip: 194.195.245.37
/***** Subprocess 1579931 exit(0) *****/

Is fossil sync working for you via xinetd?

I confirm the problem. I'll look into it.

Not just "fossil sync" - any POST request seems to fail.

Not just "fossil sync" - any POST request seems to fail.

That begs the question of whether a TLS build running in non-TLS mode will work via stunnel4. i can't test this at the moment but will do so later today.

Related post from the sqlite forum: https://sqlite.org/forum/forumpost/b585f1f4589802cf

Not just "fossil sync" - any POST request seems to fail.

That begs the question of whether a TLS build running in non-TLS mode will work via stunnel4.

Tested, including POST. That case works as expected with this stunnel config:

pid = /var/run/stunnel4/stunnel.pid
cert = /etc/letsencrypt/live/FOO/fullchain.pem
key = /etc/letsencrypt/live/FOO/privkey.pem
[https]
accept       = https
TIMEOUTclose = 0
exec         = /jail/bin/althttpsd
execargs     = /jail/bin/althttpsd -logfile /log/althttpd.log -root /jail -user USER -https 1

So standalone TLS and stunnel4 TLS work, but TLS via xinetd doesn't, for as-yet unknown reasons.

I'm using xinetd with the following "https" config file:

Wow! This is beautiful! I am going to try to set up mine tomorrow! Sweet! Thanks guys!

josé

On the homepage:

To build althttpd with built-in TLS support using libssl:

gcc -Os -o /usr/bin/althttpd -fPIC -DENABLE_SSL -lssl -lcrypto \
althttpd.c

I think that should be -DENABLE_TLS

Status of the tls-support branch...

New info: we've just discovered via a fossil /chat channel that attaching images to chat messages is breaking when using the built-in TLS but works fine via stunnel. The images are all truncated, no matter how small they are. It works with SVGs and other text files, which sort of implies that this may be a case of althttpd trying to do a tls_gets() where it shouldn't. Correction: it does not work with text files - those tests were after switching to stunnel4.

Still investigating.

Small text files seem to work, but attaching althttpd.c leads to a truncated file. The whole file is POSTed, but the resulting server-side read is truncated for as-yet-unknown reasons.

Based on:

https://www.openssl.org/docs/manmaster/man3/SSL_read.html

and what i'm seeing with debug output, SSL_read() is performing partial reads:

The read functions work based on the SSL/TLS records. The data are received in records (with a maximum record size of 16kB). Only when a record has been completely received, can it be processed (decryption and check of integrity). Therefore, data that was not retrieved at the last read call can still be buffered inside the SSL layer and will be retrieved on the next read call. If num is higher than the number of bytes buffered then the read functions will return with the bytes buffered. If no more bytes are in the buffer, the read functions will trigger the processing of the next record. Only when the record has been received and processed completely will the read functions return reporting success. At most the contents of one record will be returned. As the size of an SSL/TLS record may exceed the maximum packet size of the underlying transport (e.g. TCP), it may be necessary to read several packets from the transport layer before the record is complete and the read call can succeed.

Which corresponds to my debug output:

tls_read_server(nBuf=111141), n=15613, rc=15613
tls_read_server(nBuf=111141), n=16384, rc=31997
tls_read_server(nBuf=111141), n=16384, rc=48381
tls_read_server(nBuf=111141), n=16384, rc=64765
tls_read_server(nBuf=111141), n=16384, rc=81149
tls_read_server(nBuf=111141), n=16384, rc=97533
tls_read_server(nBuf=111141), n=13608, rc=111141

It turns out that fossil's own TLS impl has this same bug! The fix seems to be to read in a loop, but i've not yet found out how to detect EOF (and the returned chunk sizes are not always equal, so that's not helpful).

... SSL_read() is performing partial reads:

This is now resolved in althttpd trunk. It's patched and tested in fossil as well and will be checked in there momentarily.