The build system seems to claim things will work.

I do not see how such a claim is inferable from the evidence. In particular, the words, "WARNING: ..." would seem to suggest otherwise.

But it does not. ...

Very true. You must have a functioning tclsh for the overall build to succeed when you build from the source tree. This is not a bug and is unlikely to change anytime soon. Your options are to install enough of Tcl to get a working tclsh or to build from a provided amalgamation (or give up building SQLite.)

I do not see how such a claim is inferable from the evidence.

The usual convention is that configure should fail if the build cannot proceed. Warnings can easily be ignored/missed, a non-existant Makefile is a much clearer signal :)

It is even more fun since if you run make twice, the file sqlite3.h will have been created by shell redirection:

nonexistent-command >sqlite3.h

will error out, but redirect a chunk of nothingness to the output file, which shall now be a zero-byte file. Make doesn't know that this is a problem, so it will then attempt to compile e.g. src/alter.c, which fails with 15 hundred lines worth of compiler errors for unknown types, implicit function declarations, "extra brace group at end of initializer", and more -- because #include "sqlite3.h" worked fine as far as the preprocessor was concerned.

Probably no one will ignore/miss a 15 hundred line compiler error, but figuring out what such a thing means can be harder... :)

This is also technically a bug -- an unrelated bug. The bug here is that shell redirection is chancy and fraught with risk, and it is much safer to have generator tools accept an argv parameter specifying the output file to write to, which then gets written to by opening that file and passing the opened file handle as the first argument to puts.

If the tool fails to run, it doesn't open sqlite3.h for writing, so an empty (zero-byte) file is not created. If the tool encounters a fatal logic error, it can delete the malformed output file in a cleanup trap before exiting.

I agree that creation of an empty sqlite3.h is an improvable situation.

Regarding "also", I think failing to produce a Makefile would be a disservice to those building from the amalgamation(s). This needs more thought rather than the "fix" being suggested.

The usual convention is that configure should fail if the build cannot proceed.

I prefer that convention, but in this case "the build" is ambiguous. After running configure with no tclsh available, targets which use the published amalgamation (such as the CLI, sqlite3.{o,obj} and certain tools) can be built. It is only the targets which are built from the source tree that generally need tclsh.

Warnings can easily be ignored/missed, a non-existant Makefile is a much clearer signal :)

Yes, that is why we like fail-early behavior. Got any ideas how to achieve that without making amalgamation users install Tcl?

After running configure with no tclsh available, targets which use the published amalgamation (such as the CLI, sqlite3.{o,obj} and certain tools) can be built.

Ah I see! The current situation makes a lot of sense then, given the separate use cases. Maybe emitting a different recipe if tcl is not available would make it clearer eg.

sqlite3.h: 	$(TOP)/src/sqlite.h.in $(TOP)/manifest.uuid $(TOP)/VERSION
	$(error tclsh is required to build target $@)

Functionally it's exactly the same as tclsh: command not found but it's clearly intentional.

A smaller change may be to set TCLSH_CMD = tclsh_is_required_for_this_target

The change Richard made a few days ago has nearly the same effect, differing in not naming the target in the same sentence as where tclsh absence is noted. (The make program will normally state what target(s) failed to build, so this nicety is not much of a loss.)

The make program will normally state what target(s) failed to build, so this nicety is not much of a loss.

Unfortunately, this is not the case.

The change does mean that zero-byte files are not created, but the error message in such a case is:

$ make
sh /tmp/sqlite/tool/cktclsh.sh 8.4 tclsh
/tmp/sqlite/tool/cktclsh.sh: line 5: tclsh: command not found
ERROR: This makefile target requires tclsh 8.4 or later.
make: *** [Makefile:770: has_tclsh84] Error 1

It does not inform you of what target failed to build. Or rather, it does -- but the target that failed to build is the "has_tclsh84" target, with no other info.

This is why the suggestion I had for fixing the problem was to directly write to the file handle in puts from the tcl script rather than writing to stdout and using shell redirection.

My suggestion is specifically because the make program then registers a failure trying to build, for example, the "sqlite3.c" target, and the failure doesn't leave a broken output file lying around.

In such a case, Make would be able to report

make: *** [Makefile:12345: sqlite3.c] Error 1

(Or something like that. I have no idea whether sqlite3.c was the file that failed here.)

You can fail during configure, if the amalgamation does not exist and tcl does not exist.

Although I am a bit confused how one builds using the published amalgamation, which is not present in the download.

I suggested above that one possible solution is to include the published amalgamation in sqlite-src-*.zip

It's not a matter of other artifacts from https://sqlite.org/download.html either -- it appears that the only one that uses this configure script is sqlite-src, which (currently) lacks the amalgamation.

You can fail during configure, if the amalgamation does not exist and tcl does not exist.

That introduces dependencies that are more trouble than the marginal benefit that might be thereby obtained. Now, configure is about preparing to use the environment in which it runs. Simple. It does so.

Although I am a bit confused how one builds using the published amalgamation, which is not present in the download.

The amalgamation is in those downloads whose descriptions begin with "C source code as an amalgamation".

I suggested above that one possible solution is to include the published amalgamation in sqlite-src-*.zip

I have suggested and stated that the early October changes address the problem of possibly puzzling build results and messages. At this point, I do not see a genuine problem to be solved.

It's not a matter of other artifacts from https://sqlite.org/download.html either -- it appears that the only one that uses this configure script is sqlite-src, which (currently) lacks the amalgamation.

The archives named "sqlite-autoconf-#######.tar.gz", described as 'C source code as an amalgamation. Also includes a "configure" script ...', should suffice to scratch that itch.

I do not see how such a claim is inferable from the evidence. In particular, the words, "WARNING: ..." would seem to suggest otherwise.

It is incredibly obvious that a warning that says "warning: defaulting to non-amalgamation build" does not say "warning: may not work".

But in fact, it is "physically impossible to work", given the current Makefile, as you point out.

Yet, the build system asserts two indisputable facts:

1. the build system considers the lack of an installed and functioning tclsh to be a condition that the build system checks for

2. the build system does not consider that it is in an erroneous state if this check fails

The inferrable behavior from the build system is pretty straightforward: from the evidence of the words, paraphrased, "we have checked for the condition you are under and have analyzed that the consequences of your condition are that you need to use a non-amalgamation build", we (users and downloaders of sqlite) can infer that you are committing to this being the sole downside of not having a functioning tclsh.

If there were downsides of not having a functioning tclsh, aside for being required to build the non-amalgamated source, and especially if such downsides included "actually no, unfortunately the build will not succeed", then the build system would have said so when it checked for the presence of tclsh!

Very true. You must have a functioning tclsh for the overall build to succeed when you build from the source tree. This is not a bug and is unlikely to change anytime soon.

No, it is unquestionably a bug.

There are only three valid options, when a build system provides a checked warning:

1. make this work, as the warning suggests, "defaulting to non-amalgamation build" (this may mean doing what most other ./configure using projects do, and including "built sources" as part of the release asset)

2. delete the code that checks for the presence of tclsh, notes that it is not present, and emits a misleading warning; rather, do not check for tclsh at all, and simply error out during make

3. delete the code that checks for the presence of tclsh; add some different code, that checks for the presence of tclsh, notes that is not present, and emits a fatal configuration error: "you must have a functioning tclsh for the overall build to succeed when you build from the source tree"

I do not care which one you choose, but I do care that the build system shall be honest. The current situation is the worst of all possible worlds.

I do advise avoiding option number 2.

The Makefile generated by config, as the build files exist now on the repo trunk, has several changes made in response to the above report (post #1) and one below (post #5). Here are their user-visible effects:

1. The configure script is unchanged. Its effect is to generate a Makefile which can build as many of the targets specified therein as are supported by the tools discoverable on the system using automake machinations. A warning is issued to stdout if no tclsh can be found, saying in effect that builds from the amalgamations are supported but building the amalgamations is not. (same effect as the --disable-amalgamation option to configure)

2. When warnings are issued during configure execution, some Makefile targets cannot be built (as before.) However, the make failures occur early and without confusing side-effects such as leaving files empty when generation of their content failed.

3. When tclsh of a certain version or later is needed for a specified target during make execution, a clear error message is issued explaining what is needed, without extraneous error messages, and make is made to return error status.

This may not be ideal. I'm not sure what would be in light of the multiple targets and possible build steps needed for them. Constructive suggestions are welcome. However, citing general good practice is not helpful. The SQLite devs are familiar with that already.