A suggestion: temp0000, temp0001..
A suggestion for temporary file names. I usually use numbered names. That way they are easier to handle. There is no need to go through the directory. And the name is unique with absolute certainty. You know: temp0000, temp0001, ..
(2) By Scott Robison (casaderobison) on 2021-09-01 20:10:07 in reply to 1 [source]
That makes it easy for other parties to predict and wreak havoc with your system, as well. Given the number of systems SQLite is used in, predictable resource names can be a security issue.
Security is one concern -- but sensitive data should probably not be stored in world-accessible temporary files.
Another concern is speed: if there's already 500 temporary files named temp0000..0499, which your application doesn't know anything about, it takes 501 attempts (corresponcing to 501 file system access calls) to create a new temporary file, whereas with (pseudo-)randomized names, one single attempt is usually enough!
With a catalog it is O (1). Without catalog is O (log maxfile).
(6) By Scott Robison (casaderobison) on 2021-09-03 21:31:20 in reply to 5 [link] [source]
I'm working on only one hour of sleep in the last 36 hours, so I probably shouldn't even be posting. But here I am...
Catalog? What is this catalog that works across all supported operating systems and is able to provide the next number as an O(1) operation?
I'm sure I'll feel stupid for asking the question later after I get some sleep, but I've written a few file system drivers in my time and even more programs that traverse directory structures, and I'm not aware of a generally applicable way that is always up to date to issue a query that will return the next in a sequence that is itself O(1). And without a catalog is completely dependent on the underlying file system storage hierarchy. Some file systems allow you to find a file (or identify its absence) in O(log N) time, but some file systems are just linked lists of directory entries. And if you are trying to find the next number in a sequence, you wind up making one of those log N calls for each file that exists before you have a candidate you can try, and hopefully you win the race to open it before the peer process attempts and succeeds the same thing.
I just enquire if a file with the given name exists, and if so construct a new name by appending a random number from 10000 and 99999 to the filename and trying again until I'm told that this new name doesn't exist.
Store the last file number created in a global counter. Increase it by any fixed or random amount and ... Eh Voila! I speak of very easy things. The things that work. If any Hacker discovers it, he will try to exploit it. They are reckless. They love to discover silly things. Create a file with an attractive name. Record all metadata. Never use it. If something changes, even if it is the last reading date, you are not alone.
(9) By Scott Robison (casaderobison) on 2021-09-04 12:04:40 in reply to 8 [link] [source]
These are perfectly reasonable solutions for certain classes of problems. They work until they don't. In any case, the issue as I interpreted the original post was a suggestion that SQLite use this technique as a general purpose temp file generation mechanism, and I think that would be a mistake.
Is there some context I'm missing? Did someone ask about creating temp files in an application specific context? That is very different than a general post in a SQLite support forum which made me think you were calling for a change in SQLite.
Oh forget it already, it's just a stupid idea. I'm going to erase it.
(12) By Scott Robison (casaderobison) on 2021-09-04 13:10:50 in reply to 11 [link] [source]
I'm sorry, I wasn't trying to be rude. I was really just trying to understand the impetus of the original post. Without more context, I thought I might be missing something.
Oh that's nothing. American Randy Gardner holds the world record for having gone the longest without sleep. A 17-year-old high school student in San Diego, Gardner went without sleep for 11 days. I suggest you a comfortable office sofa.