SQLite Forum

How do create the neat flow diagrams for SQLite?
Login

How do create the neat flow diagrams for SQLite?

(1) By doug (doug9forester) on 2020-04-15 04:58:01 [link] [source]

The SQLite language documentation is great! I want to use the same thing to document my own stuff. How can I generate those diagrams and associated text?

Here's a link to one page, for example: https://www.sqlite.org/lang_createtable.html

(2) By Warren Young (wyoung) on 2020-04-15 06:07:38 in reply to 1 [link] [source]

(3) By doug (doug9forester) on 2020-04-15 15:48:47 in reply to 2 [link] [source]

hmmmm... I got to the web page with the list of files: https://www.sqlite.org/docsrc/dir?ci=tip

Now, how do I actually download those file? I can't find an option anywhere on the page. There are more than 70 files.

(4) By Richard Hipp (drh) on 2020-04-15 16:05:05 in reply to 3 [link] [source]

There are "Tarball" and "ZIP Archive" links on the https://www.sqlite.org/docsrc/vinfo?name=15a047bc79409af5 page.

(5) By doug (doug9forester) on 2020-04-15 17:00:07 in reply to 4 [link] [source]

I must be blind because I can find links to neither Tarball nor ZIP Archive on the page you referenced.

(6) By Warren Young (wyoung) on 2020-04-15 18:55:33 in reply to 5 [link] [source]

I must be blind because I can find links to neither Tarball nor ZIP Archive on the page you referenced.

There's a Fossil feature that controls whether they're available to users not logged in or logged in as "anonymous." The capability to allow zip and tarball access is apparently not given to such users on that repo.

Until/unless drh changes that, you can clone the whole repo, giving you everything in the repo, including historical material:


sudo apt install fossil        # or download from fossil-scm.org, or...
mkdir ~/museum                 # where one keeps Fossils
fossil clone https://www.sqlite.org/docsrc ~/museum/sqlite-docsrc.fossil
mkdir -p ~/src/sqlite/docsrc/trunk
cd ~/src/sqlite/docsrc/trunk
fossil open ~/museum/sqlite-docsrc.fossil

The ~/src/sqlite/docsrc/trunk directory structure allows for things like ~/src/sqlite/docsrc/2019-06-30 or ~/src/sqlite/sqlite3/trunk. You could flatten it if you just want a single SQLite-related Fossil repo checkout.

(7) By Richard Hipp (drh) on 2020-04-15 19:55:31 in reply to 6 [link] [source]

Thanks, Warren. I have now turned on the "z" permission bit on the docsrc repository. The Tarball and ZIP Archive lines should now be available to all.

(8) By doug (doug9forester) on 2020-04-15 20:21:04 in reply to 7 [link] [source]

Thanks, Richard :)

(10) By doug (doug9forester) on 2020-04-21 16:58:04 in reply to 4 [link] [source]

I downloaded the tarball and got the source. I am on W10 so not unix. I installed Linux Ubuntu package, so I have bash. Now I'm trying to build your stuff so I can see how it works (it's not simple). I get an error on the build no matter what make target I use:

$ make schema make: *** No rule to make target '../sqlite/src/test_md5.c', needed by 'tclsh.docsrc'. Stop.

Where is test_md5.c?

(11) By Warren Young (wyoung) on 2020-04-21 17:52:04 in reply to 10 [link] [source]

Where is test_md5.c?

You ignored step 1 on the home page of the docsrc repo: "The SQLite install should be in ../sqlite" etc.

(12.1) Originally by doug (doug9forester) with edits by Richard Hipp (drh) on 2020-04-22 11:28:20 from 12.0 in reply to 11 [source]

Sorry, I must be going blind again. I downloaded the amalgamation but no test_md5.c. In fact, no "src" folder either:

doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc/sqlite$ ls -al total 15744 drwxrwxrwx 1 root root 512 Apr 15 13:25 . drwxrwxrwx 1 root root 512 Apr 15 08:25 .. -rwxrwxrwx 1 root root 4393914 Apr 15 13:16 Documentation+Source+Text-15a047bc79.zip drwxrwxrwx 1 root root 512 Apr 14 22:15 dst -rwxrwxrwx 1 root root 591565 Jan 28 03:25 shell.c -rwxrwxrwx 1 root root 2389909 Apr 21 11:41 sqlite-amalgamation-3310100.zip -rwxrwxrwx 1 root root 8074669 Jan 28 03:25 sqlite3.c -rwxrwxrwx 1 root root 576161 Jan 28 03:25 sqlite3.h -rwxrwxrwx 1 root root 34802 Jan 28 03:25 sqlite3ext.h

(13) By Warren Young (wyoung) on 2020-04-21 19:56:27 in reply to 12.0 [link] [source]

The "source trees" referenced in the same step are not the amalgamations. It's referring to the full and complete source code trees for the software referenced.

For SQLite, that means this: https://sqlite.org/src/

(15.1) Originally by doug (doug9forester) with edits by Richard Hipp (drh) on 2020-04-22 12:09:36 from 15.0 in reply to 13 [link] [source]

The next layer of the onion: missing tclsh.docsrc:

<code>
doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc$ ls -ag
total 0
drwxrwxrwx 1 root 512 Apr 15 08:25 .
drwxrwxrwx 1 root 512 Feb  7  2019 ..
drwxrwxrwx 1 root 512 Apr 21 13:24 archive
drwxrwxrwx 1 root 512 Apr 14 22:15 dst
drwxrwxrwx 1 root 512 Apr 21 07:16 mydoc
drwxrwxrwx 1 root 512 Apr 15 13:25 sqlite
doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc$ cd dst
doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc/dst$ ls -ag
total 304
drwxrwxrwx 1 root   512 Apr 14 22:15 .
drwxrwxrwx 1 root   512 Apr 15 08:25 ..
-rwxrwxrwx 1 root  1888 Apr 21 13:59 Makefile
-rwxrwxrwx 1 root  1509 Apr 14 22:15 README.md
drwxrwxrwx 1 root   512 Apr 14 22:15 art
-rwxrwxrwx 1 root  1180 Apr 14 22:15 common_links.tcl
-rwxrwxrwx 1 root 16408 Apr 14 22:15 custom.txt
drwxrwxrwx 1 root   512 Apr 14 22:15 docapp
-rwxrwxrwx 1 root  4221 Apr 14 22:15 document_header.tcl
-rwxrwxrwx 1 root  3956 Apr 14 22:15 format_evidence.tcl
drwxrwxrwx 1 root   512 Apr 14 22:15 images
-rwxrwxrwx 1 root  3379 Apr 14 22:15 logo.gif
-rwxrwxrwx 1 root  6543 Apr 14 22:15 main.mk
-rwxrwxrwx 1 root 20496 Apr 14 22:15 matrix.tcl
drwxrwxrwx 1 root   512 Apr 14 22:15 misc
-rwxrwxrwx 1 root  1526 Apr 14 22:15 numbering.txt
drwxrwxrwx 1 root   512 Apr 14 22:15 pages
-rwxrwxrwx 1 root   533 Apr 14 22:15 push.sh
drwxrwxrwx 1 root   512 Apr 14 22:15 rawpages
-rwxrwxrwx 1 root   424 Apr 14 22:15 remove_carets.sh
drwxrwxrwx 1 root   512 Apr 14 22:15 req
-rwxrwxrwx 1 root  5266 Apr 14 22:15 scan_test_cases.tcl
-rwxrwxrwx 1 root  4720 Apr 14 22:15 schema.tcl
drwxrwxrwx 1 root   512 Apr 14 22:15 search
-rwxrwxrwx 1 root  2196 Apr 14 22:15 spell_chk.sh
-rwxrwxrwx 1 root 25779 Apr 14 22:15 wrap.tcl
doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc/dst$ make
make base;       # Build base documents
make evidence;   # Gather evidence marks
make matrix;     # Build the traceability matrix
make searchdb;   # Construct the FTS search database
make all;        # Do all of the above
make spell;      # Spell check generated docs
make fast;       # Build documentation only - no requirements
make schema;     # Run once to initialize the build process
doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc/dst$ make schema
make: *** No rule to make target '../sqlite/tclsqlite3.c', needed by 'tclsh.docsrc'.  Stop.

doug@HP610:/mnt/q/qt_projects/dancemusicmanager/doc/sqlite$ ls -al
total 1568
drwxrwxrwx 1 root root    512 Apr 15 13:25 .
drwxrwxrwx 1 root root    512 Apr 15 08:25 ..
drwxrwxrwx 1 root root    512 Apr 21  2020 .fossil-settings
-rwxrwxrwx 1 root root    263 Apr 21  2020 LICENSE.md
-rwxrwxrwx 1 root root  46673 Apr 21  2020 Makefile.in
-rwxrwxrwx 1 root root   3407 Apr 21  2020 Makefile.linux-gcc
-rwxrwxrwx 1 root root  79874 Apr 21  2020 Makefile.msc
-rwxrwxrwx 1 root root  16060 Apr 21  2020 README.md
-rwxrwxrwx 1 root root      7 Apr 21  2020 VERSION
-rwxrwxrwx 1 root root 282502 Apr 21  2020 aclocal.m4
drwxrwxrwx 1 root root    512 Apr 21  2020 art
drwxrwxrwx 1 root root    512 Apr 21  2020 autoconf
-rwxrwxrwx 1 root root  48216 Apr 21  2020 config.guess
-rwxrwxrwx 1 root root   3508 Apr 21  2020 config.h.in
-rwxrwxrwx 1 root root  31617 Apr 21  2020 config.sub
-rwxrwxrwx 1 root root 405966 Apr 21  2020 configure
-rwxrwxrwx 1 root root  22515 Apr 21  2020 configure.ac
drwxrwxrwx 1 root root    512 Apr 21  2020 contrib
drwxrwxrwx 1 root root    512 Apr 21  2020 doc
drwxrwxrwx 1 root root    512 Apr 21  2020 ext
-rwxrwxrwx 1 root root   5598 Apr 21  2020 install-sh
-rwxrwxrwx 1 root root 245160 Apr 21  2020 ltmain.sh
-rwxrwxrwx 1 root root   1515 Apr 21  2020 magic.txt
-rwxrwxrwx 1 root root  36183 Apr 21  2020 main.mk
-rwxrwxrwx 1 root root 142032 Apr 21  2020 manifest
-rwxrwxrwx 1 root root     65 Apr 21  2020 manifest.uuid
-rwxrwxrwx 1 root root    937 Apr 21  2020 mkso.sh
drwxrwxrwx 1 root root    512 Apr 21  2020 mptest
-rwxrwxrwx 1 root root   1796 Apr 21  2020 spec.template
-rwxrwxrwx 1 root root    258 Apr 21  2020 sqlite.pc.in
-rwxrwxrwx 1 root root   8928 Apr 21  2020 sqlite3.1
-rwxrwxrwx 1 root root    267 Apr 21  2020 sqlite3.pc.in
drwxrwxrwx 1 root root    512 Apr 21  2020 src
drwxrwxrwx 1 root root    512 Apr 21  2020 test
drwxrwxrwx 1 root root    512 Apr 21  2020 tool
drwxrwxrwx 1 root root    512 Apr 21  2020 vsixtest

(16) By Warren Young (wyoung) on 2020-04-21 21:35:04 in reply to 15.0 [link] [source]

Go into the sqlite directory and say make tclsqlite3.c, then return to the docsrc directory and try again.

(Basically, missing step.)

(17) By doug (doug9forester) on 2020-04-22 04:50:39 in reply to 16 [link] [source]

It turns out that I had to build Sqlite in order to get tclsqlite.c. I was successful in building that and in building the doc files.

However, my goal is to build my own docs. I can't figure out what the minimum set of files and processes are to get something that works. Can you tell me what that minimum set is and what processes I have to run (by hand) to gen the docs, please?

(18.1) By Richard Hipp (drh) on 2020-04-22 10:45:50 edited from 18.0 in reply to 17 [link] [source]

You need two files from docsrc:

  • art/syntax/bubble-generator.tcl
  • art/syntax/bubble-generator-data.tcl

The second file (the "-data.tcl" file) is the one you customize in order to generate your desired bubble diagrams. The first file is code that translates your "-data.tcl" file into art.

You will need "wish" (the Tcl/Tk command-line shell) in order to run the bubble-generator.tcl tool. And you will need "convert" to translate the Postscript output from "wish" into GIF. And you will need "display" to display the generated GIF for verification. Summary of third-party tools needed:

  • wish
  • convert
  • display

I don't recall any other requirements right this moment.

(19) By ddevienne on 2020-04-22 11:18:44 in reply to 18.1 [link] [source]

Now good luck finding these dependencies with such generic names! :)

I tried Googling display and display tool just for fun, and came up dry of course.

(20) By J. King (jking) on 2020-04-22 11:41:30 in reply to 19 [link] [source]

I tried searching for "display(1)" and found a manual page, which stated it was part of imagemagick. Convert may be the same. The names are indeed problematically generic.

(21) By doug (doug9forester) on 2020-04-24 06:46:41 in reply to 18.1 [link] [source]

Thank you, Richard, for how-to generate the bubble diagrams. I can generate the diagrams, but all the text overflows the boundaries of their containers. I changed the font value but it made no difference. Any ideas?

The next request is for the minimum set of files needed to actually generate a multi-page html document which looks like Sqlite's SQL pages. I don't want requirements, nor testing. Just want to document my language. I've spent a couple of days pouring over the files with few results. I know that the lang_xxx files contain the descriptive text and that you use an Sqlite database for the information, but I can't figure out how to populate the database, or even find out how it's used. Can you point me in the right direction? Thanks.

(22) By nor on 2020-06-30 10:21:10 in reply to 21 [link] [source]

Like doug9forester, I have a project for which this style of diagram seems ideal, and have encountered the same trouble with captions not fitting within containers. I feel there should (or even must) be some parameter which sets limits to the "ballooning", as I notice that the containers are shrunk to fit any very short captions.

I'm also wondering how to specify a rectangular shape for some of the bubbles.

Exploring the code, I've worked out how to specify the -data file on the command line, and also how to hand-code the diagrams in this file. Details are below, in the hope that someone may find this information helpful.

At the top of the file bubble-generator.tcl, Instead of

set datafile [file join [file dirname [info script]] bubble-generator-data.tcl]
source $datafile

I've put the following

if {$argc == 0} {
set datafile [file join [file dirname [info script]] bubble-generator-data.tcl]
} else {
set datafile [lindex $argv 0]
}
source $datafile

And my data file currently looks like what appears below.

# Graphs:
#
set all_graphs {
  ipv4-resolution {
      line /hostname
      {or
          nil
          {line {toploop
                  {or
                      CNAME
                      DNAME
                  }}}} A /ipv4-address-set
  }
  ipv6-resolution {
      line /hostname
      {or
          nil
          {line {loop
                  {or
                      CNAME
                      DNAME
                      }}}} AAAA /ipv6-address-set
  }
}

(14) By Larry Brasfield (LarryBrasfield) on 2020-04-21 20:08:08 in reply to 12.0 [link] [source]

The small bundle of files referred to as "the amalgamation" does not contain source files such as test_md5.c ; it only has

> unzip -v sqlite-amalgamation-3300100.zip
Archive:  sqlite-amalgamation-3300100.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
       0  Stored        0   0% 10/11/2019 07:26 00000000  sqlite-amalgamation-3300100/
 7947211  Defl:N  2050426  74% 10/11/2019 07:26 beace389  sqlite-amalgamation-3300100/sqlite3.c
  588681  Defl:N   148854  75% 10/11/2019 07:26 977b314b  sqlite-amalgamation-3300100/shell.c
   34156  Defl:N     6012  82% 10/11/2019 07:26 cd5c3e82  sqlite-amalgamation-3300100/sqlite3ext.h
  564824  Defl:N   146169  74% 10/11/2019 07:26 29978aed  sqlite-amalgamation-3300100/sqlite3.h
--------          -------  ---                            -------
 9134872          2351461  74%                            5 files

>

Peruse near the bottom of the download page for "Source Code Repositories". From there, you should be able to get to the files that go into the amalgamation.

(9) By anonymous on 2020-04-15 23:21:33 in reply to 2 [link] [source]

Is there an implementation of that syntax diagram maker in PostScript?

(23) By marco.m on 2020-06-30 16:45:16 in reply to 1 [link] [source]

The SQLite diagrams are called "syntax diagrams" or "railroad diagrams": wikipedia.

Using a search engine for "railroad diagram" shows many ready-to-use tools in different programming languages.