SQLite Forum

Timeline
Login

7 forum posts by user jlammens

2021-05-28
15:34 Reply: partial file load? (artifact: 5aba99cd59 user: jlammens)

You are all right - there was indeed an intervening process (gunzipping the db file before it was opened) that was responsible for the delay. On an uncompressed db file the response time is very good.

Apologies for the confusion, and thanks all for your help! Johan

15:15 Reply: partial file load? (artifact: 58cd4873a3 user: jlammens)

Thanks for looking into this, I will need to investigate further.

15:13 Reply: partial file load? (artifact: 10af5ea7a3 user: jlammens)

Yes, sqlite3.exe shows the same behavior.

15:12 Reply: partial file load? (artifact: 7ff52cb305 user: jlammens)

OK, thanks for your response.

14:35 Reply: partial file load? (artifact: 23f41177cc user: jlammens)

see my response just before; here is a formatted version of the script:

SELECT 
  'n'||'='||count(*)||'+'||group_concat(namelen,'+') as db_schema
FROM (
  select name, name||'='||length(sql) as namelen, type from sqlite_master
  WHERE type = 'table' and name not like 'Persistent%'
  order by name
  )
group by type
;
14:29 Reply: partial file load? (artifact: 22dab62d55 user: jlammens)

What leads me to believe that is the time it takes to open the db file in any client I've tried (e.g. SQLite Expert or python sqlite3 interface), and the corresponding disk activity I see until the database becomes available. Large db files (several GB) take minutes, small files (few MB) take seconds. The sequence of operations is open db file, run the sql script below, close db file:

SELECT 'n'||'='||count(*)||'+'||group_concat(namelen,'+') as db_schema FROM ( select name, name||'='||length(sql) as namelen, type from sqlite_master WHERE type = 'table' and name not like 'Persistent%' order by name ) group by type ;

13:58 Post: partial file load? (artifact: 0aa5e33abc user: jlammens)

Is it possible to partially load a database file upon opening it, rather than reading it completely into memory? I have a use case where I need to access just one small table from a series of database files, some of which are quite large, and the process takes long because apparently the entire file is loaded before any table can be accessed. In particular, I only need to access the sqlite_master table from these files.

Thanks for any and all help, Johan