SQLite Forum

Defragment a table ?
Login
Below are what I think are the relevant snippets from the analyzer, table A reports about 50% unused bytes in its pages, while table B reports just 0.3%...
Table A pages are significantly non-sequential. I am currently using 4k pages (running under Windows), would using larger pages help ?

The database as a whole is at 4.6% unused bytes, which is quite good, and table A is the only one with such a ratio of unused bytes, but it is also the only really "dynamically updated" table, other tables are more like indexed logs or regularly regenerated views like table B.

I can send you the whole file by mail if needed.

Thanks

~~~~~

/** Disk-Space Utilization Report

Page size in bytes................................ 4096      
Pages in the whole file (measured)................ 90532790  
Pages in the whole file (calculated).............. 90528869  
Pages that store data............................. 90528868    99.996% 
Pages on the freelist (per header)................ 0            0.0% 
Pages on the freelist (calculated)................ 3922         0.004% 
Pages of auto-vacuum overhead..................... 0            0.0% 
Number of tables in the database.................. 45        
Number of indices................................. 23        
Number of defined indices......................... 14        
Number of implied indices......................... 9         
Size of the file in bytes......................... 370822307840
Bytes of user payload stored...................... 124189467598  33.5% 

*** All tables and indices ****************************************************

Percentage of total database......................  99.996%  
Number of entries................................. 14551461088
Bytes of storage consumed......................... 370806243328
Bytes of payload.................................. 285702356053  77.0% 
Bytes of metadata................................. 68107134984  18.4% 
Average payload per entry......................... 19.63     
Average unused bytes per entry.................... 1.17      
Average metadata per entry........................ 4.68      
Average fanout.................................... 175.00    
Maximum payload per entry......................... 4033      
Entries that use overflow......................... 7            0.0% 
Index pages used.................................. 516031    
Primary pages used................................ 90012830  
Overflow pages used............................... 7         
Total pages used.................................. 90528868  
Unused bytes on index pages....................... 228253065   10.8% 
Unused bytes on primary pages..................... 16768481486   4.5% 
Unused bytes on overflow pages.................... 17740       61.9% 
Unused bytes on all pages......................... 16996752291   4.6% 

*** All tables ****************************************************************

Percentage of total database......................  44.7%    
Number of entries................................. 4881591576
Bytes of storage consumed......................... 165647638528
Bytes of payload.................................. 124189475776  75.0% 
Bytes of metadata................................. 38496387527  23.2% 
Average payload per entry......................... 25.44     
Average unused bytes per entry.................... 0.61      
Average metadata per entry........................ 7.89      
Average fanout.................................... 333.00    
Maximum payload per entry......................... 4030      
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 121285    
Primary pages used................................ 40320033  
Overflow pages used............................... 0         
Total pages used.................................. 40441318  
Unused bytes on index pages....................... 63087082    12.7% 
Unused bytes on primary pages..................... 2898688143   1.8% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 2961775225   1.8% 

*** All indices ***************************************************************

Percentage of total database......................  55.3%    
Number of entries................................. 9669869512
Bytes of storage consumed......................... 205158604800
Bytes of payload.................................. 161512880277  78.7% 
Bytes of metadata................................. 29610747457  14.4% 
Average payload per entry......................... 16.70     
Average unused bytes per entry.................... 1.45      
Average metadata per entry........................ 3.06      
Average fanout.................................... 126.00    
Maximum payload per entry......................... 4033      
Entries that use overflow......................... 7            0.0% 
Index pages used.................................. 394746    
Primary pages used................................ 49692797  
Overflow pages used............................... 7         
Total pages used.................................. 50087550  
Unused bytes on index pages....................... 165165983   10.2% 
Unused bytes on primary pages..................... 13869793343   6.8% 
Unused bytes on overflow pages.................... 17740       61.9% 
Unused bytes on all pages......................... 14034977066   6.8% 


*** Table A and all its indices **************************************

Percentage of total database......................   0.36%   
Number of entries................................. 61907152  
Bytes of storage consumed......................... 1347289088
Bytes of payload.................................. 476164577   35.3% 
Bytes of metadata................................. 336657088   25.0% 
Average payload per entry......................... 7.69      
Average unused bytes per entry.................... 8.63      
Average metadata per entry........................ 5.44      
Average fanout.................................... 193.00    
Maximum payload per entry......................... 13        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1702      
Primary pages used................................ 327226    
Overflow pages used............................... 0         
Total pages used.................................. 328928    
Unused bytes on index pages....................... 2709763     38.9% 
Unused bytes on primary pages..................... 531757660   39.7% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 534467423   39.7% 

*** Table A w/o any indices ******************************************

Percentage of total database......................   0.23%   
Number of entries................................. 30953576  
Bytes of storage consumed......................... 847339520 
Bytes of payload.................................. 176578512   20.8% 
Bytes of metadata................................. 242331668   28.6% 
B-tree depth...................................... 4         
Average payload per entry......................... 5.70      
Average unused bytes per entry.................... 13.84     
Average metadata per entry........................ 7.83      
Average fanout.................................... 185.00    
Non-sequential pages.............................. 101735      49.2% 
Maximum payload per entry......................... 9         
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1113      
Primary pages used................................ 205757    
Overflow pages used............................... 0         
Total pages used.................................. 206870    
Unused bytes on index pages....................... 2335945     51.2% 
Unused bytes on primary pages..................... 426093395   50.6% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 428429340   50.6% 


*** Table B and all its indices ****************************************

Percentage of total database......................   0.41%   
Number of entries................................. 92780529  
Bytes of storage consumed......................... 1501921280
Bytes of payload.................................. 1093102743  72.8% 
Bytes of metadata................................. 405432075   27.0% 
Average payload per entry......................... 11.78     
Average unused bytes per entry.................... 0.04      
Average metadata per entry........................ 4.37      
Average fanout.................................... 295.00    
Maximum payload per entry......................... 21        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 1241      
Primary pages used................................ 365439    
Overflow pages used............................... 0         
Total pages used.................................. 366680    
Unused bytes on index pages....................... 286291       5.6% 
Unused bytes on primary pages..................... 3100171      0.21% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 3386462      0.23% 

*** Table B w/o any indices ********************************************

Percentage of total database......................   0.20%   
Number of entries................................. 30926843  
Bytes of storage consumed......................... 758759424 
Bytes of payload.................................. 538805686   71.0% 
Bytes of metadata................................. 217693793   28.7% 
B-tree depth...................................... 4         
Average payload per entry......................... 17.42     
Average unused bytes per entry.................... 0.07      
Average metadata per entry........................ 7.04      
Average fanout.................................... 358.00    
Non-sequential pages.............................. 10           0.005% 
Maximum payload per entry......................... 21        
Entries that use overflow......................... 0            0.0% 
Index pages used.................................. 516       
Primary pages used................................ 184728    
Overflow pages used............................... 0         
Total pages used.................................. 185244    
Unused bytes on index pages....................... 271807      12.9% 
Unused bytes on primary pages..................... 1988138      0.26% 
Unused bytes on overflow pages.................... 0         
Unused bytes on all pages......................... 2259945      0.30%
~~~~~