/ Check-in [3a4751a9]
Login
SQLite training in Houston TX on 2019-11-05 (details)
Part of the 2019 Tcl Conference

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge enhancements from trunk
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | hard-heap-limit
Files: files | file ages | folders
SHA3-256: 3a4751a9f2784131f81071305b838caa63410a76533fb879627e1849d626f893
User & Date: drh 2019-05-02 14:15:12
Context
2019-08-20
17:14
Bring the hard-heap-limit branch up-to-date with trunk. Leaf check-in: 9b14eb77 user: drh tags: hard-heap-limit
2019-05-02
14:15
Merge enhancements from trunk check-in: 3a4751a9 user: drh tags: hard-heap-limit
01:41
The "LIKE Optimization" (which converts a LIKE into a BETWEEN operator that is able to use an index) does not work and cannot be used if the collating sequence of the column is not TEXT and if the pattern starts with a "+" sign. This is another case of ticket [c94369cae9b561b1f996d005]. The new test case was discovered by Manuel Rigger. check-in: b043a54c user: drh tags: trunk
2019-04-25
18:15
Add the sqlite3_hard_heap_limit64() interface and the corresponding "PRAGMA hard_heap_limit=N" command. check-in: b0ccef61 user: drh tags: hard-heap-limit
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Added ext/misc/dbdata.c.





























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
/*
** 2019-04-17
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
******************************************************************************
**
** This file contains an implementation of two eponymous virtual tables,
** "sqlite_dbdata" and "sqlite_dbptr". Both modules require that the
** "sqlite_dbpage" eponymous virtual table be available.
**
** SQLITE_DBDATA:
**   sqlite_dbdata is used to extract data directly from a database b-tree
**   page and its associated overflow pages, bypassing the b-tree layer.
**   The table schema is equivalent to:
**
**     CREATE TABLE sqlite_dbdata(
**       pgno INTEGER,
**       cell INTEGER,
**       field INTEGER,
**       value ANY,
**       schema TEXT HIDDEN
**     );
**
**   IMPORTANT: THE VIRTUAL TABLE SCHEMA ABOVE IS SUBJECT TO CHANGE. IN THE
**   FUTURE NEW NON-HIDDEN COLUMNS MAY BE ADDED BETWEEN "value" AND
**   "schema".
**
**   Each page of the database is inspected. If it cannot be interpreted as
**   a b-tree page, or if it is a b-tree page containing 0 entries, the
**   sqlite_dbdata table contains no rows for that page.  Otherwise, the
**   table contains one row for each field in the record associated with
**   each cell on the page. For intkey b-trees, the key value is stored in
**   field -1.
**
**   For example, for the database:
**
**     CREATE TABLE t1(a, b);     -- root page is page 2
**     INSERT INTO t1(rowid, a, b) VALUES(5, 'v', 'five');
**     INSERT INTO t1(rowid, a, b) VALUES(10, 'x', 'ten');
**
**   the sqlite_dbdata table contains, as well as from entries related to 
**   page 1, content equivalent to:
**
**     INSERT INTO sqlite_dbdata(pgno, cell, field, value) VALUES
**         (2, 0, -1, 5     ),
**         (2, 0,  0, 'v'   ),
**         (2, 0,  1, 'five'),
**         (2, 1, -1, 10    ),
**         (2, 1,  0, 'x'   ),
**         (2, 1,  1, 'ten' );
**
**   If database corruption is encountered, this module does not report an
**   error. Instead, it attempts to extract as much data as possible and
**   ignores the corruption.
**
** SQLITE_DBPTR:
**   The sqlite_dbptr table has the following schema:
**
**     CREATE TABLE sqlite_dbptr(
**       pgno INTEGER,
**       child INTEGER,
**       schema TEXT HIDDEN
**     );
**
**   It contains one entry for each b-tree pointer between a parent and
**   child page in the database.
*/
#if !defined(SQLITEINT_H) 
#include "sqlite3ext.h"

typedef unsigned char u8;

#endif
SQLITE_EXTENSION_INIT1
#include <string.h>
#include <assert.h>

typedef struct DbdataTable DbdataTable;
typedef struct DbdataCursor DbdataCursor;

/* Cursor object */
struct DbdataCursor {
  sqlite3_vtab_cursor base;       /* Base class.  Must be first */
  sqlite3_stmt *pStmt;            /* For fetching database pages */

  int iPgno;                      /* Current page number */
  u8 *aPage;                      /* Buffer containing page */
  int nPage;                      /* Size of aPage[] in bytes */
  int nCell;                      /* Number of cells on aPage[] */
  int iCell;                      /* Current cell number */
  int bOnePage;                   /* True to stop after one page */
  int szDb;
  sqlite3_int64 iRowid;

  /* Only for the sqlite_dbdata table */
  u8 *pRec;                       /* Buffer containing current record */
  int nRec;                       /* Size of pRec[] in bytes */
  int nHdr;                       /* Size of header in bytes */
  int iField;                     /* Current field number */
  u8 *pHdrPtr;
  u8 *pPtr;
  
  sqlite3_int64 iIntkey;          /* Integer key value */
};

/* Table object */
struct DbdataTable {
  sqlite3_vtab base;              /* Base class.  Must be first */
  sqlite3 *db;                    /* The database connection */
  sqlite3_stmt *pStmt;            /* For fetching database pages */
  int bPtr;                       /* True for sqlite3_dbptr table */
};

/* Column and schema definitions for sqlite_dbdata */
#define DBDATA_COLUMN_PGNO        0
#define DBDATA_COLUMN_CELL        1
#define DBDATA_COLUMN_FIELD       2
#define DBDATA_COLUMN_VALUE       3
#define DBDATA_COLUMN_SCHEMA      4
#define DBDATA_SCHEMA             \
      "CREATE TABLE x("           \
      "  pgno INTEGER,"           \
      "  cell INTEGER,"           \
      "  field INTEGER,"          \
      "  value ANY,"              \
      "  schema TEXT HIDDEN"      \
      ")"

/* Column and schema definitions for sqlite_dbptr */
#define DBPTR_COLUMN_PGNO         0
#define DBPTR_COLUMN_CHILD        1
#define DBPTR_COLUMN_SCHEMA       2
#define DBPTR_SCHEMA              \
      "CREATE TABLE x("           \
      "  pgno INTEGER,"           \
      "  child INTEGER,"          \
      "  schema TEXT HIDDEN"      \
      ")"

/*
** Connect to an sqlite_dbdata (pAux==0) or sqlite_dbptr (pAux!=0) virtual 
** table.
*/
static int dbdataConnect(
  sqlite3 *db,
  void *pAux,
  int argc, const char *const*argv,
  sqlite3_vtab **ppVtab,
  char **pzErr
){
  DbdataTable *pTab = 0;
  int rc = sqlite3_declare_vtab(db, pAux ? DBPTR_SCHEMA : DBDATA_SCHEMA);

  if( rc==SQLITE_OK ){
    pTab = (DbdataTable*)sqlite3_malloc64(sizeof(DbdataTable));
    if( pTab==0 ){
      rc = SQLITE_NOMEM;
    }else{
      memset(pTab, 0, sizeof(DbdataTable));
      pTab->db = db;
      pTab->bPtr = (pAux!=0);
    }
  }

  *ppVtab = (sqlite3_vtab*)pTab;
  return rc;
}

/*
** Disconnect from or destroy a sqlite_dbdata or sqlite_dbptr virtual table.
*/
static int dbdataDisconnect(sqlite3_vtab *pVtab){
  DbdataTable *pTab = (DbdataTable*)pVtab;
  if( pTab ){
    sqlite3_finalize(pTab->pStmt);
    sqlite3_free(pVtab);
  }
  return SQLITE_OK;
}

/*
** This function interprets two types of constraints:
**
**       schema=?
**       pgno=?
**
** If neither are present, idxNum is set to 0. If schema=? is present,
** the 0x01 bit in idxNum is set. If pgno=? is present, the 0x02 bit
** in idxNum is set.
**
** If both parameters are present, schema is in position 0 and pgno in
** position 1.
*/
static int dbdataBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pIdx){
  DbdataTable *pTab = (DbdataTable*)tab;
  int i;
  int iSchema = -1;
  int iPgno = -1;
  int colSchema = (pTab->bPtr ? DBPTR_COLUMN_SCHEMA : DBDATA_COLUMN_SCHEMA);

  for(i=0; i<pIdx->nConstraint; i++){
    struct sqlite3_index_constraint *p = &pIdx->aConstraint[i];
    if( p->op==SQLITE_INDEX_CONSTRAINT_EQ ){
      if( p->iColumn==colSchema ){
        if( p->usable==0 ) return SQLITE_CONSTRAINT;
        iSchema = i;
      }
      if( p->iColumn==DBDATA_COLUMN_PGNO && p->usable ){
        iPgno = i;
      }
    }
  }

  if( iSchema>=0 ){
    pIdx->aConstraintUsage[iSchema].argvIndex = 1;
    pIdx->aConstraintUsage[iSchema].omit = 1;
  }
  if( iPgno>=0 ){
    pIdx->aConstraintUsage[iPgno].argvIndex = 1 + (iSchema>=0);
    pIdx->aConstraintUsage[iPgno].omit = 1;
    pIdx->estimatedCost = 100;
    pIdx->estimatedRows =  50;

    if( pTab->bPtr==0 && pIdx->nOrderBy && pIdx->aOrderBy[0].desc==0 ){
      int iCol = pIdx->aOrderBy[0].iColumn;
      if( pIdx->nOrderBy==1 ){
        pIdx->orderByConsumed = (iCol==0 || iCol==1);
      }else if( pIdx->nOrderBy==2 && pIdx->aOrderBy[1].desc==0 && iCol==0 ){
        pIdx->orderByConsumed = (pIdx->aOrderBy[1].iColumn==1);
      }
    }

  }else{
    pIdx->estimatedCost = 100000000;
    pIdx->estimatedRows = 1000000000;
  }
  pIdx->idxNum = (iSchema>=0 ? 0x01 : 0x00) | (iPgno>=0 ? 0x02 : 0x00);
  return SQLITE_OK;
}

/*
** Open a new sqlite_dbdata or sqlite_dbptr cursor.
*/
static int dbdataOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
  DbdataCursor *pCsr;

  pCsr = (DbdataCursor*)sqlite3_malloc64(sizeof(DbdataCursor));
  if( pCsr==0 ){
    return SQLITE_NOMEM;
  }else{
    memset(pCsr, 0, sizeof(DbdataCursor));
    pCsr->base.pVtab = pVTab;
  }

  *ppCursor = (sqlite3_vtab_cursor *)pCsr;
  return SQLITE_OK;
}

/*
** Restore a cursor object to the state it was in when first allocated 
** by dbdataOpen().
*/
static void dbdataResetCursor(DbdataCursor *pCsr){
  DbdataTable *pTab = (DbdataTable*)(pCsr->base.pVtab);
  if( pTab->pStmt==0 ){
    pTab->pStmt = pCsr->pStmt;
  }else{
    sqlite3_finalize(pCsr->pStmt);
  }
  pCsr->pStmt = 0;
  pCsr->iPgno = 1;
  pCsr->iCell = 0;
  pCsr->iField = 0;
  pCsr->bOnePage = 0;
}

/*
** Close an sqlite_dbdata or sqlite_dbptr cursor.
*/
static int dbdataClose(sqlite3_vtab_cursor *pCursor){
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
  dbdataResetCursor(pCsr);
  sqlite3_free(pCsr);
  return SQLITE_OK;
}

/* 
** Utility methods to decode 16 and 32-bit big-endian unsigned integers. 
*/
static unsigned int get_uint16(unsigned char *a){
  return (a[0]<<8)|a[1];
}
static unsigned int get_uint32(unsigned char *a){
  return (a[0]<<24)|(a[1]<<16)|(a[2]<<8)|a[3];
}

/*
** Load page pgno from the database via the sqlite_dbpage virtual table.
** If successful, set (*ppPage) to point to a buffer containing the page
** data, (*pnPage) to the size of that buffer in bytes and return
** SQLITE_OK. In this case it is the responsibility of the caller to
** eventually free the buffer using sqlite3_free().
**
** Or, if an error occurs, set both (*ppPage) and (*pnPage) to 0 and
** return an SQLite error code.
*/
static int dbdataLoadPage(
  DbdataCursor *pCsr,             /* Cursor object */
  unsigned int pgno,              /* Page number of page to load */
  u8 **ppPage,                    /* OUT: pointer to page buffer */
  int *pnPage                     /* OUT: Size of (*ppPage) in bytes */
){
  int rc2;
  int rc = SQLITE_OK;
  sqlite3_stmt *pStmt = pCsr->pStmt;

  *ppPage = 0;
  *pnPage = 0;
  sqlite3_bind_int64(pStmt, 2, pgno);
  if( SQLITE_ROW==sqlite3_step(pStmt) ){
    int nCopy = sqlite3_column_bytes(pStmt, 0);
    if( nCopy>0 ){
      u8 *pPage;
      pPage = (u8*)sqlite3_malloc64(nCopy);
      if( pPage==0 ){
        rc = SQLITE_NOMEM;
      }else{
        const u8 *pCopy = sqlite3_column_blob(pStmt, 0);
        memcpy(pPage, pCopy, nCopy);
      }
      *ppPage = pPage;
      *pnPage = nCopy;
    }
  }
  rc2 = sqlite3_reset(pStmt);
  if( rc==SQLITE_OK ) rc = rc2;

  return rc;
}

/*
** Read a varint.  Put the value in *pVal and return the number of bytes.
*/
static int dbdataGetVarint(const u8 *z, sqlite3_int64 *pVal){
  sqlite3_int64 v = 0;
  int i;
  for(i=0; i<8; i++){
    v = (v<<7) + (z[i]&0x7f);
    if( (z[i]&0x80)==0 ){ *pVal = v; return i+1; }
  }
  v = (v<<8) + (z[i]&0xff);
  *pVal = v;
  return 9;
}

/*
** Return the number of bytes of space used by an SQLite value of type
** eType.
*/
static int dbdataValueBytes(int eType){
  switch( eType ){
    case 0: case 8: case 9:
    case 10: case 11:
      return 0;
    case 1:
      return 1;
    case 2:
      return 2;
    case 3:
      return 3;
    case 4:
      return 4;
    case 5:
      return 6;
    case 6:
    case 7:
      return 8;
    default:
      return ((eType-12) / 2);
  }
}

/*
** Load a value of type eType from buffer pData and use it to set the
** result of context object pCtx.
*/
static void dbdataValue(sqlite3_context *pCtx, int eType, u8 *pData){
  switch( eType ){
    case 0: 
    case 10: 
    case 11: 
      sqlite3_result_null(pCtx);
      break;
    
    case 8: 
      sqlite3_result_int(pCtx, 0);
      break;
    case 9:
      sqlite3_result_int(pCtx, 1);
      break;

    case 1: case 2: case 3: case 4: case 5: case 6: case 7: {
      sqlite3_uint64 v = (signed char)pData[0];
      pData++;
      switch( eType ){
        case 7:
        case 6:  v = (v<<16) + (pData[0]<<8) + pData[1];  pData += 2;
        case 5:  v = (v<<16) + (pData[0]<<8) + pData[1];  pData += 2;
        case 4:  v = (v<<8) + pData[0];  pData++;
        case 3:  v = (v<<8) + pData[0];  pData++;
        case 2:  v = (v<<8) + pData[0];  pData++;
      }

      if( eType==7 ){
        double r;
        memcpy(&r, &v, sizeof(r));
        sqlite3_result_double(pCtx, r);
      }else{
        sqlite3_result_int64(pCtx, (sqlite3_int64)v);
      }
      break;
    }

    default: {
      int n = ((eType-12) / 2);
      if( eType % 2 ){
        sqlite3_result_text(pCtx, (const char*)pData, n, SQLITE_TRANSIENT);
      }else{
        sqlite3_result_blob(pCtx, pData, n, SQLITE_TRANSIENT);
      }
    }
  }
}


/*
** Move an sqlite_dbdata or sqlite_dbptr cursor to the next entry.
*/
static int dbdataNext(sqlite3_vtab_cursor *pCursor){
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;

  pCsr->iRowid++;
  while( 1 ){
    int rc;
    int iOff = (pCsr->iPgno==1 ? 100 : 0);

    if( pCsr->aPage==0 ){
      while( 1 ){
        if( pCsr->bOnePage==0 && pCsr->iPgno>pCsr->szDb ) return SQLITE_OK;
        rc = dbdataLoadPage(pCsr, pCsr->iPgno, &pCsr->aPage, &pCsr->nPage);
        if( rc!=SQLITE_OK ) return rc;
        if( pCsr->aPage ) break;
        pCsr->iPgno++;
      }
      pCsr->iCell = pTab->bPtr ? -2 : 0;
      pCsr->nCell = get_uint16(&pCsr->aPage[iOff+3]);
    }

    if( pTab->bPtr ){
      if( pCsr->aPage[iOff]!=0x02 && pCsr->aPage[iOff]!=0x05 ){
        pCsr->iCell = pCsr->nCell;
      }
      pCsr->iCell++;
      if( pCsr->iCell>=pCsr->nCell ){
        sqlite3_free(pCsr->aPage);
        pCsr->aPage = 0;
        if( pCsr->bOnePage ) return SQLITE_OK;
        pCsr->iPgno++;
      }else{
        return SQLITE_OK;
      }
    }else{
      /* If there is no record loaded, load it now. */
      if( pCsr->pRec==0 ){
        int bHasRowid = 0;
        int nPointer = 0;
        sqlite3_int64 nPayload = 0;
        sqlite3_int64 nHdr = 0;
        int iHdr;
        int U, X;
        int nLocal;
  
        switch( pCsr->aPage[iOff] ){
          case 0x02:
            nPointer = 4;
            break;
          case 0x0a:
            break;
          case 0x0d:
            bHasRowid = 1;
            break;
          default:
            /* This is not a b-tree page with records on it. Continue. */
            pCsr->iCell = pCsr->nCell;
            break;
        }

        if( pCsr->iCell>=pCsr->nCell ){
          sqlite3_free(pCsr->aPage);
          pCsr->aPage = 0;
          if( pCsr->bOnePage ) return SQLITE_OK;
          pCsr->iPgno++;
          continue;
        }
  
        iOff += 8 + nPointer + pCsr->iCell*2;
        iOff = get_uint16(&pCsr->aPage[iOff]);
  
        /* For an interior node cell, skip past the child-page number */
        iOff += nPointer;
  
        /* Load the "byte of payload including overflow" field */
        iOff += dbdataGetVarint(&pCsr->aPage[iOff], &nPayload);
  
        /* If this is a leaf intkey cell, load the rowid */
        if( bHasRowid ){
          iOff += dbdataGetVarint(&pCsr->aPage[iOff], &pCsr->iIntkey);
        }
  
        /* Allocate space for payload */
        pCsr->pRec = (u8*)sqlite3_malloc64(nPayload);
        if( pCsr->pRec==0 ) return SQLITE_NOMEM;
        pCsr->nRec = nPayload;
  
        U = pCsr->nPage;
        if( bHasRowid ){
          X = U-35;
        }else{
          X = ((U-12)*64/255)-23;
        }
        if( nPayload<=X ){
          nLocal = nPayload;
        }else{
          int M, K;
          M = ((U-12)*32/255)-23;
          K = M+((nPayload-M)%(U-4));
          if( K<=X ){
            nLocal = K;
          }else{
            nLocal = M;
          }
        }
  
        /* Load the nLocal bytes of payload */
        memcpy(pCsr->pRec, &pCsr->aPage[iOff], nLocal);
        iOff += nLocal;
  
        /* Load content from overflow pages */
        if( nPayload>nLocal ){
          sqlite3_int64 nRem = nPayload - nLocal;
          unsigned int pgnoOvfl = get_uint32(&pCsr->aPage[iOff]);
          while( nRem>0 ){
            u8 *aOvfl = 0;
            int nOvfl = 0;
            int nCopy;
            rc = dbdataLoadPage(pCsr, pgnoOvfl, &aOvfl, &nOvfl);
            assert( rc!=SQLITE_OK || nOvfl==pCsr->nPage );
            if( rc!=SQLITE_OK ) return rc;
  
            nCopy = U-4;
            if( nCopy>nRem ) nCopy = nRem;
            memcpy(&pCsr->pRec[nPayload-nRem], &aOvfl[4], nCopy);
            nRem -= nCopy;
  
            pgnoOvfl = get_uint32(aOvfl);
            sqlite3_free(aOvfl);
          }
        }
  
        iHdr = dbdataGetVarint(pCsr->pRec, &nHdr);
        pCsr->nHdr = nHdr;
        pCsr->pHdrPtr = &pCsr->pRec[iHdr];
        pCsr->pPtr = &pCsr->pRec[pCsr->nHdr];
        pCsr->iField = (bHasRowid ? -1 : 0);
      }else{
        pCsr->iField++;
        if( pCsr->iField>0 ){
          sqlite3_int64 iType;
          pCsr->pHdrPtr += dbdataGetVarint(pCsr->pHdrPtr, &iType);
          pCsr->pPtr += dbdataValueBytes(iType);
        }
      }

      if( pCsr->iField<0 || pCsr->pHdrPtr<&pCsr->pRec[pCsr->nHdr] ){
        return SQLITE_OK;
      }
  
      /* Advance to the next cell. The next iteration of the loop will load
      ** the record and so on. */
      sqlite3_free(pCsr->pRec);
      pCsr->pRec = 0;
      pCsr->iCell++;
    }
  }

  assert( !"can't get here" );
  return SQLITE_OK;
}

/* 
** Return true if the cursor is at EOF.
*/
static int dbdataEof(sqlite3_vtab_cursor *pCursor){
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
  return pCsr->aPage==0;
}

/* 
** Determine the size in pages of database zSchema (where zSchema is
** "main", "temp" or the name of an attached database) and set 
** pCsr->szDb accordingly. If successful, return SQLITE_OK. Otherwise,
** an SQLite error code.
*/
static int dbdataDbsize(DbdataCursor *pCsr, const char *zSchema){
  DbdataTable *pTab = (DbdataTable*)pCsr->base.pVtab;
  char *zSql = 0;
  int rc, rc2;
  sqlite3_stmt *pStmt = 0;

  zSql = sqlite3_mprintf("PRAGMA %Q.page_count", zSchema);
  if( zSql==0 ) return SQLITE_NOMEM;
  rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pStmt, 0);
  sqlite3_free(zSql);
  if( rc==SQLITE_OK && sqlite3_step(pStmt)==SQLITE_ROW ){
    pCsr->szDb = sqlite3_column_int(pStmt, 0);
  }
  rc2 = sqlite3_finalize(pStmt);
  if( rc==SQLITE_OK ) rc = rc2;
  return rc;
}

/* 
** xFilter method for sqlite_dbdata and sqlite_dbptr.
*/
static int dbdataFilter(
  sqlite3_vtab_cursor *pCursor, 
  int idxNum, const char *idxStr,
  int argc, sqlite3_value **argv
){
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
  int rc = SQLITE_OK;
  const char *zSchema = "main";

  dbdataResetCursor(pCsr);
  assert( pCsr->iPgno==1 );
  if( idxNum & 0x01 ){
    zSchema = (const char*)sqlite3_value_text(argv[0]);
  }
  if( idxNum & 0x02 ){
    pCsr->iPgno = sqlite3_value_int(argv[(idxNum & 0x01)]);
    pCsr->bOnePage = 1;
  }else{
    pCsr->nPage = dbdataDbsize(pCsr, zSchema);
    rc = dbdataDbsize(pCsr, zSchema);
  }

  if( rc==SQLITE_OK ){
    if( pTab->pStmt ){
      pCsr->pStmt = pTab->pStmt;
      pTab->pStmt = 0;
    }else{
      rc = sqlite3_prepare_v2(pTab->db, 
          "SELECT data FROM sqlite_dbpage(?) WHERE pgno=?", -1,
          &pCsr->pStmt, 0
      );
    }
  }
  if( rc==SQLITE_OK ){
    rc = sqlite3_bind_text(pCsr->pStmt, 1, zSchema, -1, SQLITE_TRANSIENT);
  }else{
    pTab->base.zErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pTab->db));
  }
  if( rc==SQLITE_OK ){
    rc = dbdataNext(pCursor);
  }
  return rc;
}

/* 
** Return a column for the sqlite_dbdata or sqlite_dbptr table.
*/
static int dbdataColumn(
  sqlite3_vtab_cursor *pCursor, 
  sqlite3_context *ctx, 
  int i
){
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
  DbdataTable *pTab = (DbdataTable*)pCursor->pVtab;
  if( pTab->bPtr ){
    switch( i ){
      case DBPTR_COLUMN_PGNO:
        sqlite3_result_int64(ctx, pCsr->iPgno);
        break;
      case DBPTR_COLUMN_CHILD: {
        int iOff = pCsr->iPgno==1 ? 100 : 0;
        if( pCsr->iCell<0 ){
          iOff += 8;
        }else{
          iOff += 12 + pCsr->iCell*2;
          iOff = get_uint16(&pCsr->aPage[iOff]);
        }
        sqlite3_result_int64(ctx, get_uint32(&pCsr->aPage[iOff]));
        break;
      }
    }
  }else{
    switch( i ){
      case DBDATA_COLUMN_PGNO:
        sqlite3_result_int64(ctx, pCsr->iPgno);
        break;
      case DBDATA_COLUMN_CELL:
        sqlite3_result_int(ctx, pCsr->iCell);
        break;
      case DBDATA_COLUMN_FIELD:
        sqlite3_result_int(ctx, pCsr->iField);
        break;
      case DBDATA_COLUMN_VALUE: {
        if( pCsr->iField<0 ){
          sqlite3_result_int64(ctx, pCsr->iIntkey);
        }else{
          sqlite3_int64 iType;
          dbdataGetVarint(pCsr->pHdrPtr, &iType);
          dbdataValue(ctx, iType, pCsr->pPtr);
        }
        break;
      }
    }
  }
  return SQLITE_OK;
}

/* 
** Return the rowid for an sqlite_dbdata or sqlite_dptr table.
*/
static int dbdataRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
  DbdataCursor *pCsr = (DbdataCursor*)pCursor;
  *pRowid = pCsr->iRowid;
  return SQLITE_OK;
}


/*
** Invoke this routine to register the "sqlite_dbdata" virtual table module
*/
static int sqlite3DbdataRegister(sqlite3 *db){
  static sqlite3_module dbdata_module = {
    0,                            /* iVersion */
    0,                            /* xCreate */
    dbdataConnect,                /* xConnect */
    dbdataBestIndex,              /* xBestIndex */
    dbdataDisconnect,             /* xDisconnect */
    0,                            /* xDestroy */
    dbdataOpen,                   /* xOpen - open a cursor */
    dbdataClose,                  /* xClose - close a cursor */
    dbdataFilter,                 /* xFilter - configure scan constraints */
    dbdataNext,                   /* xNext - advance a cursor */
    dbdataEof,                    /* xEof - check for end of scan */
    dbdataColumn,                 /* xColumn - read data */
    dbdataRowid,                  /* xRowid - read data */
    0,                            /* xUpdate */
    0,                            /* xBegin */
    0,                            /* xSync */
    0,                            /* xCommit */
    0,                            /* xRollback */
    0,                            /* xFindMethod */
    0,                            /* xRename */
    0,                            /* xSavepoint */
    0,                            /* xRelease */
    0,                            /* xRollbackTo */
    0                             /* xShadowName */
  };

  int rc = sqlite3_create_module(db, "sqlite_dbdata", &dbdata_module, 0);
  if( rc==SQLITE_OK ){
    rc = sqlite3_create_module(db, "sqlite_dbptr", &dbdata_module, (void*)1);
  }
  return rc;
}

#ifdef _WIN32
__declspec(dllexport)
#endif
int sqlite3_dbdata_init(
  sqlite3 *db, 
  char **pzErrMsg, 
  const sqlite3_api_routines *pApi
){
  SQLITE_EXTENSION_INIT2(pApi);
  return sqlite3DbdataRegister(db);
}

Changes to main.mk.

734
735
736
737
738
739
740

741
742
743
744
745
746
747
	$(TOP)/ext/misc/fileio.c \
	$(TOP)/ext/misc/completion.c \
	$(TOP)/ext/misc/sqlar.c \
	$(TOP)/ext/expert/sqlite3expert.c \
	$(TOP)/ext/expert/sqlite3expert.h \
	$(TOP)/ext/misc/zipfile.c \
	$(TOP)/ext/misc/memtrace.c \

        $(TOP)/src/test_windirent.c

shell.c:	$(SHELL_SRC) $(TOP)/tool/mkshellc.tcl
	tclsh $(TOP)/tool/mkshellc.tcl >shell.c










>







734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
	$(TOP)/ext/misc/fileio.c \
	$(TOP)/ext/misc/completion.c \
	$(TOP)/ext/misc/sqlar.c \
	$(TOP)/ext/expert/sqlite3expert.c \
	$(TOP)/ext/expert/sqlite3expert.h \
	$(TOP)/ext/misc/zipfile.c \
	$(TOP)/ext/misc/memtrace.c \
	$(TOP)/ext/misc/dbdata.c \
        $(TOP)/src/test_windirent.c

shell.c:	$(SHELL_SRC) $(TOP)/tool/mkshellc.tcl
	tclsh $(TOP)/tool/mkshellc.tcl >shell.c



Changes to src/build.c.

1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
....
1722
1723
1724
1725
1726
1727
1728
1729


1730
1731
1732







































1733
1734
1735
1736
1737
1738
1739
....
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840

1841
1842
1843
1844
1845
1846
1847
....
1863
1864
1865
1866
1867
1868
1869

1870


1871
1872
1873
1874
1875
1876
1877
1878

1879
1880
1881
1882
1883
1884
1885
1886
....
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397

3398
3399
3400
3401
3402
3403
3404
**     CREATE TABLE xyz(a,b,c,d,e,PRIMARY KEY('a'),UNIQUE('b','c' COLLATE trim)
**     CREATE INDEX abc ON xyz('c','d' DESC,'e' COLLATE nocase DESC);
**
** This is goofy.  But to preserve backwards compatibility we continue to
** accept it.  This routine does the necessary conversion.  It converts
** the expression given in its argument from a TK_STRING into a TK_ID
** if the expression is just a TK_STRING with an optional COLLATE clause.
** If the epxression is anything other than TK_STRING, the expression is
** unchanged.
*/
static void sqlite3StringToId(Expr *p){
  if( p->op==TK_STRING ){
    p->op = TK_ID;
  }else if( p->op==TK_COLLATE && p->pLeft->op==TK_STRING ){
    p->pLeft->op = TK_ID;
................................................................................
    i16 x = pIdx->aiColumn[i];
    assert( x<pIdx->pTable->nCol );
    wIndex += x<0 ? 1 : aCol[pIdx->aiColumn[i]].szEst;
  }
  pIdx->szIdxRow = sqlite3LogEst(wIndex*4);
}

/* Return true if value x is found any of the first nCol entries of aiCol[]


*/
static int hasColumn(const i16 *aiCol, int nCol, int x){
  while( nCol-- > 0 ) if( x==*(aiCol++) ) return 1;







































  return 0;
}

/* Recompute the colNotIdxed field of the Index.
**
** colNotIdxed is a bitmask that has a 0 bit representing each indexed
** columns that are within the first 63 columns of the table.  The
................................................................................

    /*
    ** Remove all redundant columns from the PRIMARY KEY.  For example, change
    ** "PRIMARY KEY(a,b,a,b,c,b,c,d)" into just "PRIMARY KEY(a,b,c,d)".  Later
    ** code assumes the PRIMARY KEY contains no repeated columns.
    */
    for(i=j=1; i<pPk->nKeyCol; i++){
      if( hasColumn(pPk->aiColumn, j, pPk->aiColumn[i]) ){
        pPk->nColumn--;
      }else{

        pPk->aiColumn[j++] = pPk->aiColumn[i];
      }
    }
    pPk->nKeyCol = j;
  }
  assert( pPk!=0 );
  pPk->isCovering = 1;
................................................................................
  /* Update the in-memory representation of all UNIQUE indices by converting
  ** the final rowid column into one or more columns of the PRIMARY KEY.
  */
  for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
    int n;
    if( IsPrimaryKeyIndex(pIdx) ) continue;
    for(i=n=0; i<nPk; i++){

      if( !hasColumn(pIdx->aiColumn, pIdx->nKeyCol, pPk->aiColumn[i]) ) n++;


    }
    if( n==0 ){
      /* This index is a superset of the primary key */
      pIdx->nColumn = pIdx->nKeyCol;
      continue;
    }
    if( resizeIndexObject(db, pIdx, pIdx->nKeyCol+n) ) return;
    for(i=0, j=pIdx->nKeyCol; i<nPk; i++){

      if( !hasColumn(pIdx->aiColumn, pIdx->nKeyCol, pPk->aiColumn[i]) ){
        pIdx->aiColumn[j] = pPk->aiColumn[i];
        pIdx->azColl[j] = pPk->azColl[i];
        j++;
      }
    }
    assert( pIdx->nColumn>=pIdx->nKeyCol+n );
    assert( pIdx->nColumn>=j );
................................................................................
  ** tables (when pPk!=0) this will be the declared PRIMARY KEY.  For
  ** normal tables (when pPk==0) this will be the rowid.
  */
  if( pPk ){
    for(j=0; j<pPk->nKeyCol; j++){
      int x = pPk->aiColumn[j];
      assert( x>=0 );
      if( hasColumn(pIndex->aiColumn, pIndex->nKeyCol, x) ){
        pIndex->nColumn--; 
      }else{

        pIndex->aiColumn[i] = x;
        pIndex->azColl[i] = pPk->azColl[j];
        pIndex->aSortOrder[i] = pPk->aSortOrder[j];
        i++;
      }
    }
    assert( i==pIndex->nColumn );







|







 







|
>
>


|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|


>







 







>
|
>
>








>
|







 







|


>







1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
....
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
....
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
....
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
....
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
**     CREATE TABLE xyz(a,b,c,d,e,PRIMARY KEY('a'),UNIQUE('b','c' COLLATE trim)
**     CREATE INDEX abc ON xyz('c','d' DESC,'e' COLLATE nocase DESC);
**
** This is goofy.  But to preserve backwards compatibility we continue to
** accept it.  This routine does the necessary conversion.  It converts
** the expression given in its argument from a TK_STRING into a TK_ID
** if the expression is just a TK_STRING with an optional COLLATE clause.
** If the expression is anything other than TK_STRING, the expression is
** unchanged.
*/
static void sqlite3StringToId(Expr *p){
  if( p->op==TK_STRING ){
    p->op = TK_ID;
  }else if( p->op==TK_COLLATE && p->pLeft->op==TK_STRING ){
    p->pLeft->op = TK_ID;
................................................................................
    i16 x = pIdx->aiColumn[i];
    assert( x<pIdx->pTable->nCol );
    wIndex += x<0 ? 1 : aCol[pIdx->aiColumn[i]].szEst;
  }
  pIdx->szIdxRow = sqlite3LogEst(wIndex*4);
}

/* Return true if column number x is any of the first nCol entries of aiCol[].
** This is used to determine if the column number x appears in any of the
** first nCol entries of an index.
*/
static int hasColumn(const i16 *aiCol, int nCol, int x){
  while( nCol-- > 0 ){
    assert( aiCol[0]>=0 );
    if( x==*(aiCol++) ){
      return 1;
    }
  }
  return 0;
}

/*
** Return true if any of the first nKey entries of index pIdx exactly
** match the iCol-th entry of pPk.  pPk is always a WITHOUT ROWID
** PRIMARY KEY index.  pIdx is an index on the same table.  pIdx may
** or may not be the same index as pPk.
**
** The first nKey entries of pIdx are guaranteed to be ordinary columns,
** not a rowid or expression.
**
** This routine differs from hasColumn() in that both the column and the
** collating sequence must match for this routine, but for hasColumn() only
** the column name must match.
*/
static int isDupColumn(Index *pIdx, int nKey, Index *pPk, int iCol){
  int i, j;
  assert( nKey<=pIdx->nColumn );
  assert( iCol<MAX(pPk->nColumn,pPk->nKeyCol) );
  assert( pPk->idxType==SQLITE_IDXTYPE_PRIMARYKEY );
  assert( pPk->pTable->tabFlags & TF_WithoutRowid );
  assert( pPk->pTable==pIdx->pTable );
  testcase( pPk==pIdx );
  j = pPk->aiColumn[iCol];
  assert( j!=XN_ROWID && j!=XN_EXPR );
  for(i=0; i<nKey; i++){
    assert( pIdx->aiColumn[i]>=0 || j>=0 );
    if( pIdx->aiColumn[i]==j 
     && sqlite3StrICmp(pIdx->azColl[i], pPk->azColl[iCol])==0
    ){
      return 1;
    }
  }
  return 0;
}

/* Recompute the colNotIdxed field of the Index.
**
** colNotIdxed is a bitmask that has a 0 bit representing each indexed
** columns that are within the first 63 columns of the table.  The
................................................................................

    /*
    ** Remove all redundant columns from the PRIMARY KEY.  For example, change
    ** "PRIMARY KEY(a,b,a,b,c,b,c,d)" into just "PRIMARY KEY(a,b,c,d)".  Later
    ** code assumes the PRIMARY KEY contains no repeated columns.
    */
    for(i=j=1; i<pPk->nKeyCol; i++){
      if( isDupColumn(pPk, j, pPk, i) ){
        pPk->nColumn--;
      }else{
        testcase( hasColumn(pPk->aiColumn, j, pPk->aiColumn[i]) );
        pPk->aiColumn[j++] = pPk->aiColumn[i];
      }
    }
    pPk->nKeyCol = j;
  }
  assert( pPk!=0 );
  pPk->isCovering = 1;
................................................................................
  /* Update the in-memory representation of all UNIQUE indices by converting
  ** the final rowid column into one or more columns of the PRIMARY KEY.
  */
  for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
    int n;
    if( IsPrimaryKeyIndex(pIdx) ) continue;
    for(i=n=0; i<nPk; i++){
      if( !isDupColumn(pIdx, pIdx->nKeyCol, pPk, i) ){
        testcase( hasColumn(pIdx->aiColumn, pIdx->nKeyCol, pPk->aiColumn[i]) );
        n++;
      }
    }
    if( n==0 ){
      /* This index is a superset of the primary key */
      pIdx->nColumn = pIdx->nKeyCol;
      continue;
    }
    if( resizeIndexObject(db, pIdx, pIdx->nKeyCol+n) ) return;
    for(i=0, j=pIdx->nKeyCol; i<nPk; i++){
      if( !isDupColumn(pIdx, pIdx->nKeyCol, pPk, i) ){
        testcase( hasColumn(pIdx->aiColumn, pIdx->nKeyCol, pPk->aiColumn[i]) );
        pIdx->aiColumn[j] = pPk->aiColumn[i];
        pIdx->azColl[j] = pPk->azColl[i];
        j++;
      }
    }
    assert( pIdx->nColumn>=pIdx->nKeyCol+n );
    assert( pIdx->nColumn>=j );
................................................................................
  ** tables (when pPk!=0) this will be the declared PRIMARY KEY.  For
  ** normal tables (when pPk==0) this will be the rowid.
  */
  if( pPk ){
    for(j=0; j<pPk->nKeyCol; j++){
      int x = pPk->aiColumn[j];
      assert( x>=0 );
      if( isDupColumn(pIndex, pIndex->nKeyCol, pPk, j) ){
        pIndex->nColumn--; 
      }else{
        testcase( hasColumn(pIndex->aiColumn,pIndex->nKeyCol,x) );
        pIndex->aiColumn[i] = x;
        pIndex->azColl[i] = pPk->azColl[j];
        pIndex->aSortOrder[i] = pPk->aSortOrder[j];
        i++;
      }
    }
    assert( i==pIndex->nColumn );

Changes to src/func.c.

839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
...
869
870
871
872
873
874
875


876
877
878
879
880
881
882
#ifdef SQLITE_TEST
    sqlite3_like_count++;
#endif
    sqlite3_result_int(context, 0);
    return;
  }
#endif
  zB = sqlite3_value_text(argv[0]);
  zA = sqlite3_value_text(argv[1]);

  /* Limit the length of the LIKE or GLOB pattern to avoid problems
  ** of deep recursion and N*N behavior in patternCompare().
  */
  nPat = sqlite3_value_bytes(argv[0]);
  testcase( nPat==db->aLimit[SQLITE_LIMIT_LIKE_PATTERN_LENGTH] );
  testcase( nPat==db->aLimit[SQLITE_LIMIT_LIKE_PATTERN_LENGTH]+1 );
  if( nPat > db->aLimit[SQLITE_LIMIT_LIKE_PATTERN_LENGTH] ){
    sqlite3_result_error(context, "LIKE or GLOB pattern too complex", -1);
    return;
  }
  assert( zB==sqlite3_value_text(argv[0]) );  /* Encoding did not change */

  if( argc==3 ){
    /* The escape character string must consist of a single UTF-8 character.
    ** Otherwise, return an error.
    */
    const unsigned char *zEsc = sqlite3_value_text(argv[2]);
    if( zEsc==0 ) return;
    if( sqlite3Utf8CharLen((char*)zEsc, -1)!=1 ){
................................................................................
          "ESCAPE expression must be a single character", -1);
      return;
    }
    escape = sqlite3Utf8Read(&zEsc);
  }else{
    escape = pInfo->matchSet;
  }


  if( zA && zB ){
#ifdef SQLITE_TEST
    sqlite3_like_count++;
#endif
    sqlite3_result_int(context,
                      patternCompare(zB, zA, pInfo, escape)==SQLITE_MATCH);
  }







<
<











<
<







 







>
>







839
840
841
842
843
844
845


846
847
848
849
850
851
852
853
854
855
856


857
858
859
860
861
862
863
...
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
#ifdef SQLITE_TEST
    sqlite3_like_count++;
#endif
    sqlite3_result_int(context, 0);
    return;
  }
#endif



  /* Limit the length of the LIKE or GLOB pattern to avoid problems
  ** of deep recursion and N*N behavior in patternCompare().
  */
  nPat = sqlite3_value_bytes(argv[0]);
  testcase( nPat==db->aLimit[SQLITE_LIMIT_LIKE_PATTERN_LENGTH] );
  testcase( nPat==db->aLimit[SQLITE_LIMIT_LIKE_PATTERN_LENGTH]+1 );
  if( nPat > db->aLimit[SQLITE_LIMIT_LIKE_PATTERN_LENGTH] ){
    sqlite3_result_error(context, "LIKE or GLOB pattern too complex", -1);
    return;
  }


  if( argc==3 ){
    /* The escape character string must consist of a single UTF-8 character.
    ** Otherwise, return an error.
    */
    const unsigned char *zEsc = sqlite3_value_text(argv[2]);
    if( zEsc==0 ) return;
    if( sqlite3Utf8CharLen((char*)zEsc, -1)!=1 ){
................................................................................
          "ESCAPE expression must be a single character", -1);
      return;
    }
    escape = sqlite3Utf8Read(&zEsc);
  }else{
    escape = pInfo->matchSet;
  }
  zB = sqlite3_value_text(argv[0]);
  zA = sqlite3_value_text(argv[1]);
  if( zA && zB ){
#ifdef SQLITE_TEST
    sqlite3_like_count++;
#endif
    sqlite3_result_int(context,
                      patternCompare(zB, zA, pInfo, escape)==SQLITE_MATCH);
  }

Changes to src/resolve.c.

862
863
864
865
866
867
868

869

870
871
872
873
874
875
876
        }
      }
      sqlite3WalkExprList(pWalker, pList);
      if( is_agg ){
#ifndef SQLITE_OMIT_WINDOWFUNC
        if( pExpr->y.pWin ){
          Select *pSel = pNC->pWinSelect;

          sqlite3WindowUpdate(pParse, pSel->pWinDefn, pExpr->y.pWin, pDef);

          sqlite3WalkExprList(pWalker, pExpr->y.pWin->pPartition);
          sqlite3WalkExprList(pWalker, pExpr->y.pWin->pOrderBy);
          sqlite3WalkExpr(pWalker, pExpr->y.pWin->pFilter);
          if( 0==pSel->pWin 
           || 0==sqlite3WindowCompare(pParse, pSel->pWin, pExpr->y.pWin) 
          ){
            pExpr->y.pWin->pNextWin = pSel->pWin;







>
|
>







862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
        }
      }
      sqlite3WalkExprList(pWalker, pList);
      if( is_agg ){
#ifndef SQLITE_OMIT_WINDOWFUNC
        if( pExpr->y.pWin ){
          Select *pSel = pNC->pWinSelect;
          if( IN_RENAME_OBJECT==0 ){
            sqlite3WindowUpdate(pParse, pSel->pWinDefn, pExpr->y.pWin, pDef);
          }
          sqlite3WalkExprList(pWalker, pExpr->y.pWin->pPartition);
          sqlite3WalkExprList(pWalker, pExpr->y.pWin->pOrderBy);
          sqlite3WalkExpr(pWalker, pExpr->y.pWin->pFilter);
          if( 0==pSel->pWin 
           || 0==sqlite3WindowCompare(pParse, pSel->pWin, pExpr->y.pWin) 
          ){
            pExpr->y.pWin->pNextWin = pSel->pWin;

Changes to src/shell.c.in.

943
944
945
946
947
948
949




950
951
952
953
954
955
956
....
3570
3571
3572
3573
3574
3575
3576



3577
3578
3579
3580
3581
3582
3583
....
3926
3927
3928
3929
3930
3931
3932























































































































3933
3934
3935
3936
3937
3938
3939
....
3995
3996
3997
3998
3999
4000
4001



4002
4003
4004
4005
4006
4007
4008
4009
4010
4011




4012
4013
4014
4015
4016
4017
4018
....
5259
5260
5261
5262
5263
5264
5265
5266
5267
5268
5269
5270
5271
5272
5273
5274
5275
5276
....
5333
5334
5335
5336
5337
5338
5339






5340
5341
5342
5343
5344
5345
5346
....
6022
6023
6024
6025
6026
6027
6028

















































































































































































































































































































































































































































































































































































































































6029
6030
6031
6032
6033
6034
6035
....
6309
6310
6311
6312
6313
6314
6315







6316
6317
6318
6319
6320
6321
6322
....
6346
6347
6348
6349
6350
6351
6352

6353

6354
6355
6356
6357
6358
6359
6360
....
6394
6395
6396
6397
6398
6399
6400
6401
6402
6403
6404
6405
6406
6407
6408
INCLUDE ../ext/misc/memtrace.c
#ifdef SQLITE_HAVE_ZLIB
INCLUDE ../ext/misc/zipfile.c
INCLUDE ../ext/misc/sqlar.c
#endif
INCLUDE ../ext/expert/sqlite3expert.h
INCLUDE ../ext/expert/sqlite3expert.c





#if defined(SQLITE_ENABLE_SESSION)
/*
** State information for a single open session
*/
typedef struct OpenSession OpenSession;
struct OpenSession {
................................................................................
  "   --once                    Do no more than one progress interrupt",
  "   --quiet|-q                No output except at interrupts",
  "   --reset                   Reset the count for each input and interrupt",
#endif
  ".prompt MAIN CONTINUE    Replace the standard prompts",
  ".quit                    Exit this program",
  ".read FILE               Read input from FILE",



  ".restore ?DB? FILE       Restore content of DB (default \"main\") from FILE",
  ".save FILE               Write in-memory database into FILE",
  ".scanstats on|off        Turn sqlite3_stmt_scanstatus() metrics on or off",
  ".schema ?PATTERN?        Show the CREATE statements matching PATTERN",
  "     Options:",
  "         --indent            Try to pretty-print the schema",
  ".selftest ?OPTIONS?      Run tests defined in the SELFTEST table",
................................................................................
    p->lineno = nLine;
  }
  sqlite3_free(a);
  utf8_printf(stderr,"Error on line %d of --hexdb input\n", nLine);
  return 0;
}
#endif /* SQLITE_ENABLE_DESERIALIZE */
























































































































/* Flags for open_db().
**
** The default behavior of open_db() is to exit(1) if the database fails to
** open.  The OPEN_DB_KEEPALIVE flag changes that so that it prints an error
** but still returns without calling exit.
**
................................................................................
    }
#ifndef SQLITE_OMIT_LOAD_EXTENSION
    sqlite3_enable_load_extension(p->db, 1);
#endif
    sqlite3_fileio_init(p->db, 0, 0);
    sqlite3_shathree_init(p->db, 0, 0);
    sqlite3_completion_init(p->db, 0, 0);



#ifdef SQLITE_HAVE_ZLIB
    sqlite3_zipfile_init(p->db, 0, 0);
    sqlite3_sqlar_init(p->db, 0, 0);
#endif
    sqlite3_create_function(p->db, "shell_add_schema", 3, SQLITE_UTF8, 0,
                            shellAddSchemaName, 0, 0);
    sqlite3_create_function(p->db, "shell_module_schema", 1, SQLITE_UTF8, 0,
                            shellModuleSchema, 0, 0);
    sqlite3_create_function(p->db, "shell_putsnl", 1, SQLITE_UTF8, p,
                            shellPutsFunc, 0, 0);




#ifndef SQLITE_NOHAVE_SYSTEM
    sqlite3_create_function(p->db, "edit", 1, SQLITE_UTF8, 0,
                            editFunc, 0, 0);
    sqlite3_create_function(p->db, "edit", 2, SQLITE_UTF8, 0,
                            editFunc, 0, 0);
#endif
    if( p->openMode==SHELL_OPEN_ZIPFILE ){
................................................................................
 usage:
  raw_printf(stderr, "Usage %s sub-command ?switches...?\n", azArg[0]);
  raw_printf(stderr, "Where sub-commands are:\n");
  raw_printf(stderr, "    fkey-indexes\n");
  return SQLITE_ERROR;
}

#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
/*********************************************************************************
** The ".archive" or ".ar" command.
*/
static void shellPrepare(
  sqlite3 *db, 
  int *pRc, 
  const char *zSql, 
  sqlite3_stmt **ppStmt
){
  *ppStmt = 0;
................................................................................
    if( rc!=SQLITE_OK ){
      sqlite3 *db = sqlite3_db_handle(pStmt);
      raw_printf(stderr, "SQL error: %s\n", sqlite3_errmsg(db));
    }
    *pRc = rc;
  }
}






/*
** Structure representing a single ".ar" command.
*/
typedef struct ArCommand ArCommand;
struct ArCommand {
  u8 eCmd;                        /* An AR_CMD_* value */
  u8 bVerbose;                    /* True if --verbose */
................................................................................

  return rc;
}
/* End of the ".archive" or ".ar" command logic
**********************************************************************************/
#endif /* !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB) */



















































































































































































































































































































































































































































































































































































































































/*
** If an input line begins with "." then invoke this routine to
** process that line.
**
** Return 1 on error, 2 to exit, and 0 otherwise.
*/
................................................................................
    }   
  }else

  if( c=='d' && n>=3 && strncmp(azArg[0], "dbinfo", n)==0 ){
    rc = shell_dbinfo_command(p, nArg, azArg);
  }else








  if( c=='d' && strncmp(azArg[0], "dump", n)==0 ){
    const char *zLike = 0;
    int i;
    int savedShowHeader = p->showHeader;
    int savedShellFlags = p->shellFlgs;
    ShellClearFlag(p, SHFLG_PreserveRowid|SHFLG_Newlines|SHFLG_Echo);
    for(i=1; i<nArg; i++){
................................................................................
                           "?--newlines? ?LIKE-PATTERN?\n");
        rc = 1;
        goto meta_command_exit;
      }else{
        zLike = azArg[i];
      }
    }

    open_db(p, 0);

    /* When playing back a "dump", the content might appear in an order
    ** which causes immediate foreign key constraints to be violated.
    ** So disable foreign-key constraint enforcement to prevent problems. */
    raw_printf(p->out, "PRAGMA foreign_keys=OFF;\n");
    raw_printf(p->out, "BEGIN TRANSACTION;\n");
    p->writableSchema = 0;
    p->showHeader = 0;
................................................................................
    }
    if( p->writableSchema ){
      raw_printf(p->out, "PRAGMA writable_schema=OFF;\n");
      p->writableSchema = 0;
    }
    sqlite3_exec(p->db, "PRAGMA writable_schema=OFF;", 0, 0, 0);
    sqlite3_exec(p->db, "RELEASE dump;", 0, 0, 0);
    raw_printf(p->out, p->nErr ? "ROLLBACK; -- due to errors\n" : "COMMIT;\n");
    p->showHeader = savedShowHeader;
    p->shellFlgs = savedShellFlags;
  }else

  if( c=='e' && strncmp(azArg[0], "echo", n)==0 ){
    if( nArg==2 ){
      setOrClearFlag(p, SHFLG_Echo, azArg[1]);







>
>
>
>







 







>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>










>
>
>
>







 







|
<
<
<







 







>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>







 







>

>







 







|







943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
....
3574
3575
3576
3577
3578
3579
3580
3581
3582
3583
3584
3585
3586
3587
3588
3589
3590
....
3933
3934
3935
3936
3937
3938
3939
3940
3941
3942
3943
3944
3945
3946
3947
3948
3949
3950
3951
3952
3953
3954
3955
3956
3957
3958
3959
3960
3961
3962
3963
3964
3965
3966
3967
3968
3969
3970
3971
3972
3973
3974
3975
3976
3977
3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992
3993
3994
3995
3996
3997
3998
3999
4000
4001
4002
4003
4004
4005
4006
4007
4008
4009
4010
4011
4012
4013
4014
4015
4016
4017
4018
4019
4020
4021
4022
4023
4024
4025
4026
4027
4028
4029
4030
4031
4032
4033
4034
4035
4036
4037
4038
4039
4040
4041
4042
4043
4044
4045
4046
4047
4048
4049
4050
4051
4052
4053
4054
4055
4056
4057
4058
4059
4060
4061
4062
4063
4064
4065
....
4121
4122
4123
4124
4125
4126
4127
4128
4129
4130
4131
4132
4133
4134
4135
4136
4137
4138
4139
4140
4141
4142
4143
4144
4145
4146
4147
4148
4149
4150
4151
....
5392
5393
5394
5395
5396
5397
5398
5399



5400
5401
5402
5403
5404
5405
5406
....
5463
5464
5465
5466
5467
5468
5469
5470
5471
5472
5473
5474
5475
5476
5477
5478
5479
5480
5481
5482
....
6158
6159
6160
6161
6162
6163
6164
6165
6166
6167
6168
6169
6170
6171
6172
6173
6174
6175
6176
6177
6178
6179
6180
6181
6182
6183
6184
6185
6186
6187
6188
6189
6190
6191
6192
6193
6194
6195
6196
6197
6198
6199
6200
6201
6202
6203
6204
6205
6206
6207
6208
6209
6210
6211
6212
6213
6214
6215
6216
6217
6218
6219
6220
6221
6222
6223
6224
6225
6226
6227
6228
6229
6230
6231
6232
6233
6234
6235
6236
6237
6238
6239
6240
6241
6242
6243
6244
6245
6246
6247
6248
6249
6250
6251
6252
6253
6254
6255
6256
6257
6258
6259
6260
6261
6262
6263
6264
6265
6266
6267
6268
6269
6270
6271
6272
6273
6274
6275
6276
6277
6278
6279
6280
6281
6282
6283
6284
6285
6286
6287
6288
6289
6290
6291
6292
6293
6294
6295
6296
6297
6298
6299
6300
6301
6302
6303
6304
6305
6306
6307
6308
6309
6310
6311
6312
6313
6314
6315
6316
6317
6318
6319
6320
6321
6322
6323
6324
6325
6326
6327
6328
6329
6330
6331
6332
6333
6334
6335
6336
6337
6338
6339
6340
6341
6342
6343
6344
6345
6346
6347
6348
6349
6350
6351
6352
6353
6354
6355
6356
6357
6358
6359
6360
6361
6362
6363
6364
6365
6366
6367
6368
6369
6370
6371
6372
6373
6374
6375
6376
6377
6378
6379
6380
6381
6382
6383
6384
6385
6386
6387
6388
6389
6390
6391
6392
6393
6394
6395
6396
6397
6398
6399
6400
6401
6402
6403
6404
6405
6406
6407
6408
6409
6410
6411
6412
6413
6414
6415
6416
6417
6418
6419
6420
6421
6422
6423
6424
6425
6426
6427
6428
6429
6430
6431
6432
6433
6434
6435
6436
6437
6438
6439
6440
6441
6442
6443
6444
6445
6446
6447
6448
6449
6450
6451
6452
6453
6454
6455
6456
6457
6458
6459
6460
6461
6462
6463
6464
6465
6466
6467
6468
6469
6470
6471
6472
6473
6474
6475
6476
6477
6478
6479
6480
6481
6482
6483
6484
6485
6486
6487
6488
6489
6490
6491
6492
6493
6494
6495
6496
6497
6498
6499
6500
6501
6502
6503
6504
6505
6506
6507
6508
6509
6510
6511
6512
6513
6514
6515
6516
6517
6518
6519
6520
6521
6522
6523
6524
6525
6526
6527
6528
6529
6530
6531
6532
6533
6534
6535
6536
6537
6538
6539
6540
6541
6542
6543
6544
6545
6546
6547
6548
6549
6550
6551
6552
6553
6554
6555
6556
6557
6558
6559
6560
6561
6562
6563
6564
6565
6566
6567
6568
6569
6570
6571
6572
6573
6574
6575
6576
6577
6578
6579
6580
6581
6582
6583
6584
6585
6586
6587
6588
6589
6590
6591
6592
6593
6594
6595
6596
6597
6598
6599
6600
6601
6602
6603
6604
6605
6606
6607
6608
6609
6610
6611
6612
6613
6614
6615
6616
6617
6618
6619
6620
6621
6622
6623
6624
6625
6626
6627
6628
6629
6630
6631
6632
6633
6634
6635
6636
6637
6638
6639
6640
6641
6642
6643
6644
6645
6646
6647
6648
6649
6650
6651
6652
6653
6654
6655
6656
6657
6658
6659
6660
6661
6662
6663
6664
6665
6666
6667
6668
6669
6670
6671
6672
6673
6674
6675
6676
6677
6678
6679
6680
6681
6682
6683
6684
6685
6686
6687
6688
6689
6690
6691
6692
6693
6694
6695
6696
6697
6698
6699
6700
6701
6702
6703
6704
6705
6706
6707
6708
6709
6710
6711
6712
6713
6714
6715
6716
6717
6718
6719
6720
6721
6722
6723
6724
6725
6726
6727
6728
6729
6730
6731
6732
6733
6734
6735
6736
6737
6738
6739
6740
6741
6742
6743
6744
6745
6746
6747
6748
6749
6750
6751
6752
6753
6754
6755
6756
6757
6758
6759
6760
6761
6762
6763
6764
6765
6766
6767
6768
6769
6770
6771
6772
6773
6774
6775
6776
6777
6778
6779
6780
6781
6782
6783
6784
6785
6786
6787
6788
6789
6790
6791
6792
6793
6794
6795
6796
....
7070
7071
7072
7073
7074
7075
7076
7077
7078
7079
7080
7081
7082
7083
7084
7085
7086
7087
7088
7089
7090
....
7114
7115
7116
7117
7118
7119
7120
7121
7122
7123
7124
7125
7126
7127
7128
7129
7130
....
7164
7165
7166
7167
7168
7169
7170
7171
7172
7173
7174
7175
7176
7177
7178
INCLUDE ../ext/misc/memtrace.c
#ifdef SQLITE_HAVE_ZLIB
INCLUDE ../ext/misc/zipfile.c
INCLUDE ../ext/misc/sqlar.c
#endif
INCLUDE ../ext/expert/sqlite3expert.h
INCLUDE ../ext/expert/sqlite3expert.c

#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
INCLUDE ../ext/misc/dbdata.c
#endif

#if defined(SQLITE_ENABLE_SESSION)
/*
** State information for a single open session
*/
typedef struct OpenSession OpenSession;
struct OpenSession {
................................................................................
  "   --once                    Do no more than one progress interrupt",
  "   --quiet|-q                No output except at interrupts",
  "   --reset                   Reset the count for each input and interrupt",
#endif
  ".prompt MAIN CONTINUE    Replace the standard prompts",
  ".quit                    Exit this program",
  ".read FILE               Read input from FILE",
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
  ".recover                 Recover as much data as possible from corrupt db.",
#endif
  ".restore ?DB? FILE       Restore content of DB (default \"main\") from FILE",
  ".save FILE               Write in-memory database into FILE",
  ".scanstats on|off        Turn sqlite3_stmt_scanstatus() metrics on or off",
  ".schema ?PATTERN?        Show the CREATE statements matching PATTERN",
  "     Options:",
  "         --indent            Try to pretty-print the schema",
  ".selftest ?OPTIONS?      Run tests defined in the SELFTEST table",
................................................................................
    p->lineno = nLine;
  }
  sqlite3_free(a);
  utf8_printf(stderr,"Error on line %d of --hexdb input\n", nLine);
  return 0;
}
#endif /* SQLITE_ENABLE_DESERIALIZE */

/*
** Scalar function "shell_int32". The first argument to this function
** must be a blob. The second a non-negative integer. This function
** reads and returns a 32-bit big-endian integer from byte
** offset (4*<arg2>) of the blob.
*/
static void shellInt32(
  sqlite3_context *context, 
  int argc, 
  sqlite3_value **argv
){
  const unsigned char *pBlob;
  int nBlob;
  int iInt;
  
  nBlob = sqlite3_value_bytes(argv[0]);
  pBlob = (const unsigned char*)sqlite3_value_blob(argv[0]);
  iInt = sqlite3_value_int(argv[1]);

  if( iInt>=0 && (iInt+1)*4<=nBlob ){
    const unsigned char *a = &pBlob[iInt*4];
    sqlite3_int64 iVal = ((sqlite3_int64)a[0]<<24)
                       + ((sqlite3_int64)a[1]<<16)
                       + ((sqlite3_int64)a[2]<< 8)
                       + ((sqlite3_int64)a[3]<< 0);
    sqlite3_result_int64(context, iVal);
  }
}

/*
** Scalar function "shell_escape_crnl" used by the .recover command.
** The argument passed to this function is the output of built-in
** function quote(). If the first character of the input is "'", 
** indicating that the value passed to quote() was a text value,
** then this function searches the input for "\n" and "\r" characters
** and adds a wrapper similar to the following:
**
**   replace(replace(<input>, '\n', char(10), '\r', char(13));
**
** Or, if the first character of the input is not "'", then a copy
** of the input is returned.
*/
static void shellEscapeCrnl(
  sqlite3_context *context, 
  int argc, 
  sqlite3_value **argv
){
  const char *zText = (const char*)sqlite3_value_text(argv[0]);
  if( zText[0]=='\'' ){
    int nText = sqlite3_value_bytes(argv[0]);
    int i;
    char zBuf1[20];
    char zBuf2[20];
    const char *zNL = 0;
    const char *zCR = 0;
    int nCR = 0;
    int nNL = 0;

    for(i=0; zText[i]; i++){
      if( zNL==0 && zText[i]=='\n' ){
        zNL = unused_string(zText, "\\n", "\\012", zBuf1);
        nNL = (int)strlen(zNL);
      }
      if( zCR==0 && zText[i]=='\r' ){
        zCR = unused_string(zText, "\\r", "\\015", zBuf2);
        nCR = (int)strlen(zCR);
      }
    }

    if( zNL || zCR ){
      int iOut = 0;
      i64 nMax = (nNL > nCR) ? nNL : nCR;
      i64 nAlloc = nMax * nText + (nMax+64)*2;
      char *zOut = (char*)sqlite3_malloc64(nAlloc);
      if( zOut==0 ){
        sqlite3_result_error_nomem(context);
        return;
      }

      if( zNL && zCR ){
        memcpy(&zOut[iOut], "replace(replace(", 16);
        iOut += 16;
      }else{
        memcpy(&zOut[iOut], "replace(", 8);
        iOut += 8;
      }
      for(i=0; zText[i]; i++){
        if( zText[i]=='\n' ){
          memcpy(&zOut[iOut], zNL, nNL);
          iOut += nNL;
        }else if( zText[i]=='\r' ){
          memcpy(&zOut[iOut], zCR, nCR);
          iOut += nCR;
        }else{
          zOut[iOut] = zText[i];
          iOut++;
        }
      }

      if( zNL ){
        memcpy(&zOut[iOut], ",'", 2); iOut += 2;
        memcpy(&zOut[iOut], zNL, nNL); iOut += nNL;
        memcpy(&zOut[iOut], "', char(10))", 12); iOut += 12;
      }
      if( zCR ){
        memcpy(&zOut[iOut], ",'", 2); iOut += 2;
        memcpy(&zOut[iOut], zCR, nCR); iOut += nCR;
        memcpy(&zOut[iOut], "', char(13))", 12); iOut += 12;
      }

      sqlite3_result_text(context, zOut, iOut, SQLITE_TRANSIENT);
      sqlite3_free(zOut);
      return;
    }
  }

  sqlite3_result_value(context, argv[0]);
}

/* Flags for open_db().
**
** The default behavior of open_db() is to exit(1) if the database fails to
** open.  The OPEN_DB_KEEPALIVE flag changes that so that it prints an error
** but still returns without calling exit.
**
................................................................................
    }
#ifndef SQLITE_OMIT_LOAD_EXTENSION
    sqlite3_enable_load_extension(p->db, 1);
#endif
    sqlite3_fileio_init(p->db, 0, 0);
    sqlite3_shathree_init(p->db, 0, 0);
    sqlite3_completion_init(p->db, 0, 0);
#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
    sqlite3_dbdata_init(p->db, 0, 0);
#endif
#ifdef SQLITE_HAVE_ZLIB
    sqlite3_zipfile_init(p->db, 0, 0);
    sqlite3_sqlar_init(p->db, 0, 0);
#endif
    sqlite3_create_function(p->db, "shell_add_schema", 3, SQLITE_UTF8, 0,
                            shellAddSchemaName, 0, 0);
    sqlite3_create_function(p->db, "shell_module_schema", 1, SQLITE_UTF8, 0,
                            shellModuleSchema, 0, 0);
    sqlite3_create_function(p->db, "shell_putsnl", 1, SQLITE_UTF8, p,
                            shellPutsFunc, 0, 0);
    sqlite3_create_function(p->db, "shell_escape_crnl", 1, SQLITE_UTF8, 0,
                            shellEscapeCrnl, 0, 0);
    sqlite3_create_function(p->db, "shell_int32", 2, SQLITE_UTF8, 0,
                            shellInt32, 0, 0);
#ifndef SQLITE_NOHAVE_SYSTEM
    sqlite3_create_function(p->db, "edit", 1, SQLITE_UTF8, 0,
                            editFunc, 0, 0);
    sqlite3_create_function(p->db, "edit", 2, SQLITE_UTF8, 0,
                            editFunc, 0, 0);
#endif
    if( p->openMode==SHELL_OPEN_ZIPFILE ){
................................................................................
 usage:
  raw_printf(stderr, "Usage %s sub-command ?switches...?\n", azArg[0]);
  raw_printf(stderr, "Where sub-commands are:\n");
  raw_printf(stderr, "    fkey-indexes\n");
  return SQLITE_ERROR;
}

#if !defined SQLITE_OMIT_VIRTUALTABLE



static void shellPrepare(
  sqlite3 *db, 
  int *pRc, 
  const char *zSql, 
  sqlite3_stmt **ppStmt
){
  *ppStmt = 0;
................................................................................
    if( rc!=SQLITE_OK ){
      sqlite3 *db = sqlite3_db_handle(pStmt);
      raw_printf(stderr, "SQL error: %s\n", sqlite3_errmsg(db));
    }
    *pRc = rc;
  }
}
#endif /* !defined SQLITE_OMIT_VIRTUALTABLE */

#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
/*********************************************************************************
** The ".archive" or ".ar" command.
*/
/*
** Structure representing a single ".ar" command.
*/
typedef struct ArCommand ArCommand;
struct ArCommand {
  u8 eCmd;                        /* An AR_CMD_* value */
  u8 bVerbose;                    /* True if --verbose */
................................................................................

  return rc;
}
/* End of the ".archive" or ".ar" command logic
**********************************************************************************/
#endif /* !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB) */

#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
/*
** If (*pRc) is not SQLITE_OK when this function is called, it is a no-op.
** Otherwise, the SQL statement or statements in zSql are executed using
** database connection db and the error code written to *pRc before
** this function returns.
*/
static void shellExec(sqlite3 *db, int *pRc, const char *zSql){
  int rc = *pRc;
  if( rc==SQLITE_OK ){
    char *zErr = 0;
    rc = sqlite3_exec(db, zSql, 0, 0, &zErr);
    if( rc!=SQLITE_OK ){
      raw_printf(stderr, "SQL error: %s\n", zErr);
    }
    *pRc = rc;
  }
}

/*
** Like shellExec(), except that zFmt is a printf() style format string.
*/
static void shellExecPrintf(sqlite3 *db, int *pRc, const char *zFmt, ...){
  char *z = 0;
  if( *pRc==SQLITE_OK ){
    va_list ap;
    va_start(ap, zFmt);
    z = sqlite3_vmprintf(zFmt, ap);
    va_end(ap);
    if( z==0 ){
      *pRc = SQLITE_NOMEM;
    }else{
      shellExec(db, pRc, z);
    }
    sqlite3_free(z);
  }
}

/*
** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
** Otherwise, an attempt is made to allocate, zero and return a pointer
** to a buffer nByte bytes in size. If an OOM error occurs, *pRc is set
** to SQLITE_NOMEM and NULL returned.
*/
static void *shellMalloc(int *pRc, sqlite3_int64 nByte){
  void *pRet = 0;
  if( *pRc==SQLITE_OK ){
    pRet = sqlite3_malloc64(nByte);
    if( pRet==0 ){
      *pRc = SQLITE_NOMEM;
    }else{
      memset(pRet, 0, nByte);
    }
  }
  return pRet;
}

/*
** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
** Otherwise, zFmt is treated as a printf() style string. The result of
** formatting it along with any trailing arguments is written into a 
** buffer obtained from sqlite3_malloc(), and pointer to which is returned.
** It is the responsibility of the caller to eventually free this buffer
** using a call to sqlite3_free().
** 
** If an OOM error occurs, (*pRc) is set to SQLITE_NOMEM and a NULL 
** pointer returned.
*/
static char *shellMPrintf(int *pRc, const char *zFmt, ...){
  char *z = 0;
  if( *pRc==SQLITE_OK ){
    va_list ap;
    va_start(ap, zFmt);
    z = sqlite3_vmprintf(zFmt, ap);
    va_end(ap);
    if( z==0 ){
      *pRc = SQLITE_NOMEM;
    }
  }
  return z;
}

/*
** When running the ".recover" command, each output table, and the special
** orphaned row table if it is required, is represented by an instance
** of the following struct.
*/
typedef struct RecoverTable RecoverTable;
struct RecoverTable {
  char *zQuoted;                  /* Quoted version of table name */
  int nCol;                       /* Number of columns in table */
  char **azlCol;                  /* Array of column lists */
  int iPk;                        /* Index of IPK column */
};

/*
** Free a RecoverTable object allocated by recoverFindTable() or
** recoverOrphanTable().
*/
static void recoverFreeTable(RecoverTable *pTab){
  if( pTab ){
    sqlite3_free(pTab->zQuoted);
    if( pTab->azlCol ){
      int i;
      for(i=0; i<=pTab->nCol; i++){
        sqlite3_free(pTab->azlCol[i]);
      }
      sqlite3_free(pTab->azlCol);
    }
    sqlite3_free(pTab);
  }
}

/*
** This function is a no-op if (*pRc) is not SQLITE_OK when it is called.
** Otherwise, it allocates and returns a RecoverTable object based on the
** final four arguments passed to this function. It is the responsibility
** of the caller to eventually free the returned object using
** recoverFreeTable().
*/
static RecoverTable *recoverNewTable(
  int *pRc,                       /* IN/OUT: Error code */
  const char *zName,              /* Name of table */
  const char *zSql,               /* CREATE TABLE statement */
  int bIntkey, 
  int nCol
){
  sqlite3 *dbtmp = 0;             /* sqlite3 handle for testing CREATE TABLE */
  int rc = *pRc;
  RecoverTable *pTab = 0;

  pTab = (RecoverTable*)shellMalloc(&rc, sizeof(RecoverTable));
  if( rc==SQLITE_OK ){
    int nSqlCol = 0;
    int bSqlIntkey = 0;
    sqlite3_stmt *pStmt = 0;
    
    rc = sqlite3_open("", &dbtmp);
    if( rc==SQLITE_OK ){
      rc = sqlite3_exec(dbtmp, "PRAGMA writable_schema = on", 0, 0, 0);
    }
    if( rc==SQLITE_OK ){
      rc = sqlite3_exec(dbtmp, zSql, 0, 0, 0);
      if( rc==SQLITE_ERROR ){
        rc = SQLITE_OK;
        goto finished;
      }
    }
    shellPreparePrintf(dbtmp, &rc, &pStmt, 
        "SELECT count(*) FROM pragma_table_info(%Q)", zName
    );
    if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
      nSqlCol = sqlite3_column_int(pStmt, 0);
    }
    shellFinalize(&rc, pStmt);

    if( rc!=SQLITE_OK || nSqlCol<nCol ){
      goto finished;
    }

    shellPreparePrintf(dbtmp, &rc, &pStmt, 
      "SELECT ("
      "  SELECT substr(data,1,1)==X'0D' FROM sqlite_dbpage WHERE pgno=rootpage"
      ") FROM sqlite_master WHERE name = %Q", zName
    );
    if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
      bSqlIntkey = sqlite3_column_int(pStmt, 0);
    }
    shellFinalize(&rc, pStmt);

    if( bIntkey==bSqlIntkey ){
      int i;
      const char *zPk = "_rowid_";
      sqlite3_stmt *pPkFinder = 0;

      /* If this is an intkey table and there is an INTEGER PRIMARY KEY,
      ** set zPk to the name of the PK column, and pTab->iPk to the index
      ** of the column, where columns are 0-numbered from left to right.
      ** Or, if this is a WITHOUT ROWID table or if there is no IPK column,
      ** leave zPk as "_rowid_" and pTab->iPk at -2.  */
      pTab->iPk = -2;
      if( bIntkey ){
        shellPreparePrintf(dbtmp, &rc, &pPkFinder, 
          "SELECT cid, name FROM pragma_table_info(%Q) "
          "  WHERE pk=1 AND type='integer' COLLATE nocase"
          "  AND NOT EXISTS (SELECT cid FROM pragma_table_info(%Q) WHERE pk=2)"
          , zName, zName
        );
        if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pPkFinder) ){
          pTab->iPk = sqlite3_column_int(pPkFinder, 0);
          zPk = (const char*)sqlite3_column_text(pPkFinder, 1);
        }
      }

      pTab->zQuoted = shellMPrintf(&rc, "%Q", zName);
      pTab->azlCol = (char**)shellMalloc(&rc, sizeof(char*) * (nSqlCol+1));
      pTab->nCol = nSqlCol;

      if( bIntkey ){
        pTab->azlCol[0] = shellMPrintf(&rc, "%Q", zPk);
      }else{
        pTab->azlCol[0] = shellMPrintf(&rc, "");
      }
      i = 1;
      shellPreparePrintf(dbtmp, &rc, &pStmt, 
          "SELECT %Q || group_concat(name, ', ') "
          "  FILTER (WHERE cid!=%d) OVER (ORDER BY %s cid) "
          "FROM pragma_table_info(%Q)", 
          bIntkey ? ", " : "", pTab->iPk, 
          bIntkey ? "" : "(CASE WHEN pk=0 THEN 1000000 ELSE pk END), ",
          zName
      );
      while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
        const char *zText = (const char*)sqlite3_column_text(pStmt, 0);
        pTab->azlCol[i] = shellMPrintf(&rc, "%s%s", pTab->azlCol[0], zText);
        i++;
      }
      shellFinalize(&rc, pStmt);

      shellFinalize(&rc, pPkFinder);
    }
  }

 finished:
  sqlite3_close(dbtmp);
  *pRc = rc;
  if( rc!=SQLITE_OK ){
    recoverFreeTable(pTab);
    pTab = 0;
  }
  return pTab;
}

/*
** This function is called to search the schema recovered from the
** sqlite_master table of the (possibly) corrupt database as part
** of a ".recover" command. Specifically, for a table with root page
** iRoot and at least nCol columns. Additionally, if bIntkey is 0, the
** table must be a WITHOUT ROWID table, or if non-zero, not one of
** those.
**
** If a table is found, a (RecoverTable*) object is returned. Or, if
** no such table is found, but bIntkey is false and iRoot is the 
** root page of an index in the recovered schema, then (*pbNoop) is
** set to true and NULL returned. Or, if there is no such table or
** index, NULL is returned and (*pbNoop) set to 0, indicating that
** the caller should write data to the orphans table.
*/
static RecoverTable *recoverFindTable(
  ShellState *pState,             /* Shell state object */
  int *pRc,                       /* IN/OUT: Error code */
  int iRoot,                      /* Root page of table */
  int bIntkey,                    /* True for an intkey table */
  int nCol,                       /* Number of columns in table */
  int *pbNoop                     /* OUT: True if iRoot is root of index */
){
  sqlite3_stmt *pStmt = 0;
  RecoverTable *pRet = 0;
  int bNoop = 0;
  const char *zSql = 0;
  const char *zName = 0;

  /* Search the recovered schema for an object with root page iRoot. */
  shellPreparePrintf(pState->db, pRc, &pStmt,
      "SELECT type, name, sql FROM recovery.schema WHERE rootpage=%d", iRoot
  );
  while( *pRc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
    const char *zType = (const char*)sqlite3_column_text(pStmt, 0);
    if( bIntkey==0 && sqlite3_stricmp(zType, "index")==0 ){
      bNoop = 1;
      break;
    }
    if( sqlite3_stricmp(zType, "table")==0 ){
      zName = (const char*)sqlite3_column_text(pStmt, 1);
      zSql = (const char*)sqlite3_column_text(pStmt, 2);
      pRet = recoverNewTable(pRc, zName, zSql, bIntkey, nCol);
      break;
    }
  }

  shellFinalize(pRc, pStmt);
  *pbNoop = bNoop;
  return pRet;
}

/*
** Return a RecoverTable object representing the orphans table.
*/
static RecoverTable *recoverOrphanTable(
  ShellState *pState,             /* Shell state object */
  int *pRc,                       /* IN/OUT: Error code */
  const char *zLostAndFound,      /* Base name for orphans table */
  int nCol                        /* Number of user data columns */
){
  RecoverTable *pTab = 0;
  if( nCol>=0 && *pRc==SQLITE_OK ){
    int i;

    /* This block determines the name of the orphan table. The prefered
    ** name is zLostAndFound. But if that clashes with another name
    ** in the recovered schema, try zLostAndFound_0, zLostAndFound_1
    ** and so on until a non-clashing name is found.  */
    int iTab = 0;
    char *zTab = shellMPrintf(pRc, "%s", zLostAndFound);
    sqlite3_stmt *pTest = 0;
    shellPrepare(pState->db, pRc,
        "SELECT 1 FROM recovery.schema WHERE name=?", &pTest
    );
    if( pTest ) sqlite3_bind_text(pTest, 1, zTab, -1, SQLITE_TRANSIENT);
    while( *pRc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pTest) ){
      shellReset(pRc, pTest);
      sqlite3_free(zTab);
      zTab = shellMPrintf(pRc, "%s_%d", zLostAndFound, iTab++);
      sqlite3_bind_text(pTest, 1, zTab, -1, SQLITE_TRANSIENT);
    }
    shellFinalize(pRc, pTest);

    pTab = (RecoverTable*)shellMalloc(pRc, sizeof(RecoverTable));
    if( pTab ){
      pTab->zQuoted = shellMPrintf(pRc, "%Q", zTab);
      pTab->nCol = nCol;
      pTab->iPk = -2;
      if( nCol>0 ){
        pTab->azlCol = (char**)shellMalloc(pRc, sizeof(char*) * (nCol+1));
        if( pTab->azlCol ){
          pTab->azlCol[nCol] = shellMPrintf(pRc, "");
          for(i=nCol-1; i>=0; i--){
            pTab->azlCol[i] = shellMPrintf(pRc, "%s, NULL", pTab->azlCol[i+1]);
          }
        }
      }

      if( *pRc!=SQLITE_OK ){
        recoverFreeTable(pTab);
        pTab = 0;
      }else{
        raw_printf(pState->out, 
            "CREATE TABLE %s(rootpgno INTEGER, "
            "pgno INTEGER, nfield INTEGER, id INTEGER", pTab->zQuoted
        );
        for(i=0; i<nCol; i++){
          raw_printf(pState->out, ", c%d", i);
        }
        raw_printf(pState->out, ");\n");
      }
    }
    sqlite3_free(zTab);
  }
  return pTab;
}

/*
** This function is called to recover data from the database. A script
** to construct a new database containing all recovered data is output
** on stream pState->out.
*/
static int recoverDatabaseCmd(ShellState *pState, int nArg, char **azArg){
  int rc = SQLITE_OK;
  sqlite3_stmt *pLoop = 0;        /* Loop through all root pages */
  sqlite3_stmt *pPages = 0;       /* Loop through all pages in a group */
  sqlite3_stmt *pCells = 0;       /* Loop through all cells in a page */
  const char *zRecoveryDb = "";   /* Name of "recovery" database */
  const char *zLostAndFound = "lost_and_found";
  int i;
  int nOrphan = -1;
  RecoverTable *pOrphan = 0;

  int bFreelist = 1;              /* 0 if --freelist-corrupt is specified */
  for(i=1; i<nArg; i++){
    char *z = azArg[i];
    int n;
    if( z[0]=='-' && z[1]=='-' ) z++;
    n = strlen(z);
    if( n<=17 && memcmp("-freelist-corrupt", z, n)==0 ){
      bFreelist = 0;
    }else
    if( n<=12 && memcmp("-recovery-db", z, n)==0 && i<(nArg-1) ){
      i++;
      zRecoveryDb = azArg[i];
    }else
    if( n<=15 && memcmp("-lost-and-found", z, n)==0 && i<(nArg-1) ){
      i++;
      zLostAndFound = azArg[i];
    }
    else{
      raw_printf(stderr, "unexpected option: %s\n", azArg[i]); 
      raw_printf(stderr, "options are:\n");
      raw_printf(stderr, "    --freelist-corrupt\n");
      raw_printf(stderr, "    --recovery-db DATABASE\n");
      raw_printf(stderr, "    --lost-and-found TABLE-NAME\n");
      return 1;
    }
  }

  shellExecPrintf(pState->db, &rc,
    /* Attach an in-memory database named 'recovery'. Create an indexed 
    ** cache of the sqlite_dbptr virtual table. */
    "ATTACH %Q AS recovery;"
    "DROP TABLE IF EXISTS recovery.dbptr;"
    "DROP TABLE IF EXISTS recovery.freelist;"
    "DROP TABLE IF EXISTS recovery.map;"
    "DROP TABLE IF EXISTS recovery.schema;"
    "CREATE TABLE recovery.freelist(pgno INTEGER PRIMARY KEY);", zRecoveryDb
  );

  if( bFreelist ){
    shellExec(pState->db, &rc,
      "WITH trunk(pgno) AS ("
      "  SELECT shell_int32("
      "      (SELECT data FROM sqlite_dbpage WHERE pgno=1), 8) AS x "
      "      WHERE x>0"
      "    UNION"
      "  SELECT shell_int32("
      "      (SELECT data FROM sqlite_dbpage WHERE pgno=trunk.pgno), 0) AS x "
      "      FROM trunk WHERE x>0"
      "),"
      "freelist(data, n, freepgno) AS ("
      "  SELECT data, shell_int32(data, 1)-1, t.pgno "
      "      FROM trunk t, sqlite_dbpage s WHERE s.pgno=t.pgno"
      "    UNION ALL"
      "  SELECT data, n-1, shell_int32(data, 2+n) "
      "      FROM freelist WHERE n>=0"
      ")"
      "REPLACE INTO recovery.freelist SELECT freepgno FROM freelist;"
    );
  }

  shellExec(pState->db, &rc, 
    "CREATE TABLE recovery.dbptr("
    "      pgno, child, PRIMARY KEY(child, pgno)"
    ") WITHOUT ROWID;"
    "INSERT OR IGNORE INTO recovery.dbptr(pgno, child) "
    "    SELECT * FROM sqlite_dbptr"
    "      WHERE pgno NOT IN freelist AND child NOT IN freelist;"

    /* Delete any pointer to page 1. This ensures that page 1 is considered
    ** a root page, regardless of how corrupt the db is. */
    "DELETE FROM recovery.dbptr WHERE child = 1;"

    /* Delete all pointers to any pages that have more than one pointer
    ** to them. Such pages will be treated as root pages when recovering
    ** data.  */
    "DELETE FROM recovery.dbptr WHERE child IN ("
    "  SELECT child FROM recovery.dbptr GROUP BY child HAVING count(*)>1"
    ");"

    /* Create the "map" table that will (eventually) contain instructions
    ** for dealing with each page in the db that contains one or more 
    ** records. */
    "CREATE TABLE recovery.map("
      "pgno INTEGER PRIMARY KEY, maxlen INT, intkey, root INT"
    ");"

    /* Populate table [map]. If there are circular loops of pages in the
    ** database, the following adds all pages in such a loop to the map
    ** as individual root pages. This could be handled better.  */
    "WITH pages(i, maxlen) AS ("
    "  SELECT page_count, ("
    "    SELECT max(field+1) FROM sqlite_dbdata WHERE pgno=page_count"
    "  ) FROM pragma_page_count"
    "    UNION ALL"
    "  SELECT i-1, ("
    "    SELECT max(field+1) FROM sqlite_dbdata WHERE pgno=i-1"
    "  ) FROM pages WHERE i>=2"
    ")"
    "INSERT INTO recovery.map(pgno, maxlen, intkey, root) "
    "  SELECT i, maxlen, NULL, ("
    "    WITH p(orig, pgno, parent) AS ("
    "      SELECT 0, i, (SELECT pgno FROM recovery.dbptr WHERE child=i)"
    "        UNION ALL"
    "      SELECT i, p.parent, "
    "        (SELECT pgno FROM recovery.dbptr WHERE child=p.parent) FROM p"
    "    )"
    "    SELECT pgno FROM p WHERE (parent IS NULL OR pgno = orig)"
    ") "
    "FROM pages WHERE maxlen > 0 AND i NOT IN freelist;"
    "UPDATE recovery.map AS o SET intkey = ("
    "  SELECT substr(data, 1, 1)==X'0D' FROM sqlite_dbpage WHERE pgno=o.pgno"
    ");"

    /* Extract data from page 1 and any linked pages into table
    ** recovery.schema. With the same schema as an sqlite_master table.  */
    "CREATE TABLE recovery.schema(type, name, tbl_name, rootpage, sql);"
    "INSERT INTO recovery.schema SELECT "
    "  max(CASE WHEN field=0 THEN value ELSE NULL END),"
    "  max(CASE WHEN field=1 THEN value ELSE NULL END),"
    "  max(CASE WHEN field=2 THEN value ELSE NULL END),"
    "  max(CASE WHEN field=3 THEN value ELSE NULL END),"
    "  max(CASE WHEN field=4 THEN value ELSE NULL END)"
    "FROM sqlite_dbdata WHERE pgno IN ("
    "  SELECT pgno FROM recovery.map WHERE root=1"
    ")"
    "GROUP BY pgno, cell;"
    "CREATE INDEX recovery.schema_rootpage ON schema(rootpage);"
  );

  /* Open a transaction, then print out all non-virtual, non-"sqlite_%" 
  ** CREATE TABLE statements that extracted from the existing schema.  */
  if( rc==SQLITE_OK ){
    sqlite3_stmt *pStmt = 0;
    raw_printf(pState->out, "BEGIN;\n");
    raw_printf(pState->out, "PRAGMA writable_schema = on;\n");
    shellPrepare(pState->db, &rc,
        "SELECT sql FROM recovery.schema "
        "WHERE type='table' AND sql LIKE 'create table%'", &pStmt
    );
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
      const char *zCreateTable = (const char*)sqlite3_column_text(pStmt, 0);
      raw_printf(pState->out, "CREATE TABLE IF NOT EXISTS %s;\n", 
          &zCreateTable[12]
      );
    }
    shellFinalize(&rc, pStmt);
  }

  /* Figure out if an orphan table will be required. And if so, how many
  ** user columns it should contain */
  shellPrepare(pState->db, &rc, 
      "SELECT coalesce(max(maxlen), -2) FROM recovery.map" 
      "  WHERE root>1 AND root NOT IN (SELECT rootpage FROM recovery.schema)"
      , &pLoop
  );
  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pLoop) ){
    nOrphan = sqlite3_column_int(pLoop, 0);
  }
  shellFinalize(&rc, pLoop);
  pLoop = 0;
  pOrphan = recoverOrphanTable(pState, &rc, zLostAndFound, nOrphan);

  shellPrepare(pState->db, &rc,
      "SELECT pgno FROM recovery.map WHERE root=?", &pPages
  );
  shellPrepare(pState->db, &rc,
      "SELECT max(field), group_concat(shell_escape_crnl(quote(value)), ', ')"
      "FROM sqlite_dbdata WHERE pgno = ? AND field != ?"
      "GROUP BY cell", &pCells
  );

  /* Loop through each root page. */
  shellPrepare(pState->db, &rc, 
      "SELECT root, intkey, max(maxlen) FROM recovery.map" 
      " WHERE root>1 GROUP BY root, intkey ORDER BY root=("
      "  SELECT rootpage FROM recovery.schema WHERE name='sqlite_sequence'"
      ")", &pLoop
  );
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pLoop) ){
    int iRoot = sqlite3_column_int(pLoop, 0);
    int bIntkey = sqlite3_column_int(pLoop, 1);
    int nCol = sqlite3_column_int(pLoop, 2);
    int bNoop = 0;
    RecoverTable *pTab;

    pTab = recoverFindTable(pState, &rc, iRoot, bIntkey, nCol, &bNoop);
    if( bNoop || rc ) continue;
    if( pTab==0 ) pTab = pOrphan;

    if( 0==sqlite3_stricmp(pTab->zQuoted, "'sqlite_sequence'") ){
      raw_printf(pState->out, "DELETE FROM sqlite_sequence;\n");
    }
    sqlite3_bind_int(pPages, 1, iRoot);
    sqlite3_bind_int(pCells, 2, pTab->iPk);

    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pPages) ){
      int iPgno = sqlite3_column_int(pPages, 0);
      sqlite3_bind_int(pCells, 1, iPgno);
      while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pCells) ){
        int nField = sqlite3_column_int(pCells, 0);
        const char *zVal = (const char*)sqlite3_column_text(pCells, 1);

        nField = nField+1;
        if( pTab==pOrphan ){
          raw_printf(pState->out, 
              "INSERT INTO %s VALUES(%d, %d, %d, %s%s%s);\n",
              pTab->zQuoted, iRoot, iPgno, nField, 
              bIntkey ? "" : "NULL, ", zVal, pTab->azlCol[nField]
          );
        }else{
          raw_printf(pState->out, "INSERT INTO %s(%s) VALUES( %s );\n", 
              pTab->zQuoted, pTab->azlCol[nField], zVal
          );
        }
      }
      shellReset(&rc, pCells);
    }
    shellReset(&rc, pPages);
    if( pTab!=pOrphan ) recoverFreeTable(pTab);
  }
  shellFinalize(&rc, pLoop);
  shellFinalize(&rc, pPages);
  shellFinalize(&rc, pCells);
  recoverFreeTable(pOrphan);

  /* The rest of the schema */
  if( rc==SQLITE_OK ){
    sqlite3_stmt *pStmt = 0;
    shellPrepare(pState->db, &rc, 
        "SELECT sql, name FROM recovery.schema "
        "WHERE sql NOT LIKE 'create table%'", &pStmt
    );
    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
      const char *zSql = (const char*)sqlite3_column_text(pStmt, 0);
      if( sqlite3_strnicmp(zSql, "create virt", 11)==0 ){
        const char *zName = (const char*)sqlite3_column_text(pStmt, 1);
        char *zPrint = shellMPrintf(&rc, 
          "INSERT INTO sqlite_master VALUES('table', %Q, %Q, 0, %Q)",
          zName, zName, zSql
        );
        raw_printf(pState->out, "%s;\n", zPrint);
        sqlite3_free(zPrint);
      }else{
        raw_printf(pState->out, "%s;\n", zSql);
      }
    }
    shellFinalize(&rc, pStmt);
  }

  if( rc==SQLITE_OK ){
    raw_printf(pState->out, "PRAGMA writable_schema = off;\n");
    raw_printf(pState->out, "COMMIT;\n");
  }
  sqlite3_exec(pState->db, "DETACH recovery", 0, 0, 0);
  return rc;
}
#endif /* !(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB) */


/*
** If an input line begins with "." then invoke this routine to
** process that line.
**
** Return 1 on error, 2 to exit, and 0 otherwise.
*/
................................................................................
    }   
  }else

  if( c=='d' && n>=3 && strncmp(azArg[0], "dbinfo", n)==0 ){
    rc = shell_dbinfo_command(p, nArg, azArg);
  }else

#if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB)
  if( c=='r' && strncmp(azArg[0], "recover", n)==0 ){
    open_db(p, 0);
    rc = recoverDatabaseCmd(p, nArg, azArg);
  }else
#endif /* !(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB) */

  if( c=='d' && strncmp(azArg[0], "dump", n)==0 ){
    const char *zLike = 0;
    int i;
    int savedShowHeader = p->showHeader;
    int savedShellFlags = p->shellFlgs;
    ShellClearFlag(p, SHFLG_PreserveRowid|SHFLG_Newlines|SHFLG_Echo);
    for(i=1; i<nArg; i++){
................................................................................
                           "?--newlines? ?LIKE-PATTERN?\n");
        rc = 1;
        goto meta_command_exit;
      }else{
        zLike = azArg[i];
      }
    }

    open_db(p, 0);

    /* When playing back a "dump", the content might appear in an order
    ** which causes immediate foreign key constraints to be violated.
    ** So disable foreign-key constraint enforcement to prevent problems. */
    raw_printf(p->out, "PRAGMA foreign_keys=OFF;\n");
    raw_printf(p->out, "BEGIN TRANSACTION;\n");
    p->writableSchema = 0;
    p->showHeader = 0;
................................................................................
    }
    if( p->writableSchema ){
      raw_printf(p->out, "PRAGMA writable_schema=OFF;\n");
      p->writableSchema = 0;
    }
    sqlite3_exec(p->db, "PRAGMA writable_schema=OFF;", 0, 0, 0);
    sqlite3_exec(p->db, "RELEASE dump;", 0, 0, 0);
    raw_printf(p->out, p->nErr?"ROLLBACK; -- due to errors\n":"COMMIT;\n");
    p->showHeader = savedShowHeader;
    p->shellFlgs = savedShellFlags;
  }else

  if( c=='e' && strncmp(azArg[0], "echo", n)==0 ){
    if( nArg==2 ){
      setOrClearFlag(p, SHFLG_Echo, azArg[1]);

Changes to src/vdbe.c.

191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
...
518
519
520
521
522
523
524


525
526
527
528
529
530
531
....
1459
1460
1461
1462
1463
1464
1465
1466


1467
1468
1469
1470


1471



1472
1473
1474
1475












1476
1477
1478


1479
1480
1481
1482
1483
1484
1485
1486
1487
1488


1489
1490


1491
1492
1493
1494
1495
1496
1497
....
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771








2772
2773

2774
2775
2776
2777
2778
2779
2780
....
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809


2810
2811
2812
2813
2814
2815
2816
....
2929
2930
2931
2932
2933
2934
2935







2936

2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
      if( (mNever&0x08)!=0 && (I&0x05)!=0) I |= 0x05; /*NO_TEST*/
    }
    sqlite3GlobalConfig.xVdbeBranch(sqlite3GlobalConfig.pVdbeBranchArg,
                                    iSrcLine&0xffffff, I, M);
  }
#endif

/*
** Convert the given register into a string if it isn't one
** already. Return non-zero if a malloc() fails.
*/
#define Stringify(P, enc) \
   if(((P)->flags&(MEM_Str|MEM_Blob))==0 && sqlite3VdbeMemStringify(P,enc,0)) \
     { goto no_mem; }

/*
** An ephemeral string value (signified by the MEM_Ephem flag) contains
** a pointer to a dynamically allocated string where some other entity
** is responsible for deallocating that string.  Because the register
** does not control the string, it might be deleted without the register
** knowing it.
**
................................................................................
static void memTracePrint(Mem *p){
  if( p->flags & MEM_Undefined ){
    printf(" undefined");
  }else if( p->flags & MEM_Null ){
    printf(p->flags & MEM_Zero ? " NULL-nochng" : " NULL");
  }else if( (p->flags & (MEM_Int|MEM_Str))==(MEM_Int|MEM_Str) ){
    printf(" si:%lld", p->u.i);


  }else if( p->flags & MEM_Int ){
    printf(" i:%lld", p->u.i);
#ifndef SQLITE_OMIT_FLOATING_POINT
  }else if( p->flags & MEM_Real ){
    printf(" r:%g", p->u.r);
#endif
  }else if( sqlite3VdbeMemIsRowSet(p) ){
................................................................................
**   P3 = P2 || P1
**
** It is illegal for P1 and P3 to be the same register. Sometimes,
** if P3 is the same register as P2, the implementation is able
** to avoid a memcpy().
*/
case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
  i64 nByte;



  pIn1 = &aMem[pOp->p1];
  pIn2 = &aMem[pOp->p2];
  pOut = &aMem[pOp->p3];


  assert( pIn1!=pOut );



  if( (pIn1->flags | pIn2->flags) & MEM_Null ){
    sqlite3VdbeMemSetNull(pOut);
    break;
  }












  if( ExpandBlob(pIn1) || ExpandBlob(pIn2) ) goto no_mem;
  Stringify(pIn1, encoding);
  Stringify(pIn2, encoding);


  nByte = pIn1->n + pIn2->n;
  if( nByte>db->aLimit[SQLITE_LIMIT_LENGTH] ){
    goto too_big;
  }
  if( sqlite3VdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2) ){
    goto no_mem;
  }
  MemSetTypeFlag(pOut, MEM_Str);
  if( pOut!=pIn2 ){
    memcpy(pOut->z, pIn2->z, pIn2->n);


  }
  memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n);


  pOut->z[nByte]=0;
  pOut->z[nByte+1] = 0;
  pOut->flags |= MEM_Term;
  pOut->n = (int)nByte;
  pOut->enc = encoding;
  UPDATE_MAX_BLOBSIZE(pOut);
  break;
................................................................................
  const char *zAffinity;   /* The affinity to be applied */

  zAffinity = pOp->p4.z;
  assert( zAffinity!=0 );
  assert( pOp->p2>0 );
  assert( zAffinity[pOp->p2]==0 );
  pIn1 = &aMem[pOp->p1];
  do{
    assert( pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)] );
    assert( memIsValid(pIn1) );
    applyAffinity(pIn1, *(zAffinity++), encoding);








    pIn1++;
  }while( zAffinity[0] );

  break;
}

/* Opcode: MakeRecord P1 P2 P3 P4 *
** Synopsis: r[P3]=mkrec(r[P1@P2])
**
** Convert P2 registers beginning with P1 into the [record format]
................................................................................
**
** The mapping from character to affinity is given by the SQLITE_AFF_
** macros defined in sqliteInt.h.
**
** If P4 is NULL then all index fields have the affinity BLOB.
*/
case OP_MakeRecord: {
  u8 *zNewRecord;        /* A buffer to hold the data for the new record */
  Mem *pRec;             /* The new record */
  u64 nData;             /* Number of bytes of data space */
  int nHdr;              /* Number of bytes of header space */
  i64 nByte;             /* Data space required for this record */
  i64 nZero;             /* Number of zero bytes at the end of the record */
  int nVarint;           /* Number of bytes in a varint */
  u32 serial_type;       /* Type field */
  Mem *pData0;           /* First field to be combined into the record */
  Mem *pLast;            /* Last field of the record */
  int nField;            /* Number of fields in the record */
  char *zAffinity;       /* The affinity string for the record */
  int file_format;       /* File format to use for encoding */
  int i;                 /* Space used in zNewRecord[] header */
  int j;                 /* Space used in zNewRecord[] content */
  u32 len;               /* Length of a field */



  /* Assuming the record contains N fields, the record format looks
  ** like this:
  **
  ** ------------------------------------------------------------------------
  ** | hdr-size | type 0 | type 1 | ... | type N-1 | data0 | ... | data N-1 | 
  ** ------------------------------------------------------------------------
................................................................................
    if( nByte+nZero>db->aLimit[SQLITE_LIMIT_LENGTH] ){
      goto too_big;
    }
    if( sqlite3VdbeMemClearAndResize(pOut, (int)nByte) ){
      goto no_mem;
    }
  }







  zNewRecord = (u8 *)pOut->z;


  /* Write the record */
  i = putVarint32(zNewRecord, nHdr);
  j = nHdr;
  assert( pData0<=pLast );
  pRec = pData0;
  do{
    serial_type = pRec->uTemp;
    /* EVIDENCE-OF: R-06529-47362 Following the size varint are one or more
    ** additional varints, one per column. */
    i += putVarint32(&zNewRecord[i], serial_type);            /* serial type */
    /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
    ** immediately follow the header. */
    j += sqlite3VdbeSerialPut(&zNewRecord[j], pRec, serial_type); /* content */
  }while( (++pRec)<=pLast );
  assert( i==nHdr );
  assert( j==nByte );

  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pOut->n = (int)nByte;
  pOut->flags = MEM_Blob;
  if( nZero ){
    pOut->u.nZero = nZero;
    pOut->flags |= MEM_Zero;
  }
  REGISTER_TRACE(pOp->p3, pOut);
  UPDATE_MAX_BLOBSIZE(pOut);
  break;
}

/* Opcode: Count P1 P2 * * *
** Synopsis: r[P2]=count()
**
** Store the number of entries (an integer value) in the table or index 







<
<
<
<
<
<
<
<







 







>
>







 







|
>
>




>
>

>
>
>
|



>
>
>
>
>
>
>
>
>
>
>
>
|
<
<
>
>










>
>


>
>







 







|


|
>
>
>
>
>
>
>
>

<
>







 







<












<
<

>
>







 







>
>
>
>
>
>
>
|
>


|
<






|


|

|
|


<
<
<
<
<
<

<







191
192
193
194
195
196
197








198
199
200
201
202
203
204
...
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
....
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489


1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
....
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797

2798
2799
2800
2801
2802
2803
2804
2805
....
2812
2813
2814
2815
2816
2817
2818

2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830


2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
....
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971

2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986






2987

2988
2989
2990
2991
2992
2993
2994
      if( (mNever&0x08)!=0 && (I&0x05)!=0) I |= 0x05; /*NO_TEST*/
    }
    sqlite3GlobalConfig.xVdbeBranch(sqlite3GlobalConfig.pVdbeBranchArg,
                                    iSrcLine&0xffffff, I, M);
  }
#endif









/*
** An ephemeral string value (signified by the MEM_Ephem flag) contains
** a pointer to a dynamically allocated string where some other entity
** is responsible for deallocating that string.  Because the register
** does not control the string, it might be deleted without the register
** knowing it.
**
................................................................................
static void memTracePrint(Mem *p){
  if( p->flags & MEM_Undefined ){
    printf(" undefined");
  }else if( p->flags & MEM_Null ){
    printf(p->flags & MEM_Zero ? " NULL-nochng" : " NULL");
  }else if( (p->flags & (MEM_Int|MEM_Str))==(MEM_Int|MEM_Str) ){
    printf(" si:%lld", p->u.i);
  }else if( (p->flags & (MEM_Int|MEM_IntReal))==(MEM_Int|MEM_IntReal) ){
    printf(" ir:%lld", p->u.i);
  }else if( p->flags & MEM_Int ){
    printf(" i:%lld", p->u.i);
#ifndef SQLITE_OMIT_FLOATING_POINT
  }else if( p->flags & MEM_Real ){
    printf(" r:%g", p->u.r);
#endif
  }else if( sqlite3VdbeMemIsRowSet(p) ){
................................................................................
**   P3 = P2 || P1
**
** It is illegal for P1 and P3 to be the same register. Sometimes,
** if P3 is the same register as P2, the implementation is able
** to avoid a memcpy().
*/
case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
  i64 nByte;          /* Total size of the output string or blob */
  u16 flags1;         /* Initial flags for P1 */
  u16 flags2;         /* Initial flags for P2 */

  pIn1 = &aMem[pOp->p1];
  pIn2 = &aMem[pOp->p2];
  pOut = &aMem[pOp->p3];
  testcase( pIn1==pIn2 );
  testcase( pOut==pIn2 );
  assert( pIn1!=pOut );
  flags1 = pIn1->flags;
  testcase( flags1 & MEM_Null );
  testcase( pIn2->flags & MEM_Null );
  if( (flags1 | pIn2->flags) & MEM_Null ){
    sqlite3VdbeMemSetNull(pOut);
    break;
  }
  if( (flags1 & (MEM_Str|MEM_Blob))==0 ){
    if( sqlite3VdbeMemStringify(pIn1,encoding,0) ) goto no_mem;
    flags1 = pIn1->flags & ~MEM_Str;
  }else if( (flags1 & MEM_Zero)!=0 ){
    if( sqlite3VdbeMemExpandBlob(pIn1) ) goto no_mem;
    flags1 = pIn1->flags & ~MEM_Str;
  }
  flags2 = pIn2->flags;
  if( (flags2 & (MEM_Str|MEM_Blob))==0 ){
    if( sqlite3VdbeMemStringify(pIn2,encoding,0) ) goto no_mem;
    flags2 = pIn2->flags & ~MEM_Str;
  }else if( (flags2 & MEM_Zero)!=0 ){
    if( sqlite3VdbeMemExpandBlob(pIn2) ) goto no_mem;


    flags2 = pIn2->flags & ~MEM_Str;
  }
  nByte = pIn1->n + pIn2->n;
  if( nByte>db->aLimit[SQLITE_LIMIT_LENGTH] ){
    goto too_big;
  }
  if( sqlite3VdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2) ){
    goto no_mem;
  }
  MemSetTypeFlag(pOut, MEM_Str);
  if( pOut!=pIn2 ){
    memcpy(pOut->z, pIn2->z, pIn2->n);
    assert( (pIn2->flags & MEM_Dyn) == (flags2 & MEM_Dyn) );
    pIn2->flags = flags2;
  }
  memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n);
  assert( (pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn) );
  pIn1->flags = flags1;
  pOut->z[nByte]=0;
  pOut->z[nByte+1] = 0;
  pOut->flags |= MEM_Term;
  pOut->n = (int)nByte;
  pOut->enc = encoding;
  UPDATE_MAX_BLOBSIZE(pOut);
  break;
................................................................................
  const char *zAffinity;   /* The affinity to be applied */

  zAffinity = pOp->p4.z;
  assert( zAffinity!=0 );
  assert( pOp->p2>0 );
  assert( zAffinity[pOp->p2]==0 );
  pIn1 = &aMem[pOp->p1];
  while( 1 /*edit-by-break*/ ){
    assert( pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)] );
    assert( memIsValid(pIn1) );
    applyAffinity(pIn1, zAffinity[0], encoding);
    if( zAffinity[0]==SQLITE_AFF_REAL && (pIn1->flags & MEM_Int)!=0 ){
      /* When applying REAL affinity, if the result is still MEM_Int, 
      ** indicate that REAL is actually desired */
      pIn1->flags |= MEM_IntReal;
    }
    REGISTER_TRACE((int)(pIn1-aMem), pIn1);
    zAffinity++;
    if( zAffinity[0]==0 ) break;
    pIn1++;

  }
  break;
}

/* Opcode: MakeRecord P1 P2 P3 P4 *
** Synopsis: r[P3]=mkrec(r[P1@P2])
**
** Convert P2 registers beginning with P1 into the [record format]
................................................................................
**
** The mapping from character to affinity is given by the SQLITE_AFF_
** macros defined in sqliteInt.h.
**
** If P4 is NULL then all index fields have the affinity BLOB.
*/
case OP_MakeRecord: {

  Mem *pRec;             /* The new record */
  u64 nData;             /* Number of bytes of data space */
  int nHdr;              /* Number of bytes of header space */
  i64 nByte;             /* Data space required for this record */
  i64 nZero;             /* Number of zero bytes at the end of the record */
  int nVarint;           /* Number of bytes in a varint */
  u32 serial_type;       /* Type field */
  Mem *pData0;           /* First field to be combined into the record */
  Mem *pLast;            /* Last field of the record */
  int nField;            /* Number of fields in the record */
  char *zAffinity;       /* The affinity string for the record */
  int file_format;       /* File format to use for encoding */


  u32 len;               /* Length of a field */
  u8 *zHdr;              /* Where to write next byte of the header */
  u8 *zPayload;          /* Where to write next byte of the payload */

  /* Assuming the record contains N fields, the record format looks
  ** like this:
  **
  ** ------------------------------------------------------------------------
  ** | hdr-size | type 0 | type 1 | ... | type N-1 | data0 | ... | data N-1 | 
  ** ------------------------------------------------------------------------
................................................................................
    if( nByte+nZero>db->aLimit[SQLITE_LIMIT_LENGTH] ){
      goto too_big;
    }
    if( sqlite3VdbeMemClearAndResize(pOut, (int)nByte) ){
      goto no_mem;
    }
  }
  pOut->n = (int)nByte;
  pOut->flags = MEM_Blob;
  if( nZero ){
    pOut->u.nZero = nZero;
    pOut->flags |= MEM_Zero;
  }
  UPDATE_MAX_BLOBSIZE(pOut);
  zHdr = (u8 *)pOut->z;
  zPayload = zHdr + nHdr;

  /* Write the record */
  zHdr += putVarint32(zHdr, nHdr);

  assert( pData0<=pLast );
  pRec = pData0;
  do{
    serial_type = pRec->uTemp;
    /* EVIDENCE-OF: R-06529-47362 Following the size varint are one or more
    ** additional varints, one per column. */
    zHdr += putVarint32(zHdr, serial_type);            /* serial type */
    /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
    ** immediately follow the header. */
    zPayload += sqlite3VdbeSerialPut(zPayload, pRec, serial_type); /* content */
  }while( (++pRec)<=pLast );
  assert( nHdr==(int)(zHdr - (u8*)pOut->z) );
  assert( nByte==(int)(zPayload - (u8*)pOut->z) );

  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );






  REGISTER_TRACE(pOp->p3, pOut);

  break;
}

/* Opcode: Count P1 P2 * * *
** Synopsis: r[P2]=count()
**
** Store the number of entries (an integer value) in the table or index 

Changes to src/vdbeInt.h.

243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
#define MEM_Null      0x0001   /* Value is NULL (or a pointer) */
#define MEM_Str       0x0002   /* Value is a string */
#define MEM_Int       0x0004   /* Value is an integer */
#define MEM_Real      0x0008   /* Value is a real number */
#define MEM_Blob      0x0010   /* Value is a BLOB */
#define MEM_AffMask   0x001f   /* Mask of affinity bits */
#define MEM_FromBind  0x0020   /* Value originates from sqlite3_bind() */
/* Available          0x0040   */
#define MEM_Undefined 0x0080   /* Value is undefined */
#define MEM_Cleared   0x0100   /* NULL set by OP_Null, not from data */
#define MEM_TypeMask  0xc1df   /* Mask of type bits */


/* Whenever Mem contains a valid string or blob representation, one of
** the following flags must be set to determine the memory management







|







243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
#define MEM_Null      0x0001   /* Value is NULL (or a pointer) */
#define MEM_Str       0x0002   /* Value is a string */
#define MEM_Int       0x0004   /* Value is an integer */
#define MEM_Real      0x0008   /* Value is a real number */
#define MEM_Blob      0x0010   /* Value is a BLOB */
#define MEM_AffMask   0x001f   /* Mask of affinity bits */
#define MEM_FromBind  0x0020   /* Value originates from sqlite3_bind() */
#define MEM_IntReal   0x0040   /* MEM_Int that stringifies like MEM_Real */
#define MEM_Undefined 0x0080   /* Value is undefined */
#define MEM_Cleared   0x0100   /* NULL set by OP_Null, not from data */
#define MEM_TypeMask  0xc1df   /* Mask of type bits */


/* Whenever Mem contains a valid string or blob representation, one of
** the following flags must be set to determine the memory management

Changes to src/vdbemem.c.

87
88
89
90
91
92
93



















94
95
96
97
98
99
100
...
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
...
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
...
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
...
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
      ((p->flags&MEM_Ephem)!=0 ? 1 : 0) +
      ((p->flags&MEM_Static)!=0 ? 1 : 0) == 1
    );
  }
  return 1;
}
#endif




















#ifdef SQLITE_DEBUG
/*
** Check that string value of pMem agrees with its integer or real value.
**
** A single int or real value always converts to the same strings.  But
** many different strings can be converted into the same int or real.
................................................................................
*/
int sqlite3VdbeMemConsistentDualRep(Mem *p){
  char zBuf[100];
  char *z;
  int i, j, incr;
  if( (p->flags & MEM_Str)==0 ) return 1;
  if( (p->flags & (MEM_Int|MEM_Real))==0 ) return 1;
  if( p->flags & MEM_Int ){
    sqlite3_snprintf(sizeof(zBuf),zBuf,"%lld",p->u.i);
  }else{
    sqlite3_snprintf(sizeof(zBuf),zBuf,"%!.15g",p->u.r);
  }
  z = p->z;
  i = j = 0;
  incr = 1;
  if( p->enc!=SQLITE_UTF8 ){
    incr = 2;
    if( p->enc==SQLITE_UTF16BE ) z++;
  }
................................................................................
  assert( CORRUPT_DB || szNew>0 );
  assert( (pMem->flags & MEM_Dyn)==0 || pMem->szMalloc==0 );
  if( pMem->szMalloc<szNew ){
    return sqlite3VdbeMemGrow(pMem, szNew, 0);
  }
  assert( (pMem->flags & MEM_Dyn)==0 );
  pMem->z = pMem->zMalloc;
  pMem->flags &= (MEM_Null|MEM_Int|MEM_Real);
  return SQLITE_OK;
}

/*
** It is already known that pMem contains an unterminated string.
** Add the zero terminator.
*/
................................................................................
** A MEM_Null value will never be passed to this function. This function is
** used for converting values to text for returning to the user (i.e. via
** sqlite3_value_text()), or for ensuring that values to be used as btree
** keys are strings. In the former case a NULL pointer is returned the
** user and the latter is an internal programming error.
*/
int sqlite3VdbeMemStringify(Mem *pMem, u8 enc, u8 bForce){
  int fg = pMem->flags;
  const int nByte = 32;

  assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) );
  assert( !(fg&MEM_Zero) );
  assert( !(fg&(MEM_Str|MEM_Blob)) );
  assert( fg&(MEM_Int|MEM_Real) );
  assert( !sqlite3VdbeMemIsRowSet(pMem) );
  assert( EIGHT_BYTE_ALIGNMENT(pMem) );


  if( sqlite3VdbeMemClearAndResize(pMem, nByte) ){
    pMem->enc = 0;
    return SQLITE_NOMEM_BKPT;
  }

  /* For a Real or Integer, use sqlite3_snprintf() to produce the UTF-8
  ** string representation of the value. Then, if the required encoding
  ** is UTF-16le or UTF-16be do a translation.
  ** 
  ** FIX ME: It would be better if sqlite3_snprintf() could do UTF-16.
  */
  if( fg & MEM_Int ){
    sqlite3_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
  }else{
    assert( fg & MEM_Real );
    sqlite3_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
  }
  assert( pMem->z!=0 );
  pMem->n = sqlite3Strlen30NN(pMem->z);
  pMem->enc = SQLITE_UTF8;
  pMem->flags |= MEM_Str|MEM_Term;
  if( bForce ) pMem->flags &= ~(MEM_Int|MEM_Real);
  sqlite3VdbeChangeEncoding(pMem, enc);
  return SQLITE_OK;
}

/*
** Memory cell pMem contains the context of an aggregate function.
** This routine calls the finalize method for that function.  The
................................................................................
    }
    default: {
      assert( aff==SQLITE_AFF_TEXT );
      assert( MEM_Str==(MEM_Blob>>3) );
      pMem->flags |= (pMem->flags&MEM_Blob)>>3;
      sqlite3ValueApplyAffinity(pMem, SQLITE_AFF_TEXT, encoding);
      assert( pMem->flags & MEM_Str || pMem->db->mallocFailed );
      pMem->flags &= ~(MEM_Int|MEM_Real|MEM_Blob|MEM_Zero);
      break;
    }
  }
}

/*
** Initialize bulk memory to be a consistent Mem object.







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
<
<
<
<







 







|







 







<



|
|
|









<
<
<
<
<
<
<
|
<
<
<
<




|







 







|







87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
...
133
134
135
136
137
138
139
140




141
142
143
144
145
146
147
...
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
...
360
361
362
363
364
365
366

367
368
369
370
371
372
373
374
375
376
377
378
379
380
381







382




383
384
385
386
387
388
389
390
391
392
393
394
...
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
      ((p->flags&MEM_Ephem)!=0 ? 1 : 0) +
      ((p->flags&MEM_Static)!=0 ? 1 : 0) == 1
    );
  }
  return 1;
}
#endif

/*
** Render a Mem object which is either MEM_Int or MEM_Real into a
** buffer.
*/
static void vdbeMemRenderNum(int sz, char *zBuf, Mem *p){
  StrAccum acc;
  assert( p->flags & (MEM_Int|MEM_Real) );
  sqlite3StrAccumInit(&acc, 0, zBuf, sz, 0);
  if( p->flags & MEM_IntReal ){
    sqlite3_str_appendf(&acc, "%!.15g", (double)p->u.i);
  }else if( p->flags & MEM_Int ){
    sqlite3_str_appendf(&acc, "%lld", p->u.i);
  }else{
    sqlite3_str_appendf(&acc, "%!.15g", p->u.r);
  }
  assert( acc.zText==zBuf && acc.mxAlloc<=0 );
  zBuf[acc.nChar] = 0; /* Fast version of sqlite3StrAccumFinish(&acc) */
}

#ifdef SQLITE_DEBUG
/*
** Check that string value of pMem agrees with its integer or real value.
**
** A single int or real value always converts to the same strings.  But
** many different strings can be converted into the same int or real.
................................................................................
*/
int sqlite3VdbeMemConsistentDualRep(Mem *p){
  char zBuf[100];
  char *z;
  int i, j, incr;
  if( (p->flags & MEM_Str)==0 ) return 1;
  if( (p->flags & (MEM_Int|MEM_Real))==0 ) return 1;
  vdbeMemRenderNum(sizeof(zBuf), zBuf, p);




  z = p->z;
  i = j = 0;
  incr = 1;
  if( p->enc!=SQLITE_UTF8 ){
    incr = 2;
    if( p->enc==SQLITE_UTF16BE ) z++;
  }
................................................................................
  assert( CORRUPT_DB || szNew>0 );
  assert( (pMem->flags & MEM_Dyn)==0 || pMem->szMalloc==0 );
  if( pMem->szMalloc<szNew ){
    return sqlite3VdbeMemGrow(pMem, szNew, 0);
  }
  assert( (pMem->flags & MEM_Dyn)==0 );
  pMem->z = pMem->zMalloc;
  pMem->flags &= (MEM_Null|MEM_Int|MEM_Real|MEM_IntReal);
  return SQLITE_OK;
}

/*
** It is already known that pMem contains an unterminated string.
** Add the zero terminator.
*/
................................................................................
** A MEM_Null value will never be passed to this function. This function is
** used for converting values to text for returning to the user (i.e. via
** sqlite3_value_text()), or for ensuring that values to be used as btree
** keys are strings. In the former case a NULL pointer is returned the
** user and the latter is an internal programming error.
*/
int sqlite3VdbeMemStringify(Mem *pMem, u8 enc, u8 bForce){

  const int nByte = 32;

  assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) );
  assert( !(pMem->flags&MEM_Zero) );
  assert( !(pMem->flags&(MEM_Str|MEM_Blob)) );
  assert( pMem->flags&(MEM_Int|MEM_Real) );
  assert( !sqlite3VdbeMemIsRowSet(pMem) );
  assert( EIGHT_BYTE_ALIGNMENT(pMem) );


  if( sqlite3VdbeMemClearAndResize(pMem, nByte) ){
    pMem->enc = 0;
    return SQLITE_NOMEM_BKPT;
  }








  vdbeMemRenderNum(nByte, pMem->z, pMem);




  assert( pMem->z!=0 );
  pMem->n = sqlite3Strlen30NN(pMem->z);
  pMem->enc = SQLITE_UTF8;
  pMem->flags |= MEM_Str|MEM_Term;
  if( bForce ) pMem->flags &= ~(MEM_Int|MEM_Real|MEM_IntReal);
  sqlite3VdbeChangeEncoding(pMem, enc);
  return SQLITE_OK;
}

/*
** Memory cell pMem contains the context of an aggregate function.
** This routine calls the finalize method for that function.  The
................................................................................
    }
    default: {
      assert( aff==SQLITE_AFF_TEXT );
      assert( MEM_Str==(MEM_Blob>>3) );
      pMem->flags |= (pMem->flags&MEM_Blob)>>3;
      sqlite3ValueApplyAffinity(pMem, SQLITE_AFF_TEXT, encoding);
      assert( pMem->flags & MEM_Str || pMem->db->mallocFailed );
      pMem->flags &= ~(MEM_Int|MEM_Real|MEM_IntReal|MEM_Blob|MEM_Zero);
      break;
    }
  }
}

/*
** Initialize bulk memory to be a consistent Mem object.

Changes to src/vtab.c.

837
838
839
840
841
842
843

844
845
846
847
848
849
850
851

852
853
854
855
856
857
858
      if( p->pVtab->nRef>0 ){
        return SQLITE_LOCKED;
      }
    }
    p = vtabDisconnectAll(db, pTab);
    xDestroy = p->pMod->pModule->xDestroy;
    assert( xDestroy!=0 );  /* Checked before the virtual table is created */

    rc = xDestroy(p->pVtab);
    /* Remove the sqlite3_vtab* from the aVTrans[] array, if applicable */
    if( rc==SQLITE_OK ){
      assert( pTab->pVTable==p && p->pNext==0 );
      p->pVtab = 0;
      pTab->pVTable = 0;
      sqlite3VtabUnlock(p);
    }

  }

  return rc;
}

/*
** This function invokes either the xRollback or xCommit method







>








>







837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
      if( p->pVtab->nRef>0 ){
        return SQLITE_LOCKED;
      }
    }
    p = vtabDisconnectAll(db, pTab);
    xDestroy = p->pMod->pModule->xDestroy;
    assert( xDestroy!=0 );  /* Checked before the virtual table is created */
    pTab->nTabRef++;
    rc = xDestroy(p->pVtab);
    /* Remove the sqlite3_vtab* from the aVTrans[] array, if applicable */
    if( rc==SQLITE_OK ){
      assert( pTab->pVTable==p && p->pNext==0 );
      p->pVtab = 0;
      pTab->pVTable = 0;
      sqlite3VtabUnlock(p);
    }
    sqlite3DeleteTable(db, pTab);
  }

  return rc;
}

/*
** This function invokes either the xRollback or xCommit method

Changes to src/whereexpr.c.

259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279

280
281
282
283
284
285
286
        zNew[cnt] = 0;
        for(iFrom=iTo=0; iFrom<cnt; iFrom++){
          if( zNew[iFrom]==wc[3] ) iFrom++;
          zNew[iTo++] = zNew[iFrom];
        }
        zNew[iTo] = 0;

        /* If the RHS begins with a digit or a minus sign, then the LHS must be
        ** an ordinary column (not a virtual table column) with TEXT affinity.
        ** Otherwise the LHS might be numeric and "lhs >= rhs" would be false
        ** even though "lhs LIKE rhs" is true.  But if the RHS does not start
        ** with a digit or '-', then "lhs LIKE rhs" will always be false if
        ** the LHS is numeric and so the optimization still works.
        **
        ** 2018-09-10 ticket c94369cae9b561b1f996d0054bfab11389f9d033
        ** The RHS pattern must not be '/%' because the termination condition
        ** will then become "x<'0'" and if the affinity is numeric, will then
        ** be converted into "x<0", which is incorrect.
        */
        if( sqlite3Isdigit(zNew[0])
         || zNew[0]=='-'

         || (zNew[0]+1=='0' && iTo==1)
        ){
          if( pLeft->op!=TK_COLUMN 
           || sqlite3ExprAffinity(pLeft)!=SQLITE_AFF_TEXT 
           || IsVirtual(pLeft->y.pTab)  /* Value might be numeric */
          ){
            sqlite3ExprDelete(db, pPrefix);







|



|









>







259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
        zNew[cnt] = 0;
        for(iFrom=iTo=0; iFrom<cnt; iFrom++){
          if( zNew[iFrom]==wc[3] ) iFrom++;
          zNew[iTo++] = zNew[iFrom];
        }
        zNew[iTo] = 0;

        /* If the RHS begins with a digit or a +/- sign, then the LHS must be
        ** an ordinary column (not a virtual table column) with TEXT affinity.
        ** Otherwise the LHS might be numeric and "lhs >= rhs" would be false
        ** even though "lhs LIKE rhs" is true.  But if the RHS does not start
        ** with a digit or +/-, then "lhs LIKE rhs" will always be false if
        ** the LHS is numeric and so the optimization still works.
        **
        ** 2018-09-10 ticket c94369cae9b561b1f996d0054bfab11389f9d033
        ** The RHS pattern must not be '/%' because the termination condition
        ** will then become "x<'0'" and if the affinity is numeric, will then
        ** be converted into "x<0", which is incorrect.
        */
        if( sqlite3Isdigit(zNew[0])
         || zNew[0]=='-'
         || zNew[0]=='+'
         || (zNew[0]+1=='0' && iTo==1)
        ){
          if( pLeft->op!=TK_COLUMN 
           || sqlite3ExprAffinity(pLeft)!=SQLITE_AFF_TEXT 
           || IsVirtual(pLeft->y.pTab)  /* Value might be numeric */
          ){
            sqlite3ExprDelete(db, pPrefix);

Changes to test/altertab3.test.

137
138
139
140
141
142
143
144

































145
146
147
    FOREIGN KEY (Col0) REFERENCES Table0
  );
}

do_execsql_test 6.1 {
  ALTER TABLE Table0 RENAME Col0 TO Col0;
}


































finish_test










>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>



137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
    FOREIGN KEY (Col0) REFERENCES Table0
  );
}

do_execsql_test 6.1 {
  ALTER TABLE Table0 RENAME Col0 TO Col0;
}

#-------------------------------------------------------------------------
reset_db
do_execsql_test 7.1.0 {
  CREATE TABLE t1(a,b,c);
  CREATE TRIGGER AFTER INSERT ON t1 BEGIN
    SELECT a, rank() OVER w1 FROM t1
    WINDOW w1 AS (PARTITION BY b, percent_rank() OVER w1);
  END;
}

do_execsql_test 7.1.2 {
  ALTER TABLE t1 RENAME TO t1x;
  SELECT sql FROM sqlite_master;
} {
  {CREATE TABLE "t1x"(a,b,c)}
  {CREATE TRIGGER AFTER INSERT ON "t1x" BEGIN
    SELECT a, rank() OVER w1 FROM "t1x"
    WINDOW w1 AS (PARTITION BY b, percent_rank() OVER w1);
  END}
}

do_execsql_test 7.2.1 {
  DROP TRIGGER after;
  CREATE TRIGGER AFTER INSERT ON t1x BEGIN
    SELECT a, rank() OVER w1 FROM t1x
    WINDOW w1 AS (PARTITION BY b, percent_rank() OVER w1 ORDER BY d);
  END;
}

do_catchsql_test 7.2.2 {
  ALTER TABLE t1x RENAME TO t1;
} {1 {error in trigger AFTER: no such column: d}}

finish_test


Added test/dbdata.test.







































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
# 2019-04-11
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this file is testing the sqlite_dbpage virtual table.
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
set testprefix dbdata

ifcapable !vtab||!compound {
  finish_test
  return
}
if { [catch { db enable_load_extension 1 }]
  || [catch { db eval { SELECT load_extension('../dbdata') } }] 
} {
  finish_test
  return
}

do_execsql_test 1.0 {
  CREATE TABLE T1(a, b);
  INSERT INTO t1(rowid, a ,b) VALUES(5, 'v', 'five');
  INSERT INTO t1(rowid, a, b) VALUES(10, 'x', 'ten');
}

do_execsql_test 1.1 {
  SELECT pgno, cell, field, quote(value) FROM sqlite_dbdata WHERE pgno=2;
} {
  2 0 -1 5 
  2 0  0 'v' 
  2 0  1 'five' 
  2 1 -1 10 
  2 1  0 'x' 
  2 1  1 'ten'
}

breakpoint
do_execsql_test 1.2 {
  SELECT pgno, cell, field, quote(value) FROM sqlite_dbdata;
} {
  1 0 -1 1 
  1 0 0 'table' 
  1 0 1 'T1' 
  1 0 2 'T1' 
  1 0 3 2 
  1 0 4 {'CREATE TABLE T1(a, b)'}
  2 0 -1 5 
  2 0  0 'v' 
  2 0  1 'five' 
  2 1 -1 10 
  2 1  0 'x' 
  2 1  1 'ten'
}

set big [string repeat big 2000]
do_execsql_test 1.3 {
  INSERT INTO t1 VALUES(NULL, $big);
  SELECT value FROM sqlite_dbdata WHERE pgno=2 AND cell=2 AND field=1;
} $big

do_execsql_test 1.4 {
  DELETE FROM t1;
  INSERT INTO t1 VALUES(NULL, randomblob(5050));
}
do_test 1.5 {
  execsql {
    SELECT quote(value) FROM sqlite_dbdata WHERE pgno=2 AND cell=0 AND field=1;
  }
} [db one {SELECT quote(b) FROM t1}]

#-------------------------------------------------------------------------
reset_db
db enable_load_extension 1
db eval { SELECT load_extension('../dbdata') }

do_execsql_test 2.0 {
  CREATE TABLE t1(a);
  CREATE INDEX i1 ON t1(a);
  WITH s(i) AS (
    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<10
  )
  INSERT INTO t1 SELECT randomblob(900) FROM s;
}

do_execsql_test 2.1 {
  SELECT * FROM sqlite_dbptr WHERE pgno=2;
} {
  2 25   2 6   2 7   2 9   2 11   2 13   2 15   2 17   2 19   2 21
}

do_execsql_test 2.2 {
  SELECT * FROM sqlite_dbptr WHERE pgno=3;
} {
  3 24   3 23
}

do_execsql_test 2.3 {
  SELECT * FROM sqlite_dbptr
} {
  2 25   2 6   2 7   2 9   2 11   2 13   2 15   2 17   2 19   2 21
  3 24   3 23
}


finish_test

Added test/fts4rename.test.

























































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# 2019 April 30
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#*************************************************************************
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/fts3_common.tcl
set ::testprefix fts4rename

# If SQLITE_ENABLE_FTS3 is defined, omit this file.
ifcapable !fts3 {
  finish_test
  return
}

do_execsql_test 1.0 {
  CREATE VIRTUAL TABLE temp.t1 USING fts3(a);
  BEGIN;
  CREATE TABLE t2(x);
} {}

do_catchsql_test 1.1 {
  ALTER TABLE t1_content RENAME c0a TO docid;
} {1 {duplicate column name: docid}}

do_catchsql_test 1.2 {
  UPDATE t1 SET Col0 = 1 ;
} {1 {no such column: Col0}}

do_catchsql_test 1.3 {
  ROLLBACK;
  DROP TABLE t1;
} {0 {}}

finish_test

Changes to test/index.test.

734
735
736
737
738
739
740









741









742
743
     CREATE TEMP TABLE t6(x);
     INSERT INTO temp.t6 values(1),(5),(9);
     CREATE INDEX temp.i21 ON t6(x);
     SELECT x FROM t6 ORDER BY x DESC;
  }
} {0 {9 5 1}}










   










finish_test







>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>


734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
     CREATE TEMP TABLE t6(x);
     INSERT INTO temp.t6 values(1),(5),(9);
     CREATE INDEX temp.i21 ON t6(x);
     SELECT x FROM t6 ORDER BY x DESC;
  }
} {0 {9 5 1}}

# 2019-05-01 ticket https://www.sqlite.org/src/info/3be1295b264be2fa
do_execsql_test index-22.0 {
  DROP TABLE IF EXISTS t1;
  CREATE TABLE t1(a, b TEXT);
  CREATE UNIQUE INDEX IF NOT EXISTS x1 ON t1(b==0);
  CREATE INDEX IF NOT EXISTS x2 ON t1(a || 0) WHERE b;
  INSERT INTO t1(a,b) VALUES('a',1),('a',0);
  SELECT a, b, '|' FROM t1;
} {a 1 | a 0 |}

# 2019-05-10 ticket https://www.sqlite.org/src/info/ae0f637bddc5290b
do_execsql_test index-23.0 {
  DROP TABLE t1;
  CREATE TABLE t1(a TEXT, b REAL);
  CREATE UNIQUE INDEX t1x1 ON t1(a GLOB b);
  INSERT INTO t1(a,b) VALUES('0.0','1'),('1.0','1');
  SELECT * FROM t1;
  REINDEX;
} {0.0 1.0 1.0 1.0}

finish_test

Changes to test/like3.test.

173
174
175
176
177
178
179










180
181
182
183
184

185
186
187
188
189
190
191
...
224
225
226
227
228
229
230

231
232
233
} {/abc}
do_eqp_test like3-5.211 {
  SELECT x FROM t5b WHERE x GLOB '/a*';
} {
  QUERY PLAN
  `--SEARCH TABLE t5b USING COVERING INDEX sqlite_autoindex_t5b_1 (x>? AND x<?)
}











# 2019-02-27
# Verify that the LIKE optimization works with an ESCAPE clause when
# using PRAGMA case_sensitive_like=ON.
#

do_execsql_test like3-6.100 {
  DROP TABLE IF EXISTS t1;
  CREATE TABLE t1(path TEXT COLLATE nocase PRIMARY KEY,a,b,c) WITHOUT ROWID;
}
do_eqp_test like3-6.110 {
  SELECT * FROM t1 WHERE path LIKE 'a%';
} {
................................................................................
  `--SEARCH TABLE t2 USING INDEX t2path2 (path>? AND path<?)
}
do_eqp_test like3-6.240 {
  SELECT * FROM t2 WHERE path LIKE 'a%' ESCAPE '_';
} {
  QUERY PLAN
  `--SEARCH TABLE t2 USING INDEX t2path2 (path>? AND path<?)

}

finish_test







>
>
>
>
>
>
>
>
>
>





>







 







>



173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
...
235
236
237
238
239
240
241
242
243
244
245
} {/abc}
do_eqp_test like3-5.211 {
  SELECT x FROM t5b WHERE x GLOB '/a*';
} {
  QUERY PLAN
  `--SEARCH TABLE t5b USING COVERING INDEX sqlite_autoindex_t5b_1 (x>? AND x<?)
}

# 2019-05-01
# another case of the above reported on the mailing list by Manual Rigger.
#
do_execsql_test like3-5.300 {
  CREATE TABLE t5c (c0 REAL);
  CREATE INDEX t5c_0 ON t5c(c0 COLLATE NOCASE);
  INSERT INTO t5c(rowid, c0) VALUES (99,'+/');
  SELECT * FROM t5c WHERE (c0 LIKE '+/');
} {+/}

# 2019-02-27
# Verify that the LIKE optimization works with an ESCAPE clause when
# using PRAGMA case_sensitive_like=ON.
#
ifcapable !icu {
do_execsql_test like3-6.100 {
  DROP TABLE IF EXISTS t1;
  CREATE TABLE t1(path TEXT COLLATE nocase PRIMARY KEY,a,b,c) WITHOUT ROWID;
}
do_eqp_test like3-6.110 {
  SELECT * FROM t1 WHERE path LIKE 'a%';
} {
................................................................................
  `--SEARCH TABLE t2 USING INDEX t2path2 (path>? AND path<?)
}
do_eqp_test like3-6.240 {
  SELECT * FROM t2 WHERE path LIKE 'a%' ESCAPE '_';
} {
  QUERY PLAN
  `--SEARCH TABLE t2 USING INDEX t2path2 (path>? AND path<?)
}
}

finish_test

Changes to test/oserror.test.

48
49
50
51
52
53
54



55
56
57
58
59




60





61
62

63
64
65
66

67
68
69
70
71
72
73
#
# The xOpen() method of the unix VFS calls getcwd() as well as open().
# Although this does not appear to be documented in the man page, on OSX
# a call to getcwd() may fail if there are no free file descriptors. So
# an error may be reported for either open() or getcwd() here.
#
if {![clang_sanitize_address]} {



  do_test 1.1.1 {
    set ::log [list]
    list [catch {
      for {set i 0} {$i < 20000} {incr i} { sqlite3 dbh_$i test.db -readonly 1 }
    } msg] $msg




  } {1 {unable to open database file}}





  do_test 1.1.2 {
    catch { for {set i 0} {$i < 20000} {incr i} { dbh_$i close } }

  } {1}
  do_re_test 1.1.3 { 
    lindex $::log 0 
  } {^os_unix.c:\d+: \(\d+\) (open|getcwd)\(.*test.db\) - }

}


# Test a failure in open() due to the path being a directory.
#
do_test 1.2.1 {
  file mkdir dir.db







>
>
>


|
|
|
>
>
>
>
|
>
>
>
>
>

|
>
|
|
|
|
>







48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
#
# The xOpen() method of the unix VFS calls getcwd() as well as open().
# Although this does not appear to be documented in the man page, on OSX
# a call to getcwd() may fail if there are no free file descriptors. So
# an error may be reported for either open() or getcwd() here.
#
if {![clang_sanitize_address]} {
  unset -nocomplain rc
  unset -nocomplain nOpen
  set nOpen 20000
  do_test 1.1.1 {
    set ::log [list]
    set ::rc [catch {
      for {set i 0} {$i < $::nOpen} {incr i} { sqlite3 dbh_$i test.db -readonly 1 }
    } msg]
    if {$::rc==0} {
      # Some system (ex: Debian) are able to create 20000+ file descriptiors
      # such systems will not fail here
      set x ok
    } elseif {$::rc==1 && $msg=="unable to open database file"} {
      set x ok
    } else {
      set x [list $::rc $msg]
    }
  } {ok}
  do_test 1.1.2 {
    catch { for {set i 0} {$i < $::nOpen} {incr i} { dbh_$i close } }
  } $::rc
  if {$rc} {
    do_re_test 1.1.3 { 
      lindex $::log 0 
    } {^os_unix.c:\d+: \(\d+\) (open|getcwd)\(.*test.db\) - }
  }
}


# Test a failure in open() due to the path being a directory.
#
do_test 1.2.1 {
  file mkdir dir.db

Added test/recover.test.





































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
# 2019 April 23
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# Test the shell tool ".ar" command.
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
set testprefix recover

ifcapable !vtab {
  finish_test; return
}
set CLI [test_find_cli]

proc compare_result {db1 db2 sql} {
  set r1 [$db1 eval $sql]
  set r2 [$db2 eval $sql]
  if {$r1 != $r2} {
  puts "r1: $r1"
  puts "r2: $r2"
    error "mismatch for $sql"
  }
  return ""
}

proc compare_dbs {db1 db2} {
  compare_result $db1 $db2 "SELECT sql FROM sqlite_master ORDER BY 1"
  foreach tbl [$db1 eval {SELECT name FROM sqlite_master WHERE type='table'}] {
    compare_result $db1 $db2 "SELECT * FROM $tbl"
  }
}

proc do_recover_test {tn {tsql {}} {res {}}} {
  set fd [open "|$::CLI test.db .recover"]
  fconfigure $fd -encoding binary
  fconfigure $fd -translation binary
  set sql [read $fd]
  close $fd

  forcedelete test.db2
  sqlite3 db2 test.db2
  execsql $sql db2
  if {$tsql==""} {
    uplevel [list do_test $tn [list compare_dbs db db2] {}]
  } else {
    uplevel [list do_execsql_test -db db2 $tn $tsql $res]
  }
  db2 close
}

set doc {
  hello
  world
}
do_execsql_test 1.1.1 {
  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
  INSERT INTO t1 VALUES(1, 4, X'1234567800');
  INSERT INTO t1 VALUES(2, 'test', 8.1);
  INSERT INTO t1 VALUES(3, $doc, 8.4);
}
do_recover_test 1.1.2

do_execsql_test 1.2.1 "
  DELETE FROM t1;
  INSERT INTO t1 VALUES(13, 'hello\r\nworld', 13);
"
do_recover_test 1.2.2

do_execsql_test 1.3.1 "
  CREATE TABLE t2(i INTEGER PRIMARY KEY AUTOINCREMENT, b, c);
  INSERT INTO t2 VALUES(NULL, 1, 2);
  INSERT INTO t2 VALUES(NULL, 3, 4);
  INSERT INTO t2 VALUES(NULL, 5, 6);
  CREATE TABLE t3(i INTEGER PRIMARY KEY AUTOINCREMENT, b, c);
  INSERT INTO t3 VALUES(NULL, 1, 2);
  INSERT INTO t3 VALUES(NULL, 3, 4);
  INSERT INTO t3 VALUES(NULL, 5, 6);
  DELETE FROM t2;
"
do_recover_test 1.3.2

#-------------------------------------------------------------------------
reset_db
do_execsql_test 2.1.0 {
  PRAGMA auto_vacuum = 0;
  CREATE TABLE t1(a, b, c, PRIMARY KEY(b, c)) WITHOUT ROWID;
  INSERT INTO t1 VALUES(1, 2, 3);
  INSERT INTO t1 VALUES(4, 5, 6);
  INSERT INTO t1 VALUES(7, 8, 9);
}

do_recover_test 2.1.1

do_execsql_test 2.2.0 {
  PRAGMA writable_schema = 1;
  DELETE FROM sqlite_master WHERE name='t1';
}
do_recover_test 2.2.1 {
  SELECT name FROM sqlite_master
} {lost_and_found}

do_execsql_test 2.3.0 {
  CREATE TABLE lost_and_found(a, b, c);
}
do_recover_test 2.3.1 {
  SELECT name FROM sqlite_master
} {lost_and_found lost_and_found_0}

do_execsql_test 2.4.0 {
  CREATE TABLE lost_and_found_0(a, b, c);
}
do_recover_test 2.4.1 {
  SELECT name FROM sqlite_master;
  SELECT * FROM lost_and_found_1;
} {lost_and_found lost_and_found_0 lost_and_found_1
  2 2 3 {} 2 3 1
  2 2 3 {} 5 6 4
  2 2 3 {} 8 9 7
}

finish_test

Changes to test/wapptest.tcl.

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
..
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

74
75

76

77
78
79
80
81
82
83
...
204
205
206
207
208
209
210




211
212
213
214
215
216
217
218
219
220
221
222
223











224












225
226
227
228
229
230
231
...
234
235
236
237
238
239
240





























































































241
242
243
244
245
246
247
...
271
272
273
274
275
276
277

278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
...
299
300
301
302
303
304
305
306
307

308
309
310
311
312
313
314
#   G(keep)     - Boolean. True to delete no files after each test.
#   G(msvc)     - Boolean. True to use MSVC as the compiler.
#   G(tcl)      - Use Tcl from this directory for builds.
#   G(jobs)     - How many sub-processes to run simultaneously.
#
set G(platform) $::tcl_platform(os)-$::tcl_platform(machine)
set G(test)     Normal
set G(keep)     0
set G(msvc)     0
set G(tcl)      [::tcl::pkgconfig get libdir,install]
set G(jobs)     3
set G(debug)    0

proc wapptest_init {} {
  global G

................................................................................
  foreach k $lSave { set A($k) $G($k) }
  array unset G
  foreach k $lSave { set G($k) $A($k) }

  # The root of the SQLite source tree.
  set G(srcdir)   [file dirname [file dirname [info script]]]

  # releasetest.tcl script
  set G(releaseTest) [file join [file dirname [info script]] releasetest.tcl]

  set G(sqlite_version) "unknown"

  # Either "config", "running" or "stopped":
  set G(state) "config"

  set G(hostname) "(unknown host)"
  catch { set G(hostname) [exec hostname] } 
  set G(host) $G(hostname)
  append G(host) " $::tcl_platform(os) $::tcl_platform(osVersion)"
  append G(host) " $::tcl_platform(machine) $::tcl_platform(byteOrder)"
}

# Check to see if there are uncommitted changes in the SQLite source
# directory. Return true if there are, or false otherwise.
#
proc check_uncommitted {} {
  global G
  set ret 0
  set pwd [pwd]
  cd $G(srcdir)
  if {[catch {exec fossil changes} res]==0 && [string trim $res]!=""} {
    set ret 1
  }
  cd $pwd
  return $ret
}

proc generate_fossil_info {} {
  global G
  set pwd [pwd]
  cd $G(srcdir)

  if {[catch {exec fossil info}    r1]} return
  if {[catch {exec fossil changes} r2]} return

  cd $pwd


  foreach line [split $r1 "\n"] {
    if {[regexp {^checkout: *(.*)$} $line -> co]} {
      wapp-trim { <br> %html($co) }
    }
  }

................................................................................
    set G(test.$name.errmsg) "Test did not complete"
    if {[file readable core]} {
      append G(test.$name.errmsg) " - core file exists"
    }
  }
}





proc slave_test_done {name rc} {
  global G
  set G(test.$name.done) [clock seconds]
  set G(test.$name.nError) 0
  set G(test.$name.nTest) 0
  set G(test.$name.errmsg) ""
  if {$rc} {
    incr G(test.$name.nError)
  }
  if {[file exists $G(test.$name.log)]} {
    count_tests_and_errors $name $G(test.$name.log)
  }
}
























proc slave_fileevent {name} {
  global G
  set fd $G(test.$name.channel)

  if {[eof $fd]} {
    fconfigure $fd -blocking 1
    set rc [catch { close $fd }]
................................................................................
  } else {
    set line [gets $fd]
    if {[string trim $line] != ""} { puts "Trace   : $name - \"$line\"" }
  }

  do_some_stuff
}






























































































proc do_some_stuff {} {
  global G

  # Count the number of running jobs. A running job has an entry named
  # "channel" in its dictionary.
  set nRunning 0
................................................................................
    set nLaunch [expr $G(jobs) - $nRunning]
    foreach j $G(test_array) {
      if {$nLaunch<=0} break
      set name [dict get $j config]
      if { ![info exists G(test.$name.channel)]
        && ![info exists G(test.$name.done)]
      } {

        set target [dict get $j target]
        set G(test.$name.start) [clock seconds]
        set fd [open "|[info nameofexecutable] $G(releaseTest) --slave" r+]
        set G(test.$name.channel) $fd
        fconfigure $fd -blocking 0
        fileevent $fd readable [list slave_fileevent $name]

        puts $fd [list 0 $G(msvc) 0 $G(keep)]

        set wtcl ""
        if {$G(tcl)!=""} { set wtcl "--with-tcl=$G(tcl)" }

        # If this configuration is named <name>-(Debug) or <name>-(NDebug),
        # then add or remove the SQLITE_DEBUG option from the base
        # configuration before running the test.
        if {[regexp -- {(.*)-(\(.*\))} $name -> head tail]} {
................................................................................
            regsub { *-DSQLITE_DEBUG[^ ]* *} $opts { } opts
          }
        } else {
          set opts $::Configs($name)
        }

        set L [make_test_suite $G(msvc) $wtcl $name $target $opts]
        puts $fd $L
        flush $fd

        set G(test.$name.log) [file join [lindex $L 1] test.log]
        incr nLaunch -1
      }
    }
  }
}








|
|







 







<
<
<












|
|
<
|
<
<
<
<
<
<
<
<
<
<
|




>
|
|
>

>







 







>
>
>
>












|
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>


<
<
<
<
<
<
<







 







|
|
>







16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
..
33
34
35
36
37
38
39



40
41
42
43
44
45
46
47
48
49
50
51
52
53

54










55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
...
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
...
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
...
380
381
382
383
384
385
386
387
388
389







390
391
392
393
394
395
396
...
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
#   G(keep)     - Boolean. True to delete no files after each test.
#   G(msvc)     - Boolean. True to use MSVC as the compiler.
#   G(tcl)      - Use Tcl from this directory for builds.
#   G(jobs)     - How many sub-processes to run simultaneously.
#
set G(platform) $::tcl_platform(os)-$::tcl_platform(machine)
set G(test)     Normal
set G(keep)     1
set G(msvc)     [expr {$::tcl_platform(platform)=="windows"}]
set G(tcl)      [::tcl::pkgconfig get libdir,install]
set G(jobs)     3
set G(debug)    0

proc wapptest_init {} {
  global G

................................................................................
  foreach k $lSave { set A($k) $G($k) }
  array unset G
  foreach k $lSave { set G($k) $A($k) }

  # The root of the SQLite source tree.
  set G(srcdir)   [file dirname [file dirname [info script]]]




  set G(sqlite_version) "unknown"

  # Either "config", "running" or "stopped":
  set G(state) "config"

  set G(hostname) "(unknown host)"
  catch { set G(hostname) [exec hostname] } 
  set G(host) $G(hostname)
  append G(host) " $::tcl_platform(os) $::tcl_platform(osVersion)"
  append G(host) " $::tcl_platform(machine) $::tcl_platform(byteOrder)"
}

# Generate the text for the box at the top of the UI. The current SQLite
# version, according to fossil, along with a warning if there are 

# uncommitted changes in the checkout.










#
proc generate_fossil_info {} {
  global G
  set pwd [pwd]
  cd $G(srcdir)
  set rc [catch {
    set r1 [exec fossil info]
    set r2 [exec fossil changes]
  }]
  cd $pwd
  if {$rc} return

  foreach line [split $r1 "\n"] {
    if {[regexp {^checkout: *(.*)$} $line -> co]} {
      wapp-trim { <br> %html($co) }
    }
  }

................................................................................
    set G(test.$name.errmsg) "Test did not complete"
    if {[file readable core]} {
      append G(test.$name.errmsg) " - core file exists"
    }
  }
}

# This command is invoked once a slave process has finished running its
# tests, successfully or otherwise. Parameter $name is the name of the 
# test, $rc the exit code returned by the slave process.
#
proc slave_test_done {name rc} {
  global G
  set G(test.$name.done) [clock seconds]
  set G(test.$name.nError) 0
  set G(test.$name.nTest) 0
  set G(test.$name.errmsg) ""
  if {$rc} {
    incr G(test.$name.nError)
  }
  if {[file exists $G(test.$name.log)]} {
    count_tests_and_errors $name $G(test.$name.log)
  }

  # If the "keep files" checkbox is clear, delete all files except for
  # the executables and test logs. And any core file that is present.
  if {$G(keep)==0} {
    set keeplist {
      testfixture testfixture.exe
      sqlite3 sqlite3.exe
      test.log test-out.txt
      core
      wapptest_make.sh
      wapptest_configure.sh
      wapptest_run.tcl
    }
    foreach f [glob -nocomplain [file join $G(test.$name.dir) *]] {
      set t [file tail $f]
      if {[lsearch $keeplist $t]<0} {
        catch { file delete -force $f }
      }
    }
  }
}

# This is a fileevent callback invoked each time a file-descriptor that
# connects this process to a slave process is readable.
#
proc slave_fileevent {name} {
  global G
  set fd $G(test.$name.channel)

  if {[eof $fd]} {
    fconfigure $fd -blocking 1
    set rc [catch { close $fd }]
................................................................................
  } else {
    set line [gets $fd]
    if {[string trim $line] != ""} { puts "Trace   : $name - \"$line\"" }
  }

  do_some_stuff
}

# Return the contents of the "slave script" - the script run by slave 
# processes to actually perform the test. It does two things:
#
#   1. Reads and [exec]s the contents of file wapptest_configure.sh.
#   2. Reads and [exec]s the contents of file wapptest_make.sh.
#
# Step 1 is omitted if the test uses MSVC (which does not use configure).
#
proc wapptest_slave_script {} {
  global G
  set res {
    proc readfile {filename} {
      set fd [open $filename]
      set data [read $fd]
      close $fd
      return $data
    }
  }

  if {$G(msvc)==0} { 
    append res {
      set cfg  [readfile wapptest_configure.sh]
      set rc [catch { exec {*}$cfg >& test.log } msg]
      if {$rc==0} {
        set make [readfile wapptest_make.sh]
        set rc [catch { exec {*}$make >>& test.log }]
      }
    } 
  } else { 
    append res {
      set make [readfile wapptest_make.sh]
      set rc [catch { exec {*}$make >>& test.log }]
    }
  }

  append res { exit $rc }

  set res
}


# Launch a slave process to run a test.
#
proc slave_launch {
  name wtcl title dir configOpts testtarget makeOpts cflags opts
} {
  global G

  catch { file mkdir $dir } msg
  foreach f [glob -nocomplain [file join $dir *]] {
    catch { file delete -force $f }
  }
  set G(test.$name.dir) $dir

  # Write the configure command to wapptest_configure.sh. This file
  # is empty if using MSVC - MSVC does not use configure.
  #
  set fd1 [open [file join $dir wapptest_configure.sh] w]
  if {$G(msvc)==0} {
    puts $fd1 "[file join .. $G(srcdir) configure] $wtcl $configOpts"
  }
  close $fd1

  # Write the make command to wapptest_make.sh. Using nmake for MSVC and
  # make for all other systems.
  #
  set makecmd "make"
  if {$G(msvc)} { 
    set nativedir [file nativename $G(srcdir)]
    set nativedir [string map [list "\\" "\\\\"] $nativedir]
    set makecmd "nmake /f [file join $nativedir Makefile.msc] TOP=$nativedir"
  }
  set fd2 [open [file join $dir wapptest_make.sh] w]
  puts $fd2 "$makecmd $makeOpts $testtarget \"CFLAGS=$cflags\" \"OPTS=$opts\""
  close $fd2

  # Write the wapptest_run.tcl script to the test directory. To run the
  # commands in the other two files.
  #
  set fd3 [open [file join $dir wapptest_run.tcl] w]
  puts $fd3 [wapptest_slave_script]
  close $fd3

  set pwd [pwd]
  cd $dir
  set fd [open "|[info nameofexecutable] wapptest_run.tcl" r+]
  cd $pwd

  set G(test.$name.channel) $fd
  fconfigure $fd -blocking 0
  fileevent $fd readable [list slave_fileevent $name]
}

proc do_some_stuff {} {
  global G

  # Count the number of running jobs. A running job has an entry named
  # "channel" in its dictionary.
  set nRunning 0
................................................................................
    set nLaunch [expr $G(jobs) - $nRunning]
    foreach j $G(test_array) {
      if {$nLaunch<=0} break
      set name [dict get $j config]
      if { ![info exists G(test.$name.channel)]
        && ![info exists G(test.$name.done)]
      } {

        set target [dict get $j target]
        set G(test.$name.start) [clock seconds]







        set wtcl ""
        if {$G(tcl)!=""} { set wtcl "--with-tcl=$G(tcl)" }

        # If this configuration is named <name>-(Debug) or <name>-(NDebug),
        # then add or remove the SQLITE_DEBUG option from the base
        # configuration before running the test.
        if {[regexp -- {(.*)-(\(.*\))} $name -> head tail]} {
................................................................................
            regsub { *-DSQLITE_DEBUG[^ ]* *} $opts { } opts
          }
        } else {
          set opts $::Configs($name)
        }

        set L [make_test_suite $G(msvc) $wtcl $name $target $opts]
        set G(test.$name.log) [file join [lindex $L 1] test.log]
        slave_launch $name $wtcl {*}$L

        set G(test.$name.log) [file join [lindex $L 1] test.log]
        incr nLaunch -1
      }
    }
  }
}

Changes to test/without_rowid1.test.

387
388
389
390
391
392
393








394
395
  CREATE TRIGGER t1_tr BEFORE UPDATE ON t1 BEGIN
    DELETE FROM t1 WHERE a = new.a;
  END;
  UPDATE t1 SET c = c+1 WHERE a = 'a';
  SELECT * FROM t1;
} {b a 3  b b 4}









  
finish_test







>
>
>
>
>
>
>
>


387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
  CREATE TRIGGER t1_tr BEFORE UPDATE ON t1 BEGIN
    DELETE FROM t1 WHERE a = new.a;
  END;
  UPDATE t1 SET c = c+1 WHERE a = 'a';
  SELECT * FROM t1;
} {b a 3  b b 4}

# 2019-04-29 ticket https://www.sqlite.org/src/info/3182d3879020ef3
do_execsql_test 11.1 {
  CREATE TABLE t11(a TEXT PRIMARY KEY, b INT) WITHOUT ROWID;
  CREATE INDEX t11a ON t11(a COLLATE NOCASE);
  INSERT INTO t11(a,b) VALUES ('A',1),('a',2);
  PRAGMA integrity_check;
  SELECT a FROM t11 ORDER BY a COLLATE binary;
} {ok A a}
  
finish_test

Changes to tool/lemon.c.

3844
3845
3846
3847
3848
3849
3850
3851
3852
3853
3854
3855
3856
3857
3858
    if( rp->rhsalias[i] ){
      if( i>0 ){
        int j;
        if( rp->lhsalias && strcmp(rp->lhsalias,rp->rhsalias[i])==0 ){
          ErrorMsg(lemp->filename,rp->ruleline,
            "%s(%s) has the same label as the LHS but is not the left-most "
            "symbol on the RHS.",
            rp->rhs[i]->name, rp->rhsalias);
          lemp->errorcnt++;
        }
        for(j=0; j<i; j++){
          if( rp->rhsalias[j] && strcmp(rp->rhsalias[j],rp->rhsalias[i])==0 ){
            ErrorMsg(lemp->filename,rp->ruleline,
              "Label %s used for multiple symbols on the RHS of a rule.",
              rp->rhsalias[i]);







|







3844
3845
3846
3847
3848
3849
3850
3851
3852
3853
3854
3855
3856
3857
3858
    if( rp->rhsalias[i] ){
      if( i>0 ){
        int j;
        if( rp->lhsalias && strcmp(rp->lhsalias,rp->rhsalias[i])==0 ){
          ErrorMsg(lemp->filename,rp->ruleline,
            "%s(%s) has the same label as the LHS but is not the left-most "
            "symbol on the RHS.",
            rp->rhs[i]->name, rp->rhsalias[i]);
          lemp->errorcnt++;
        }
        for(j=0; j<i; j++){
          if( rp->rhsalias[j] && strcmp(rp->rhsalias[j],rp->rhsalias[i])==0 ){
            ErrorMsg(lemp->filename,rp->ruleline,
              "Label %s used for multiple symbols on the RHS of a rule.",
              rp->rhsalias[i]);

Changes to tool/mkshellc.tcl.

36
37
38
39
40
41
42

43
44
45

46
47
48

49
50
51
52


53
54
55
56
57
58
59
60

61
62
63
64
65
66
    if {[info exists typedef_seen($line)]} {
      return "/* $line */"
    }
    set typedef_seen($line) 1
  }
  return $line
}

while {1} {
  set lx [omit_redundant_typedefs [gets $in]]
  if {[eof $in]} break;

  if {[regexp {^INCLUDE } $lx]} {
    set cfile [lindex $lx 1]
    puts $out "/************************* Begin $cfile ******************/"

    set in2 [open $topdir/src/$cfile rb]
    while {![eof $in2]} {
      set lx [omit_redundant_typedefs [gets $in2]]
      if {[regexp {^#include "sqlite} $lx]} continue


      if {[regexp {^# *include "test_windirent.h"} $lx]} {
        set lx "/* $lx */"
      }
      set lx [string map [list __declspec(dllexport) {}] $lx]
      puts $out $lx
    }
    close $in2
    puts $out "/************************* End $cfile ********************/"

    continue
  }
  puts $out $lx
}
close $in
close $out







>



>



>



|
>
>








>






36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
    if {[info exists typedef_seen($line)]} {
      return "/* $line */"
    }
    set typedef_seen($line) 1
  }
  return $line
}
set iLine 0
while {1} {
  set lx [omit_redundant_typedefs [gets $in]]
  if {[eof $in]} break;
  incr iLine
  if {[regexp {^INCLUDE } $lx]} {
    set cfile [lindex $lx 1]
    puts $out "/************************* Begin $cfile ******************/"
#   puts $out "#line 1 \"$cfile\""
    set in2 [open $topdir/src/$cfile rb]
    while {![eof $in2]} {
      set lx [omit_redundant_typedefs [gets $in2]]
      if {[regexp {^#include "sqlite} $lx]} {
        set lx "/* $lx */"
      }
      if {[regexp {^# *include "test_windirent.h"} $lx]} {
        set lx "/* $lx */"
      }
      set lx [string map [list __declspec(dllexport) {}] $lx]
      puts $out $lx
    }
    close $in2
    puts $out "/************************* End $cfile ********************/"
#   puts $out "#line [expr $iLine+1] \"shell.c.in\""
    continue
  }
  puts $out $lx
}
close $in
close $out