/ Check-in [42d07c70]
Login
SQLite training in Houston TX on 2019-11-05 (details)
Part of the 2019 Tcl Conference

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Rework accessPayload() function for clarity. (CVS 3912)
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 42d07c70ec1eb6dd9619c97d753c9d2824aeae32
User & Date: danielk1977 2007-05-04 08:32:14
Context
2007-05-04
11:59
Add compile-time option -DSQLITE_MIXED_ENDIAN_64BIT_FLOAT=1 that uses mixed-endian doubles. This is needed on ARM7 to make database file formats compatible with all other processors. Tickets #2278 and #2335. (CVS 3913) check-in: 2a178d0c user: drh tags: trunk
08:32
Rework accessPayload() function for clarity. (CVS 3912) check-in: 42d07c70 user: danielk1977 tags: trunk
2007-05-03
20:06
fix from Gentoo for allowing TCLLIBDIR to be overridden in env (CVS 3911) check-in: e54a49e2 user: vapier tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to src/btree.c.

     5      5   ** a legal notice, here is a blessing:
     6      6   **
     7      7   **    May you do good and not evil.
     8      8   **    May you find forgiveness for yourself and forgive others.
     9      9   **    May you share freely, never taking more than you give.
    10     10   **
    11     11   *************************************************************************
    12         -** $Id: btree.c,v 1.370 2007/05/03 13:11:32 danielk1977 Exp $
           12  +** $Id: btree.c,v 1.371 2007/05/04 08:32:14 danielk1977 Exp $
    13     13   **
    14     14   ** This file implements a external (disk-based) database using BTrees.
    15     15   ** For a detailed discussion of BTrees, refer to
    16     16   **
    17     17   **     Donald E. Knuth, THE ART OF COMPUTER PROGRAMMING, Volume 3:
    18     18   **     "Sorting And Searching", pages 473-480. Addison-Wesley
    19     19   **     Publishing Company, Reading, Massachusetts.
................................................................................
  3133   3133       }
  3134   3134     }
  3135   3135     *pPgnoNext = next;
  3136   3136   
  3137   3137     return rc;
  3138   3138   }
  3139   3139   
         3140  +/*
         3141  +** Copy data from a buffer to a page, or from a page to a buffer.
         3142  +**
         3143  +** pPayload is a pointer to data stored on database page pDbPage.
         3144  +** If argument eOp is false, then nByte bytes of data are copied
         3145  +** from pPayload to the buffer pointed at by pBuf. If eOp is true,
         3146  +** then sqlite3PagerWrite() is called on pDbPage and nByte bytes
         3147  +** of data are copied from the buffer pBuf to pPayload.
         3148  +**
         3149  +** SQLITE_OK is returned on success, otherwise an error code.
         3150  +*/
         3151  +static int copyPayload(
         3152  +  void *pPayload,           /* Pointer to page data */
         3153  +  void *pBuf,               /* Pointer to buffer */
         3154  +  int nByte,                /* Number of bytes to copy */
         3155  +  int eOp,                  /* 0 -> copy from page, 1 -> copy to page */
         3156  +  DbPage *pDbPage           /* Page containing pPayload */
         3157  +){
         3158  +  if( eOp ){
         3159  +    /* Copy data from buffer to page (a write operation) */
         3160  +    int rc = sqlite3PagerWrite(pDbPage);
         3161  +    if( rc!=SQLITE_OK ){
         3162  +      return rc;
         3163  +    }
         3164  +    memcpy(pPayload, pBuf, nByte);
         3165  +  }else{
         3166  +    /* Copy data from page to buffer (a read operation) */
         3167  +    memcpy(pBuf, pPayload, nByte);
         3168  +  }
         3169  +  return SQLITE_OK;
         3170  +}
  3140   3171   
  3141   3172   /*
  3142   3173   ** This function is used to read or overwrite payload information
  3143   3174   ** for the entry that the pCur cursor is pointing to. If the eOp
  3144   3175   ** parameter is 0, this is a read operation (data copied into
  3145   3176   ** buffer pBuf). If it is non-zero, a write (data copied from
  3146   3177   ** buffer pBuf).
................................................................................
  3148   3179   ** A total of "amt" bytes are read or written beginning at "offset".
  3149   3180   ** Data is read to or from the buffer pBuf.
  3150   3181   **
  3151   3182   ** This routine does not make a distinction between key and data.
  3152   3183   ** It just reads or writes bytes from the payload area.  Data might 
  3153   3184   ** appear on the main page or be scattered out on multiple overflow 
  3154   3185   ** pages.
         3186  +**
         3187  +** If the BtCursor.cacheOverflow flag is set, and the current
         3188  +** cursor entry uses one or more overflow pages, this function
         3189  +** allocates space for and lazily popluates the overflow page-list 
         3190  +** cache array (BtCursor.aOverflow). Subsequent calls use this
         3191  +** cache to make seeking to the supplied offset more efficient.
         3192  +**
         3193  +** Once an overflow page-list cache has been allocated, it may be
         3194  +** invalidated if some other cursor writes to the same table, or if
         3195  +** the cursor is moved to a different row. Additionally, in auto-vacuum
         3196  +** mode, the following events may invalidate an overflow page-list cache.
         3197  +**
         3198  +**   * An incremental vacuum,
         3199  +**   * A commit in auto_vacuum="full" mode,
         3200  +**   * Creating a table (may require moving an overflow page).
  3155   3201   */
  3156   3202   #define getPayload(a,b,c,d,e) accessPayload(a,b,c,d,e,0)
  3157   3203   static int accessPayload(
  3158   3204     BtCursor *pCur,      /* Cursor pointing to entry to read from */
  3159   3205     int offset,          /* Begin reading this far into payload */
  3160   3206     int amt,             /* Read this many bytes */
  3161   3207     unsigned char *pBuf, /* Write the bytes into this buffer */ 
  3162   3208     int skipKey,         /* offset begins at data if this is true */
  3163   3209     int eOp              /* zero to read. non-zero to write. */
  3164   3210   ){
  3165   3211     unsigned char *aPayload;
  3166         -  Pgno nextPage;
  3167         -  int rc;
  3168         -  MemPage *pPage;
  3169         -  BtShared *pBt;
  3170         -  int ovflSize;
         3212  +  int rc = SQLITE_OK;
  3171   3213     u32 nKey;
  3172   3214     int iIdx = 0;
         3215  +  MemPage *pPage = pCur->pPage;        /* Btree page of current cursor entry */
         3216  +  BtShared *pBt = pCur->pBtree->pBt;   /* Btree this cursor belongs to */
  3173   3217   
  3174         -  assert( pCur!=0 && pCur->pPage!=0 );
         3218  +  assert( pPage );
  3175   3219     assert( pCur->eState==CURSOR_VALID );
  3176         -  pBt = pCur->pBtree->pBt;
  3177         -  pPage = pCur->pPage;
  3178   3220     assert( pCur->idx>=0 && pCur->idx<pPage->nCell );
         3221  +  assert( offset>=0 );
         3222  +
  3179   3223     getCellInfo(pCur);
  3180   3224     aPayload = pCur->info.pCell + pCur->info.nHeader;
  3181         -  if( pPage->intKey ){
  3182         -    nKey = 0;
  3183         -  }else{
  3184         -    nKey = pCur->info.nKey;
  3185         -  }
  3186         -  assert( offset>=0 );
         3225  +  nKey = (pPage->intKey ? 0 : pCur->info.nKey);
         3226  +
  3187   3227     if( skipKey ){
  3188   3228       offset += nKey;
  3189   3229     }
  3190   3230     if( offset+amt > nKey+pCur->info.nData ){
         3231  +    /* Trying to read or write past the end of the data is an error */
  3191   3232       return SQLITE_ERROR;
  3192   3233     }
         3234  +
         3235  +  /* Check if data must be read/written to/from the btree page itself. */
  3193   3236     if( offset<pCur->info.nLocal ){
  3194   3237       int a = amt;
  3195   3238       if( a+offset>pCur->info.nLocal ){
  3196   3239         a = pCur->info.nLocal - offset;
  3197   3240       }
  3198         -    if( eOp ){
  3199         -      /* A write operation. */
  3200         -      rc = sqlite3PagerWrite(pPage->pDbPage);
  3201         -      if( rc!=SQLITE_OK ){
  3202         -        return rc;
  3203         -      }
  3204         -      memcpy(&aPayload[offset], pBuf, a);
  3205         -    }else{
  3206         -      /* A read operation */
  3207         -      memcpy(pBuf, &aPayload[offset], a);
  3208         -    }
  3209         -    if( a==amt ){
  3210         -      return SQLITE_OK;
  3211         -    }
         3241  +    rc = copyPayload(&aPayload[offset], pBuf, a, eOp, pPage->pDbPage);
  3212   3242       offset = 0;
  3213   3243       pBuf += a;
  3214   3244       amt -= a;
  3215   3245     }else{
  3216   3246       offset -= pCur->info.nLocal;
  3217   3247     }
  3218         -  ovflSize = pBt->usableSize - 4;
  3219         -  if( amt>0 ){
         3248  +
         3249  +  if( rc==SQLITE_OK && amt>0 ){
         3250  +    const int ovflSize = pBt->usableSize - 4;  /* Bytes content per ovfl page */
         3251  +    Pgno nextPage;
         3252  +
  3220   3253       nextPage = get4byte(&aPayload[pCur->info.nLocal]);
         3254  +
  3221   3255   #ifndef SQLITE_OMIT_INCRBLOB
         3256  +    /* If the cacheOverflow flag is set and the BtCursor.aOverflow[]
         3257  +    ** has not been allocated, allocate it now. The array is sized at
         3258  +    ** one entry for each overflow page in the overflow chain. The
         3259  +    ** page number of the first overflow page is stored in aOverflow[0],
         3260  +    ** etc. A value of 0 in the aOverflow[] array means "not yet known"
         3261  +    ** (the cache is lazily populated).
         3262  +    */
  3222   3263       if( pCur->cacheOverflow && !pCur->aOverflow ){
  3223   3264         int nOvfl = (pCur->info.nPayload-pCur->info.nLocal+ovflSize-1)/ovflSize;
  3224   3265         pCur->aOverflow = (Pgno *)sqliteMalloc(sizeof(Pgno)*nOvfl);
  3225   3266         if( nOvfl && !pCur->aOverflow ){
  3226         -        return SQLITE_NOMEM;
         3267  +        rc = SQLITE_NOMEM;
  3227   3268         }
  3228   3269       }
         3270  +
         3271  +    /* If the overflow page-list cache has been allocated and the
         3272  +    ** entry for the first required overflow page is valid, skip
         3273  +    ** directly to it.
         3274  +    */
  3229   3275       if( pCur->aOverflow && pCur->aOverflow[offset/ovflSize] ){
  3230   3276         iIdx = (offset/ovflSize);
  3231   3277         nextPage = pCur->aOverflow[iIdx];
  3232   3278         offset = (offset%ovflSize);
  3233   3279       }
  3234   3280   #endif
  3235         -    for(iIdx++; amt>0 && nextPage; iIdx++){
         3281  +
         3282  +    for( ; rc==SQLITE_OK && amt>0 && nextPage; iIdx++){
         3283  +
         3284  +#ifndef SQLITE_OMIT_INCRBLOB
         3285  +      /* If required, populate the overflow page-list cache. */
         3286  +      if( pCur->aOverflow ){
         3287  +        assert(!pCur->aOverflow[iIdx] || pCur->aOverflow[iIdx]==nextPage);
         3288  +        pCur->aOverflow[iIdx] = nextPage;
         3289  +      }
         3290  +#endif
         3291  +
  3236   3292         if( offset>=ovflSize ){
  3237   3293           /* The only reason to read this page is to obtain the page
  3238         -        ** number for the next page in the overflow chain. So try
  3239         -        ** the getOverflowPage() shortcut.
         3294  +        ** number for the next page in the overflow chain. The page
         3295  +	** data is not required. So first try to lookup the overflow
         3296  +	** page-list cache, if any, then fall back to the getOverflowPage()
         3297  +        ** function.
  3240   3298           */
  3241         -        rc = getOverflowPage(pBt, nextPage, 0, &nextPage);
  3242         -        if( rc!=SQLITE_OK ){
  3243         -          return rc;
  3244         -        }
  3245         -        offset -= ovflSize;
  3246   3299   #ifndef SQLITE_OMIT_INCRBLOB
  3247         -        if( pCur->aOverflow ){
  3248         -          assert(nextPage);
  3249         -          pCur->aOverflow[iIdx] = nextPage;
  3250         -        }
         3300  +        if( pCur->aOverflow && pCur->aOverflow[iIdx+1] ){
         3301  +          nextPage = pCur->aOverflow[iIdx+1];
         3302  +        } else 
  3251   3303   #endif
         3304  +          rc = getOverflowPage(pBt, nextPage, 0, &nextPage);
         3305  +        assert(rc==SQLITE_OK || nextPage==0);
         3306  +        offset -= ovflSize;
  3252   3307         }else{
  3253   3308           /* Need to read this page properly. It contains some of the
  3254   3309           ** range of data that is being read (eOp==0) or written (eOp!=0).
  3255   3310           */
  3256   3311           DbPage *pDbPage;
  3257   3312           int a = amt;
  3258   3313           rc = sqlite3PagerGet(pBt->pPager, nextPage, &pDbPage);
  3259         -        if( rc!=0 ){
  3260         -          return rc;
  3261         -        }
  3262         -        aPayload = sqlite3PagerGetData(pDbPage);
  3263         -        nextPage = get4byte(aPayload);
  3264         -        if( a + offset > ovflSize ){
  3265         -          a = ovflSize - offset;
  3266         -        }
  3267         -        if( eOp ){
  3268         -          /* A write operation. */
  3269         -          rc = sqlite3PagerWrite(pDbPage);
  3270         -          if( rc!=SQLITE_OK ){
  3271         -            sqlite3PagerUnref(pDbPage);
  3272         -            return rc;
         3314  +        if( rc==SQLITE_OK ){
         3315  +          aPayload = sqlite3PagerGetData(pDbPage);
         3316  +          nextPage = get4byte(aPayload);
         3317  +          if( a + offset > ovflSize ){
         3318  +            a = ovflSize - offset;
  3273   3319             }
  3274         -          memcpy(&aPayload[offset+4], pBuf, a);
  3275         -        }else{
  3276         -          /* A read operation */
  3277         -          memcpy(pBuf, &aPayload[offset+4], a);
         3320  +          rc = copyPayload(&aPayload[offset+4], pBuf, a, eOp, pDbPage);
         3321  +          sqlite3PagerUnref(pDbPage);
         3322  +          offset = 0;
         3323  +          amt -= a;
         3324  +          pBuf += a;
  3278   3325           }
  3279         -        offset = 0;
  3280         -        amt -= a;
  3281         -        pBuf += a;
  3282         -        sqlite3PagerUnref(pDbPage);
  3283         -#ifndef SQLITE_OMIT_INCRBLOB
  3284         -        if( pCur->aOverflow && nextPage ){
  3285         -          pCur->aOverflow[iIdx] = nextPage;
  3286         -        }
  3287         -#endif
  3288   3326         }
  3289   3327       }
  3290   3328     }
  3291   3329   
  3292         -  if( amt>0 ){
         3330  +  if( rc==SQLITE_OK && amt>0 ){
  3293   3331       return SQLITE_CORRUPT_BKPT;
  3294   3332     }
  3295         -  return SQLITE_OK;
         3333  +  return rc;
  3296   3334   }
  3297   3335   
  3298   3336   /*
  3299   3337   ** Read part of the key associated with cursor pCur.  Exactly
  3300   3338   ** "amt" bytes will be transfered into pBuf[].  The transfer
  3301   3339   ** begins at "offset".
  3302   3340   **
................................................................................
  6979   7017     }
  6980   7018   
  6981   7019     return accessPayload(pCsr, offset, amt, (unsigned char *)z, 0, 1);
  6982   7020   }
  6983   7021   
  6984   7022   /* 
  6985   7023   ** Set a flag on this cursor to cache the locations of pages from the 
  6986         -** overflow list for the current row.
         7024  +** overflow list for the current row. This is used by cursors opened
         7025  +** for incremental blob IO only.
         7026  +**
         7027  +** This function sets a flag only. The actual page location cache
         7028  +** (stored in BtCursor.aOverflow[]) is allocated and used by function
         7029  +** accessPayload() (the worker function for sqlite3BtreeData() and
         7030  +** sqlite3BtreePutData()).
  6987   7031   */
  6988   7032   void sqlite3BtreeCacheOverflow(BtCursor *pCur){
  6989   7033     assert(!pCur->cacheOverflow);
  6990   7034     assert(!pCur->aOverflow);
  6991   7035     pCur->cacheOverflow = 1;
  6992   7036   }
  6993   7037   #endif