Request for extension of sqlite3_vfs struct
(1) By anonymous on 2022-08-06 21:46:45 [source]
Hello,
I have attached a patch below which improves the memvfs example. The memvfs example currently only works in readonly mode (i.e. SELECT queries). If you execute for example an INSERT statement on this database, you'll get an "unable to open database file" error.
The error is caused because it tries to access the journaling filesystem (which doesn't exist). You can get around this issue by extending the sqlite3_vfs struct with a btree_flags variable and using this variable in a sqlite3BtreeOpen() call.
In the function openDatabase you have the following call:
/* Open the backend database driver */
rc = sqlite3BtreeOpen(db->pVfs, zOpen, db, &db->aDb[0].pBt, 0,
flags | SQLITE_OPEN_MAIN_DB);
The 0 at the end of the 1st line represents the flags parameter, which is forwarded to the sqlite3PagerOpen function.
In this function you have the following code:
int useJournal = (flags & PAGER_OMIT_JOURNAL)==0; /* False to omit journal */
Because flags is zero, useJournal will always be true in this case.
When I updated the code above with the following code:
/* Open the backend database driver */
rc = sqlite3BtreeOpen(db->pVfs, zOpen, db, &db->aDb[0].pBt, db->pVfs->btree_flags,
flags | SQLITE_OPEN_MAIN_DB);
and I use the BTREE_OMIT_JOURNAL flag in my mem_vfs struct I was able to execute DML queries on the memvfs-database:
static sqlite3_vfs mem_vfs = {
2, /* iVersion */
0, /* szOsFile (set when registered) */
1024, /* mxPathname */
0, /* pNext */
"memvfs", /* zName */
0, /* pAppData (set when registered) */
BTREE_OMIT_JOURNAL, /* btree_flags */
memOpen, /* xOpen */
memDelete, /* xDelete */
memAccess, /* xAccess */
memFullPathname, /* xFullPathname */
memDlOpen, /* xDlOpen */
memDlError, /* xDlError */
memDlSym, /* xDlSym */
memDlClose, /* xDlClose */
memRandomness, /* xRandomness */
memSleep, /* xSleep */
memCurrentTime, /* xCurrentTime */
memGetLastError, /* xGetLastError */
memCurrentTimeInt64, /* xCurrentTimeInt64 */
};
Below you can find the complete patch, the modified memvfs and my main example program. The modified memvfs and my main program example are Windows-only and work with shared memory. This way a SQLite database in memory can be accessed by multiple processes (on the same computer).
This is the complete patch:
Index: ext/async/sqlite3async.c
==================================================================
--- ext/async/sqlite3async.c
+++ ext/async/sqlite3async.c
@@ -1303,10 +1303,11 @@
sizeof(AsyncFile), /* szOsFile */
0, /* mxPathname */
0, /* pNext */
SQLITEASYNC_VFSNAME, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
asyncOpen, /* xOpen */
asyncDelete, /* xDelete */
asyncAccess, /* xAccess */
asyncFullPathname, /* xFullPathname */
asyncDlOpen, /* xDlOpen */
Index: ext/misc/appendvfs.c
==================================================================
--- ext/misc/appendvfs.c
+++ ext/misc/appendvfs.c
@@ -179,10 +179,11 @@
0, /* szOsFile (set when registered) */
1024, /* mxPathname */
0, /* pNext */
"apndvfs", /* zName */
0, /* pAppData (set when registered) */
+ 0, /* btree_flags */
apndOpen, /* xOpen */
apndDelete, /* xDelete */
apndAccess, /* xAccess */
apndFullPathname, /* xFullPathname */
apndDlOpen, /* xDlOpen */
Index: ext/misc/cksumvfs.c
==================================================================
--- ext/misc/cksumvfs.c
+++ ext/misc/cksumvfs.c
@@ -249,10 +249,11 @@
0, /* szOsFile (set when registered) */
1024, /* mxPathname */
0, /* pNext */
"cksmvfs", /* zName */
0, /* pAppData (set when registered) */
+ 0, /* btree_flags */
cksmOpen, /* xOpen */
cksmDelete, /* xDelete */
cksmAccess, /* xAccess */
cksmFullPathname, /* xFullPathname */
cksmDlOpen, /* xDlOpen */
Index: ext/misc/memvfs.c
==================================================================
--- ext/misc/memvfs.c
+++ ext/misc/memvfs.c
@@ -108,10 +108,11 @@
0, /* szOsFile (set when registered) */
1024, /* mxPathname */
0, /* pNext */
"memvfs", /* zName */
0, /* pAppData (set when registered) */
+ 0, /* btree_flags */
memOpen, /* xOpen */
memDelete, /* xDelete */
memAccess, /* xAccess */
memFullPathname, /* xFullPathname */
memDlOpen, /* xDlOpen */
Index: ext/misc/vfslog.c
==================================================================
--- ext/misc/vfslog.c
+++ ext/misc/vfslog.c
@@ -131,10 +131,11 @@
0, /* szOsFile (set by register_vlog()) */
1024, /* mxPathname */
0, /* pNext */
"vfslog", /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
vlogOpen, /* xOpen */
vlogDelete, /* xDelete */
vlogAccess, /* xAccess */
vlogFullPathname, /* xFullPathname */
vlogDlOpen, /* xDlOpen */
Index: ext/misc/vfsstat.c
==================================================================
--- ext/misc/vfsstat.c
+++ ext/misc/vfsstat.c
@@ -187,10 +187,11 @@
0, /* szOsFile (set by register_vstat()) */
1024, /* mxPathname */
0, /* pNext */
"vfslog", /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
vstatOpen, /* xOpen */
vstatDelete, /* xDelete */
vstatAccess, /* xAccess */
vstatFullPathname, /* xFullPathname */
vstatDlOpen, /* xDlOpen */
Index: ext/rbu/sqlite3rbu.c
==================================================================
--- ext/rbu/sqlite3rbu.c
+++ ext/rbu/sqlite3rbu.c
@@ -5259,10 +5259,11 @@
0, /* szOsFile */
0, /* mxPathname */
0, /* pNext */
0, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
rbuVfsOpen, /* xOpen */
rbuVfsDelete, /* xDelete */
rbuVfsAccess, /* xAccess */
rbuVfsFullPathname, /* xFullPathname */
Index: src/main.c
==================================================================
--- src/main.c
+++ src/main.c
@@ -3379,11 +3379,11 @@
sqlite3_free(zErrMsg);
goto opendb_out;
}
/* Open the backend database driver */
- rc = sqlite3BtreeOpen(db->pVfs, zOpen, db, &db->aDb[0].pBt, 0,
+ rc = sqlite3BtreeOpen(db->pVfs, zOpen, db, &db->aDb[0].pBt, db->pVfs->btree_flags,
flags | SQLITE_OPEN_MAIN_DB);
if( rc!=SQLITE_OK ){
if( rc==SQLITE_IOERR_NOMEM ){
rc = SQLITE_NOMEM_BKPT;
}
Index: src/memdb.c
==================================================================
--- src/memdb.c
+++ src/memdb.c
@@ -138,10 +138,11 @@
0, /* szOsFile (set when registered) */
1024, /* mxPathname */
0, /* pNext */
"memdb", /* zName */
0, /* pAppData (set when registered) */
+ 0, /* btree_flags */
memdbOpen, /* xOpen */
0, /* memdbDelete, */ /* xDelete */
memdbAccess, /* xAccess */
memdbFullPathname, /* xFullPathname */
memdbDlOpen, /* xDlOpen */
Index: src/os_unix.c
==================================================================
--- src/os_unix.c
+++ src/os_unix.c
@@ -7988,10 +7988,11 @@
sizeof(unixFile), /* szOsFile */ \
MAX_PATHNAME, /* mxPathname */ \
0, /* pNext */ \
VFSNAME, /* zName */ \
(void*)&FINDER, /* pAppData */ \
+ 0, /* btree_flags */ \
unixOpen, /* xOpen */ \
unixDelete, /* xDelete */ \
unixAccess, /* xAccess */ \
unixFullPathname, /* xFullPathname */ \
unixDlOpen, /* xDlOpen */ \
Index: src/os_win.c
==================================================================
--- src/os_win.c
+++ src/os_win.c
@@ -6011,10 +6011,11 @@
sizeof(winFile), /* szOsFile */
SQLITE_WIN32_MAX_PATH_BYTES, /* mxPathname */
0, /* pNext */
"win32", /* zName */
&winAppData, /* pAppData */
+ 0, /* btree_flags */
winOpen, /* xOpen */
winDelete, /* xDelete */
winAccess, /* xAccess */
winFullPathname, /* xFullPathname */
winDlOpen, /* xDlOpen */
@@ -6036,10 +6037,11 @@
sizeof(winFile), /* szOsFile */
SQLITE_WINNT_MAX_PATH_BYTES, /* mxPathname */
0, /* pNext */
"win32-longpath", /* zName */
&winAppData, /* pAppData */
+ 0, /* btree_flags */
winOpen, /* xOpen */
winDelete, /* xDelete */
winAccess, /* xAccess */
winFullPathname, /* xFullPathname */
winDlOpen, /* xDlOpen */
@@ -6061,10 +6063,11 @@
sizeof(winFile), /* szOsFile */
SQLITE_WIN32_MAX_PATH_BYTES, /* mxPathname */
0, /* pNext */
"win32-none", /* zName */
&winNolockAppData, /* pAppData */
+ 0, /* btree_flags */
winOpen, /* xOpen */
winDelete, /* xDelete */
winAccess, /* xAccess */
winFullPathname, /* xFullPathname */
winDlOpen, /* xDlOpen */
@@ -6086,10 +6089,11 @@
sizeof(winFile), /* szOsFile */
SQLITE_WINNT_MAX_PATH_BYTES, /* mxPathname */
0, /* pNext */
"win32-longpath-none", /* zName */
&winNolockAppData, /* pAppData */
+ 0, /* btree_flags */
winOpen, /* xOpen */
winDelete, /* xDelete */
winAccess, /* xAccess */
winFullPathname, /* xFullPathname */
winDlOpen, /* xDlOpen */
Index: src/sqlite.h.in
==================================================================
--- src/sqlite.h.in
+++ src/sqlite.h.in
@@ -1429,10 +1429,11 @@
int szOsFile; /* Size of subclassed sqlite3_file */
int mxPathname; /* Maximum file pathname length */
sqlite3_vfs *pNext; /* Next registered VFS */
const char *zName; /* Name of this virtual file system */
void *pAppData; /* Pointer to application-specific data */
+ int btree_flags; /* Flags used in sqlite3BtreeOpen() */
int (*xOpen)(sqlite3_vfs*, const char *zName, sqlite3_file*,
int flags, int *pOutFlags);
int (*xDelete)(sqlite3_vfs*, const char *zName, int syncDir);
int (*xAccess)(sqlite3_vfs*, const char *zName, int flags, int *pResOut);
int (*xFullPathname)(sqlite3_vfs*, const char *zName, int nOut, char *zOut);
Index: src/test6.c
==================================================================
--- src/test6.c
+++ src/test6.c
@@ -851,10 +851,11 @@
0, /* szOsFile */
0, /* mxPathname */
0, /* pNext */
"crash", /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
cfOpen, /* xOpen */
cfDelete, /* xDelete */
cfAccess, /* xAccess */
cfFullPathname, /* xFullPathname */
Index: src/test_demovfs.c
==================================================================
--- src/test_demovfs.c
+++ src/test_demovfs.c
@@ -622,10 +622,11 @@
sizeof(DemoFile), /* szOsFile */
MAXPATHNAME, /* mxPathname */
0, /* pNext */
"demo", /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
demoOpen, /* xOpen */
demoDelete, /* xDelete */
demoAccess, /* xAccess */
demoFullPathname, /* xFullPathname */
demoDlOpen, /* xDlOpen */
Index: src/test_devsym.c
==================================================================
--- src/test_devsym.c
+++ src/test_devsym.c
@@ -423,10 +423,11 @@
sizeof(devsym_file), /* szOsFile */
DEVSYM_MAX_PATHNAME, /* mxPathname */
0, /* pNext */
DEVSYM_VFS_NAME, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
devsymOpen, /* xOpen */
devsymDelete, /* xDelete */
devsymAccess, /* xAccess */
devsymFullPathname, /* xFullPathname */
#ifndef SQLITE_OMIT_LOAD_EXTENSION
@@ -452,10 +453,11 @@
sizeof(devsym_file), /* szOsFile */
DEVSYM_MAX_PATHNAME, /* mxPathname */
0, /* pNext */
WRITECRASH_NAME, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
writecrashOpen, /* xOpen */
devsymDelete, /* xDelete */
devsymAccess, /* xAccess */
devsymFullPathname, /* xFullPathname */
#ifndef SQLITE_OMIT_LOAD_EXTENSION
Index: src/test_journal.c
==================================================================
--- src/test_journal.c
+++ src/test_journal.c
@@ -167,10 +167,11 @@
sizeof(jt_file), /* szOsFile */
JT_MAX_PATHNAME, /* mxPathname */
0, /* pNext */
JT_VFS_NAME, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
jtOpen, /* xOpen */
jtDelete, /* xDelete */
jtAccess, /* xAccess */
jtFullPathname, /* xFullPathname */
jtDlOpen, /* xDlOpen */
Index: src/test_onefile.c
==================================================================
--- src/test_onefile.c
+++ src/test_onefile.c
@@ -186,10 +186,11 @@
0, /* szOsFile */
0, /* mxPathname */
0, /* pNext */
FS_VFS_NAME, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
fsOpen, /* xOpen */
fsDelete, /* xDelete */
fsAccess, /* xAccess */
fsFullPathname, /* xFullPathname */
fsDlOpen, /* xDlOpen */
Index: src/test_osinst.c
==================================================================
--- src/test_osinst.c
+++ src/test_osinst.c
@@ -182,10 +182,11 @@
sizeof(VfslogFile), /* szOsFile */
INST_MAX_PATHNAME, /* mxPathname */
0, /* pNext */
0, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
vfslogOpen, /* xOpen */
vfslogDelete, /* xDelete */
vfslogAccess, /* xAccess */
vfslogFullPathname, /* xFullPathname */
vfslogDlOpen, /* xDlOpen */
Index: src/test_vfs.c
==================================================================
--- src/test_vfs.c
+++ src/test_vfs.c
@@ -1446,10 +1446,11 @@
0, /* szOsFile */
0, /* mxPathname */
0, /* pNext */
0, /* zName */
0, /* pAppData */
+ 0, /* btree_flags */
tvfsOpen, /* xOpen */
tvfsDelete, /* xDelete */
tvfsAccess, /* xAccess */
tvfsFullPathname, /* xFullPathname */
#ifndef SQLITE_OMIT_LOAD_EXTENSION
This is my modified memvfs example.
/*
** 2016-09-07
**
** The author disclaims copyright to this source code. In place of
** a legal notice, here is a blessing:
**
** May you do good and not evil.
** May you find forgiveness for yourself and forgive others.
** May you share freely, never taking more than you give.
**
******************************************************************************
**
** This is an in-memory VFS implementation. The application supplies
** a chunk of memory to hold the database file.
**
** Because there is place to store a rollback or wal journal, the database
** must use one of journal_mode=MEMORY or journal_mode=NONE.
**
** USAGE:
**
** sqlite3_open_v2("file:/whatever?ptr=0xf05538&sz=14336&max=65536", &db,
** SQLITE_OPEN_READWRITE | SQLITE_OPEN_URI,
** "memvfs");
**
** These are the query parameters:
**
** ptr= The address of the memory buffer that holds the database.
**
** sz= The current size the database file
**
** maxsz= The maximum size of the database. In other words, the
** amount of space allocated for the ptr= buffer.
**
** The ptr= and sz= query parameters are required. If maxsz= is omitted,
** then it defaults to the sz= value. Parameter values can be in either
** decimal or hexadecimal. The filename in the URI is ignored.
*/
#define BTREE_OMIT_JOURNAL 1
#include "sqlite3ext.h"
SQLITE_EXTENSION_INIT1
#include <string.h>
#include <assert.h>
#include <windows.h>
HANDLE hMutex = NULL;
/*
** Forward declaration of objects used by this utility
*/
typedef struct sqlite3_vfs MemVfs;
typedef struct MemoryFile MemoryFile;
/* Access to a lower-level VFS that (might) implement dynamic loading,
** access to randomness, etc.
*/
#define ORIGVFS(p) ((sqlite3_vfs*)((p)->pAppData))
/* An open file */
struct MemoryFile {
sqlite3_file base; /* IO methods */
sqlite3_int64 sz; /* Size of the file */
sqlite3_int64 szMax; /* Space allocated to aData */
unsigned char* aData; /* content of the file */
HANDLE mutex;
};
/*
** Methods for MemoryFile
*/
static int memClose(sqlite3_file*);
static int memRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
static int memWrite(sqlite3_file*, const void*, int iAmt, sqlite3_int64 iOfst);
static int memTruncate(sqlite3_file*, sqlite3_int64 size);
static int memSync(sqlite3_file*, int flags);
static int memFileSize(sqlite3_file*, sqlite3_int64* pSize);
static int memLock(sqlite3_file*, int);
static int memUnlock(sqlite3_file*, int);
static int memCheckReservedLock(sqlite3_file*, int* pResOut);
static int memFileControl(sqlite3_file*, int op, void* pArg);
static int memSectorSize(sqlite3_file*);
static int memDeviceCharacteristics(sqlite3_file*);
static int memShmMap(sqlite3_file*, int iPg, int pgsz, int, void volatile**);
static int memShmLock(sqlite3_file*, int offset, int n, int flags);
static void memShmBarrier(sqlite3_file*);
static int memShmUnmap(sqlite3_file*, int deleteFlag);
static int memFetch(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void** pp);
static int memUnfetch(sqlite3_file*, sqlite3_int64 iOfst, void* p);
/*
** Methods for MemVfs
*/
static int memOpen(sqlite3_vfs*, const char*, sqlite3_file*, int, int*);
static int memDelete(sqlite3_vfs*, const char* zName, int syncDir);
static int memAccess(sqlite3_vfs*, const char* zName, int flags, int*);
static int memFullPathname(sqlite3_vfs*, const char* zName, int, char* zOut);
static void* memDlOpen(sqlite3_vfs*, const char* zFilename);
static void memDlError(sqlite3_vfs*, int nByte, char* zErrMsg);
static void (*memDlSym(sqlite3_vfs* pVfs, void* p, const char* zSym))(void);
static void memDlClose(sqlite3_vfs*, void*);
static int memRandomness(sqlite3_vfs*, int nByte, char* zOut);
static int memSleep(sqlite3_vfs*, int microseconds);
static int memCurrentTime(sqlite3_vfs*, double*);
static int memGetLastError(sqlite3_vfs*, int, char*);
static int memCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*);
static sqlite3_vfs mem_vfs = {
2, /* iVersion */
0, /* szOsFile (set when registered) */
1024, /* mxPathname */
0, /* pNext */
"memvfs", /* zName */
0, /* pAppData (set when registered) */
BTREE_OMIT_JOURNAL, /* btree_flags */
memOpen, /* xOpen */
memDelete, /* xDelete */
memAccess, /* xAccess */
memFullPathname, /* xFullPathname */
memDlOpen, /* xDlOpen */
memDlError, /* xDlError */
memDlSym, /* xDlSym */
memDlClose, /* xDlClose */
memRandomness, /* xRandomness */
memSleep, /* xSleep */
memCurrentTime, /* xCurrentTime */
memGetLastError, /* xGetLastError */
memCurrentTimeInt64, /* xCurrentTimeInt64 */
};
static const sqlite3_io_methods mem_io_methods = {
3, /* iVersion */
memClose, /* xClose */
memRead, /* xRead */
memWrite, /* xWrite */
memTruncate, /* xTruncate */
memSync, /* xSync */
memFileSize, /* xFileSize */
memLock, /* xLock */
memUnlock, /* xUnlock */
memCheckReservedLock, /* xCheckReservedLock */
memFileControl, /* xFileControl */
memSectorSize, /* xSectorSize */
memDeviceCharacteristics, /* xDeviceCharacteristics */
memShmMap, /* xShmMap */
memShmLock, /* xShmLock */
memShmBarrier, /* xShmBarrier */
memShmUnmap, /* xShmUnmap */
memFetch, /* xFetch */
memUnfetch /* xUnfetch */
};
/*
** Close an mem-file.
**
** The pData pointer is owned by the application, so there is nothing
** to free.
*/
static int memClose(sqlite3_file* pFile) {
if (hMutex != NULL && hMutex != INVALID_HANDLE_VALUE)
{
CloseHandle(hMutex);
}
return SQLITE_OK;
}
/*
** Read data from an mem-file.
*/
static int memRead(
sqlite3_file* pFile,
void* zBuf,
int iAmt,
sqlite_int64 iOfst
) {
if (WaitForSingleObject(hMutex, INFINITE) == WAIT_OBJECT_0)
{
MemoryFile* p = (MemoryFile*)pFile;
memcpy(zBuf, p->aData + iOfst, iAmt);
ReleaseMutex(hMutex);
return SQLITE_OK;
}
return SQLITE_ERROR;
}
/*
** Write data to an mem-file.
*/
static int memWrite(
sqlite3_file* pFile,
const void* z,
int iAmt,
sqlite_int64 iOfst
) {
if (WaitForSingleObject(hMutex, INFINITE) == WAIT_OBJECT_0)
{
MemoryFile* p = (MemoryFile*)pFile;
if (iOfst + iAmt > p->sz) {
if (iOfst + iAmt > p->szMax) return SQLITE_FULL;
if (iOfst > p->sz) memset(p->aData + p->sz, 0, iOfst - p->sz);
p->sz = iOfst + iAmt;
}
memcpy(p->aData + iOfst, z, iAmt);
ReleaseMutex(hMutex);
return SQLITE_OK;
}
return SQLITE_ERROR;
}
/*
** Truncate an mem-file.
*/
static int memTruncate(sqlite3_file* pFile, sqlite_int64 size) {
if (WaitForSingleObject(hMutex, INFINITE) == WAIT_OBJECT_0)
{
MemoryFile* p = (MemoryFile*)pFile;
if (size > p->sz) {
if (size > p->szMax) return SQLITE_FULL;
memset(p->aData + p->sz, 0, size - p->sz);
}
p->sz = size;
ReleaseMutex(hMutex);
return SQLITE_OK;
}
return SQLITE_ERROR;
}
/*
** Sync an mem-file.
*/
static int memSync(sqlite3_file* pFile, int flags) {
return SQLITE_OK;
}
/*
** Return the current file-size of an mem-file.
*/
static int memFileSize(sqlite3_file* pFile, sqlite_int64* pSize) {
MemoryFile* p = (MemoryFile*)pFile;
*pSize = p->sz;
return SQLITE_OK;
}
/*
** Lock an mem-file.
*/
static int memLock(sqlite3_file* pFile, int eLock) {
return SQLITE_OK;
}
/*
** Unlock an mem-file.
*/
static int memUnlock(sqlite3_file* pFile, int eLock) {
return SQLITE_OK;
}
/*
** Check if another file-handle holds a RESERVED lock on an mem-file.
*/
static int memCheckReservedLock(sqlite3_file* pFile, int* pResOut) {
*pResOut = 0;
return SQLITE_OK;
}
/*
** File control method. For custom operations on an mem-file.
*/
static int memFileControl(sqlite3_file* pFile, int op, void* pArg) {
if (WaitForSingleObject(hMutex, INFINITE) == WAIT_OBJECT_0)
{
MemoryFile* p = (MemoryFile*)pFile;
int rc = SQLITE_NOTFOUND;
if (op == SQLITE_FCNTL_VFSNAME) {
*(char**)pArg = sqlite3_mprintf("mem(%p,%lld)", p->aData, p->sz);
rc = SQLITE_OK;
}
ReleaseMutex(hMutex);
return rc;
}
return SQLITE_ERROR;
}
/*
** Return the sector-size in bytes for an mem-file.
*/
static int memSectorSize(sqlite3_file* pFile) {
return 1024;
}
/*
** Return the device characteristic flags supported by an mem-file.
*/
static int memDeviceCharacteristics(sqlite3_file* pFile) {
return SQLITE_IOCAP_ATOMIC |
SQLITE_IOCAP_POWERSAFE_OVERWRITE |
SQLITE_IOCAP_SAFE_APPEND |
SQLITE_IOCAP_SEQUENTIAL;
}
/* Create a shared memory file mapping */
static int memShmMap(
sqlite3_file* pFile,
int iPg,
int pgsz,
int bExtend,
void volatile** pp
) {
return SQLITE_IOERR_SHMMAP;
}
/* Perform locking on a shared-memory segment */
static int memShmLock(sqlite3_file* pFile, int offset, int n, int flags) {
return SQLITE_IOERR_SHMLOCK;
}
/* Memory barrier operation on shared memory */
static void memShmBarrier(sqlite3_file* pFile) {
return;
}
/* Unmap a shared memory segment */
static int memShmUnmap(sqlite3_file* pFile, int deleteFlag) {
return SQLITE_OK;
}
/* Fetch a page of a memory-mapped file */
static int memFetch(
sqlite3_file* pFile,
sqlite3_int64 iOfst,
int iAmt,
void** pp
) {
MemoryFile* p = (MemoryFile*)pFile;
*pp = (void*)(p->aData + iOfst);
return SQLITE_OK;
}
/* Release a memory-mapped page */
static int memUnfetch(sqlite3_file* pFile, sqlite3_int64 iOfst, void* pPage) {
return SQLITE_OK;
}
/*
** Open an mem file handle.
*/
static int memOpen(
sqlite3_vfs* pVfs,
const char* zName,
sqlite3_file* pFile,
int flags,
int* pOutFlags
) {
MemoryFile* p = (MemoryFile*)pFile;
memset(p, 0, sizeof(*p));
if ((flags & SQLITE_OPEN_MAIN_DB) == 0)
return SQLITE_CANTOPEN;
p->aData = (unsigned char*)sqlite3_uri_int64(zName, "ptr", 0);
if (p->aData == 0) return SQLITE_CANTOPEN;
p->sz = sqlite3_uri_int64(zName, "sz", 0);
if (p->sz < 0) return SQLITE_CANTOPEN;
p->szMax = sqlite3_uri_int64(zName, "max", p->sz);
if (p->szMax < p->sz) return SQLITE_CANTOPEN;
pFile->pMethods = &mem_io_methods;
hMutex = CreateMutex(NULL, FALSE, L"MTX_SQLITESHM");
return SQLITE_OK;
}
/*
** Delete the file located at zPath. If the dirSync argument is true,
** ensure the file-system modifications are synced to disk before
** returning.
*/
static int memDelete(sqlite3_vfs* pVfs, const char* zPath, int dirSync) {
return SQLITE_IOERR_DELETE;
}
/*
** Test for access permissions. Return true if the requested permission
** is available, or false otherwise.
*/
static int memAccess(
sqlite3_vfs* pVfs,
const char* zPath,
int flags,
int* pResOut
) {
*pResOut = 0;
return SQLITE_OK;
}
/*
** Populate buffer zOut with the full canonical pathname corresponding
** to the pathname in zPath. zOut is guaranteed to point to a buffer
** of at least (INST_MAX_PATHNAME+1) bytes.
*/
static int memFullPathname(
sqlite3_vfs* pVfs,
const char* zPath,
int nOut,
char* zOut
) {
sqlite3_snprintf(nOut, zOut, "%s", zPath);
return SQLITE_OK;
}
/*
** Open the dynamic library located at zPath and return a handle.
*/
static void* memDlOpen(sqlite3_vfs* pVfs, const char* zPath) {
return ORIGVFS(pVfs)->xDlOpen(ORIGVFS(pVfs), zPath);
}
/*
** Populate the buffer zErrMsg (size nByte bytes) with a human readable
** utf-8 string describing the most recent error encountered associated
** with dynamic libraries.
*/
static void memDlError(sqlite3_vfs* pVfs, int nByte, char* zErrMsg) {
ORIGVFS(pVfs)->xDlError(ORIGVFS(pVfs), nByte, zErrMsg);
}
/*
** Return a pointer to the symbol zSymbol in the dynamic library pHandle.
*/
static void (*memDlSym(sqlite3_vfs* pVfs, void* p, const char* zSym))(void) {
return ORIGVFS(pVfs)->xDlSym(ORIGVFS(pVfs), p, zSym);
}
/*
** Close the dynamic library handle pHandle.
*/
static void memDlClose(sqlite3_vfs* pVfs, void* pHandle) {
ORIGVFS(pVfs)->xDlClose(ORIGVFS(pVfs), pHandle);
}
/*
** Populate the buffer pointed to by zBufOut with nByte bytes of
** random data.
*/
static int memRandomness(sqlite3_vfs* pVfs, int nByte, char* zBufOut) {
return ORIGVFS(pVfs)->xRandomness(ORIGVFS(pVfs), nByte, zBufOut);
}
/*
** Sleep for nMicro microseconds. Return the number of microseconds
** actually slept.
*/
static int memSleep(sqlite3_vfs* pVfs, int nMicro) {
return ORIGVFS(pVfs)->xSleep(ORIGVFS(pVfs), nMicro);
}
/*
** Return the current time as a Julian Day number in *pTimeOut.
*/
static int memCurrentTime(sqlite3_vfs* pVfs, double* pTimeOut) {
return ORIGVFS(pVfs)->xCurrentTime(ORIGVFS(pVfs), pTimeOut);
}
static int memGetLastError(sqlite3_vfs* pVfs, int a, char* b) {
return ORIGVFS(pVfs)->xGetLastError(ORIGVFS(pVfs), a, b);
}
static int memCurrentTimeInt64(sqlite3_vfs* pVfs, sqlite3_int64* p) {
return ORIGVFS(pVfs)->xCurrentTimeInt64(ORIGVFS(pVfs), p);
}
#ifdef _WIN32
__declspec(dllexport)
#endif
/*
** This routine is called when the extension is loaded.
** Register the new VFS.
*/
int sqlite3_memvfs_init(
sqlite3* db,
char** pzErrMsg,
const sqlite3_api_routines* pApi
) {
int rc = SQLITE_OK;
SQLITE_EXTENSION_INIT2(pApi);
mem_vfs.pAppData = sqlite3_vfs_find(0);
if (mem_vfs.pAppData == 0) return SQLITE_ERROR;
mem_vfs.szOsFile = sizeof(MemoryFile);
rc = sqlite3_vfs_register(&mem_vfs, 1);
if (rc == SQLITE_OK) rc = SQLITE_OK_LOAD_PERMANENTLY;
return rc;
}
This is my (Windows-only) example program:
#include <stdio.h>
#include <stdint.h>
#include <assert.h>
#include <windows.h>
#include "sqlite3.h"
#define DATABASE_SIZE_IN_KB 65536
#define FILEMAPPING_NAME L"SQLITE_SHM"
int main(int argc, char* argv[])
{
sqlite3* db = 0;
char* err;
int rc;
char* zErrMsg = 0;
void* dbMemory = NULL;
// initialize shared memory
BOOL initializeMemoryBlock = FALSE;
HANDLE hFileMapping = OpenFileMapping(FILE_MAP_ALL_ACCESS, FALSE, FILEMAPPING_NAME);
if (hFileMapping == 0 || hFileMapping == INVALID_HANDLE_VALUE)
{
hFileMapping = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE, 0, DATABASE_SIZE_IN_KB * 1024, FILEMAPPING_NAME);
if (hFileMapping == 0 || hFileMapping == INVALID_HANDLE_VALUE)
{
printf("CreateFileMapping error -> Windows error %d\n", GetLastError());
return;
}
initializeMemoryBlock = TRUE;
}
dbMemory = MapViewOfFileEx(hFileMapping, FILE_MAP_READ | FILE_MAP_WRITE, 0, 0, 0, NULL);
if (dbMemory == NULL)
{
printf("MapViewOfFile error -> Windows error %d\n", GetLastError());
return;
}
// Open an in-memory database to use as a handle for loading the memvfs extension
if (sqlite3_open(":memory:", &db) != SQLITE_OK) {
printf("open :memory: %s\n", sqlite3_errmsg(db));
return EXIT_FAILURE;
}
sqlite3_enable_load_extension(db, 1);
if (sqlite3_load_extension(db, argv[0], "sqlite3_memvfs_init", &err) != SQLITE_OK) {
printf("load extension: %s\n", err);
return EXIT_FAILURE;
}
// Done with this database
sqlite3_close(db);
if (initializeMemoryBlock)
{
memset(dbMemory, 0, DATABASE_SIZE_IN_KB * 1024);
// And open that memory with memvfs now that it holds a valid database
char* memuri = sqlite3_mprintf("file:whatever?ptr=0x%p&sz=%lld",
dbMemory, (long long)DATABASE_SIZE_IN_KB * 1024);
printf("Trying to open '%s'\n", memuri);
if (sqlite3_open_v2(memuri, &db, SQLITE_OPEN_READWRITE | SQLITE_OPEN_URI,
"memvfs") != SQLITE_OK) {
printf("open memvfs: %s\n", sqlite3_errmsg(db));
return EXIT_FAILURE;
}
sqlite3_free(memuri);
// initialize SQLite database
rc = sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 1, 0);
rc = sqlite3_exec(db, "VACUUM", 0, 0, &zErrMsg);
rc = sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 0, 0);
// create tables
rc = sqlite3_exec(db, "CREATE TABLE dept(deptno CONSTRAINT PK_DEPT PRIMARY KEY, dname TEXT, loc TEXT);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "CREATE TABLE emp(empno INT CONSTRAINT PK_EMP PRIMARY KEY, ename TEXT, job TEXT, mgr INT, hiredate DATETIME, sal REAL, comm REAL, deptno INT, FOREIGN KEY(deptno) REFERENCES dept(deptno));", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO dept VALUES (10, 'ACCOUNTING', 'NEW YORK');", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO dept VALUES(20, 'RESEARCH', 'DALLAS');", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO dept VALUES(30, 'SALES', 'CHICAGO');", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO dept VALUES(40, 'OPERATIONS', 'BOSTON');", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7369, 'SMITH', 'CLERK', 7902, '17-12-1980', 800, NULL, 20);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7499, 'ALLEN', 'SALESMAN', 7698, '20-2-1981', 1600, 300, 30);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7521, 'WARD', 'SALESMAN', 7698, '22-2-1981', 1250, 500, 30);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7566, 'JONES', 'MANAGER', 7839, '2-4-1981', 2975, NULL, 20);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7654, 'MARTIN', 'SALESMAN', 7698, '28-9-1981', 1250, 1400, 30);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7698, 'BLAKE', 'MANAGER', 7839, '1-5-1981', 2850, NULL, 30);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7782, 'CLARK', 'MANAGER', 7839, '9-6-1981', 2450, NULL, 10);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7788, 'SCOTT', 'ANALYST', 7566, '13-07-1987', 3000, NULL, 20);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7839, 'KING', 'PRESIDENT', NULL, '17-11-1981', 5000, NULL, 10);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7844, 'TURNER', 'SALESMAN', 7698, '8-9-1981', 1500, 0, 30);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7876, 'ADAMS', 'CLERK', 7788, '13-07-1987', 1100, NULL, 20);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7900, 'JAMES', 'CLERK', 7698, '3-12-1981', 950, NULL, 30);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7902, 'FORD', 'ANALYST', 7566, '3-12-1981', 3000, NULL, 20);", 0, 0, &zErrMsg);
rc = sqlite3_exec(db, "INSERT INTO emp VALUES(7934, 'MILLER', 'CLERK', 7782, '23-1-1982', 1300, NULL, 10);", 0, 0, &zErrMsg);
}
char input[4096];
while (gets(input) != 0)
{
if (input[0] == 0) break;
if (_strnicmp(input, "INSERT", sizeof("INSERT") - 1) == 0
|| _strnicmp(input, "UPDATE", sizeof("UPDATE") - 1) == 0
|| _strnicmp(input, "DELETE", sizeof("DELETE") - 1) == 0
)
{
rc = sqlite3_exec(db, input, 0, 0, &zErrMsg);
if (rc != SQLITE_OK) {
printf(">> sqlite3_exec: %s\n", zErrMsg);
continue;
}
}
if (_strnicmp(input, "SELECT", sizeof("SELECT") - 1) == 0)
{
// Try querying the database to show it works.
sqlite3_stmt* stmt;
if (sqlite3_prepare_v2(db, input, -1, &stmt, NULL) !=
SQLITE_OK) {
printf(">> sqlite3_prepare_v2: %s\n", sqlite3_errmsg(db));
continue;
}
int nbrColumns = sqlite3_column_count(stmt);
for (int rc = sqlite3_step(stmt); rc == SQLITE_ROW; rc = sqlite3_step(stmt)) {
printf(">> ");
for (int c = 0; c < nbrColumns; c++)
{
switch (sqlite3_column_type(stmt, c))
{
case SQLITE_INTEGER:
printf("%d\t", sqlite3_column_int(stmt, c));
break;
case SQLITE_FLOAT:
printf("%.2f\t", sqlite3_column_double(stmt, c));
break;
case SQLITE_TEXT:
printf("%s \t", sqlite3_column_text(stmt, c));
break;
case SQLITE_BLOB:
printf("%X\t", sqlite3_column_blob(stmt, c));
break;
case SQLITE_NULL:
printf("(null)\t");
break;
default:
printf("(invalid)\t");
break;
}
}
printf("\n");
}
sqlite3_finalize(stmt);
}
}
// close database
sqlite3_close(db);
// terminate shared memory
UnmapViewOfFile(dbMemory);
CloseHandle(hFileMapping);
}
(2) By anonymous on 2022-08-07 00:30:01 in reply to 1 [link] [source]
New fields should not be added into the middle of existing public structures; they should be added to the end. However, there are other problems with doing such things (such as affecting official public structures at all) in unofficial patches too; see the comments about my own patch for details.
Anyways, you can use the pragmas to disable the journal if desired, and I should not expect that you should need the feature you describe. It may be possible for a VFS to obtain a pointer to the database connection by the SQLITE_FCNTL_PDB file control, if needed (although note that I have not tried this yet, and the documentation does not describe what it does).
I had started to make up a list of unofficial patches of SQLite, and a repository that they may be included in.
(An alternative would be to make a separate project which is a fork of SQLite but a different name, which isn't designed to be compatible with SQLite, and then many more possibilities can be done than the patches are (including avoiding breaking by combining patches), and might be more likely to be stable (if tests are made), too. This is more complicated than patches are, though, but has different advantages and disadvantages.)