sqlite over NFS slowness with RedHat 7
(1) By brasolin on 2020-04-30 16:37:31 [link] [source]
our NetApp filer host an SQLite DB, exported via NFS to our Linux clients. It is not really a big database: ~300 MB. The clients parse (read) the DB with a python script. There's a dramatically difference in the execution time between the RHEL6 (faster, ~ 1 min) and the RHEL7 (slower, 15/20 mins).
RHEL 6 kernel: 2.6.32-754.24.3.el6.x86_64 sqlite-126.96.36.199-0.3.el6.x86_64
RHEL 7 kernel: 3.10.0-1062.4.3.el7.x86_64 sqlite-3.7.17-8.el7.x86_64
both clients mount the Netapp NFS share with NFSv3
Recreating the DB from a native RHEL7 PC improve the execution time to ~ 4 mins, but still there's a lot of difference compared to RHEL6.
BTW, I know that using DB over NFS is not the best option, but still, on RHEL6 it works fine.
Any clue ?
thank you very much in advance, and cheers Franco
(2) By Warren Young (wyoung) on 2020-05-01 02:23:18 in reply to 1 [source]
Perhaps file locking isn't properly configured on the RHEL 6 machines, allowing them to be faster but dangerous.
(3) By Rowan Worth (sqweek) on 2020-05-04 07:22:33 in reply to 1 [link] [source]
Huh, that's bizarre. Has the python version changed? It might be using a different sqlite version again. Can you time how long
.dump from an
sqlite3 shell takes from each client?
In my work we have countless SQLite databases hosted on nfs, with up to a dozen concurrent readers/writers on different machines, and many times the size of yours. We have very few problems due to nfs although the approach is limited in terms of scalability/concurrency.
I don't remember any significant slowdowns when we moved from CentOS6 to CentOS7, but that doesn't necessarily translate to RHEL. We're also using nfs4.
(4) By brasolin on 2020-05-04 07:56:11 in reply to 3 [link] [source]
thank you Rowan for your answer.
RHEL6 python 2.6.6 sqlite 3.6.23 RHEL7 python 2.7.5 sqlite 3.7.17
The .dump time is pretty fast (comparable) on both clients. On RHEL7 there's no difference in the execution time between NFSv3 and NFSv4