Hi all,
I'm been running basex on two environments, one is my laptop with an SSD drive and ext4, the other is a server with raid-1 HDD and ext3.
Using iotop, I am watching the disk access of the server and laptop, and comparing the two.
After some update queries, I notice that the server (ext3) has a process, [kjournald], which writes for about 20 seconds at 600-1000KB/sec. This only begins after the update is complete and the java process actually stops writing. I am not experiencing such a thing on my laptop. If I do too many of these queries on the server, it will eventually hang for 20-30 seconds.
I've copied the exact data on both the laptop and the server.
Are there any recommended filesystem types for basex? Does it sound like I've configured something incorrectly, or should I add some more drives to my server and have them using ext4? Or is the difference SSD vs HDD? I never saw my SSD write speed go above 200KB/sec or so during the same queries.
Any help would be greatly appreciated!
- Adrian
Just an update,
I have installed a new ext4 volume on the server and moved basex data over to it. Updates are now fast, so there was definitely an improvement. I will wait to see it running for a whole day before I draw conclusions, but I wasn't able to reproduce my high-io kjournald problem (or any other similar problem) with the data on ext4.
- Adrian
On Sat, Jan 7, 2012 at 1:33 AM, Adrian Hossu adibadi@gmail.com wrote:
Hi all,
I'm been running basex on two environments, one is my laptop with an SSD drive and ext4, the other is a server with raid-1 HDD and ext3.
Using iotop, I am watching the disk access of the server and laptop, and comparing the two.
After some update queries, I notice that the server (ext3) has a process, [kjournald], which writes for about 20 seconds at 600-1000KB/sec. This only begins after the update is complete and the java process actually stops writing. I am not experiencing such a thing on my laptop. If I do too many of these queries on the server, it will eventually hang for 20-30 seconds.
I've copied the exact data on both the laptop and the server.
Are there any recommended filesystem types for basex? Does it sound like I've configured something incorrectly, or should I add some more drives to my server and have them using ext4? Or is the difference SSD vs HDD? I never saw my SSD write speed go above 200KB/sec or so during the same queries.
Any help would be greatly appreciated!
- Adrian
Hi Adrian,
Am Samstag, 7. Januar 2012, 01:33:24 schrieb Adrian Hossu:
Hi all,
I'm been running basex on two environments, one is my laptop with an SSD drive and ext4, the other is a server with raid-1 HDD and ext3.
Using iotop, I am watching the disk access of the server and laptop, and comparing the two.
After some update queries, I notice that the server (ext3) has a process, [kjournald], which writes for about 20 seconds at 600-1000KB/sec.
kjournald is a kernel thread which updates the journal [1] of the file system.
This only begins after the update is complete and the java process actually stops writing. I am not experiencing such a thing on my laptop. If I do too many of these queries on the server, it will eventually hang for 20-30 seconds.
Ext3 (and ext4) have different levels of journaling [2], defining what and when gets written to the journal and the main file system. It sounds to me, as if your file system is journaling both meta-data and file contents, since kjournald write a lot of data. I suggest, that you check with what arguments your file systems are mounted. Generally, you can see that in /etc/mtab; the corresponding line should contain something like "data=ordered". You could also check the documentation of your linux distribution.
I've copied the exact data on both the laptop and the server.
Are there any recommended filesystem types for basex?
Personally, I'm not aware if there is a recommended file system, I think no, but generally it Should Just Work(tm). However, if someone has experimented with different types of file systems and/or types of disks may share results.
Does it sound like I've configured something incorrectly, or should I add some more drives to my server and have them using ext4? Or is the difference SSD vs HDD? I never saw my SSD write speed go above 200KB/sec or so during the same queries.
Any help would be greatly appreciated!
- Adrian
Regards, Dimitar
P.S. it's recommended to use Btrfs with SSDs, since the file system is designed to improve the wear leveling of the disk [3].
[1] http://en.wikipedia.org/wiki/Journaling_file_system [2] http://en.wikipedia.org/wiki/Ext3#Journaling_levels [3] http://en.wikipedia.org/wiki/Btrfs
basex-talk@mailman.uni-konstanz.de
-
Adrian Hossu -
Dimitar Popov