Christian,
I had played around with the AUTOFLUSH option and did notice some improvement, which is great for my own uses when appropriate. If that option is set to false, how often does the database actually get flushed? Only on close? Does it flush before index rebuilds from Optimize?
With the OutputStream question, I was more interested in systematic sources of delay that could potentially be mitigated (thinking about other users of Nxdb). This seemed like an instance where there might be room for improvement - however, I'm not sure the problem even exists outside the cross-compiled case.
Thanks,
Dave
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Wednesday, March 14, 2012 5:03 PM To: Dave Glick Cc: BaseX Subject: Re: [basex-talk] Any way to persist OutputStream in DiskData?
Hi Dave,
thanks for your analysis - profound as usual. You may get much better results by setting the AUTOFLUSH option to false [1]. Please tell me if you've done that already..
Christian
[1] http://docs.basex.org/wiki/Options#AUTOFLUSH ___________________________
On Wed, Mar 14, 2012 at 8:09 PM, Dave Glick dglick@dracorp.com wrote:
Hi,
I've been doing some profiling recently and noticed one area in particular that appears to be using a lot of cycles. In calls to DiskData.write() the final DataOutput.close() call results in 20% - 40% of overall execution time in a variety of cases. The following screen capture shows the aggregated call stack (though it might look a little funny since this is the cross-compiled version used for Nxdb). Admittedly, this is probably an issue that just impacts me. Based on the trace, I suspect the issue is related to the releasing of handles rather than the actual disk writes (though I could be wrong - I'm not entirely sure what the cross-compiler does behind the scenes to support Java file descriptors).
In any case, my question boils down to wondering if there is a way to force the DiskData class to use a persistent DataOutput instance instead of creating and closing a new one for every write? Instead, could a DataOutput be kept open until the database is "unpinned" and simply flushed on each write? Would this cause any problems with the current GUI, server, etc. uses?
Thanks,
Dave
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk