Hi Christian,
I wish I had more time to contribute. However I'm interested in having a look at the documentation of the XQuery engine structure. I might then be able to do some suggestions. Where should I start reading from?
Gilles.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: jeudi 22 mai 2014 10:20 To: Gilles CARRY Cc: the joj; BaseX Subject: Re: [basex-talk] multi-core performance of BaseX
Hi Gilles,
true, the CPU is often a bottleneck today.
We are always interested in suggestions on how to parallelize XQuery in an effective way; suggestions are welcome.
Christian
On Thu, May 22, 2014 at 10:14 AM, Gilles CARRY gilles.carry@st.com wrote:
Hi,
Just my 2cts: We have a server running BaseX and configured to hold all data in memory (RAM=64GB). After collecting a few metrics, it turns out the bottleneck is the Cpu (100% and virtually no disk access) and unfortunately, BaseX only uses 1 cpu over the 8 the machine has.
Best regards, Gilles.
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Christian Grün Sent: lundi 19 mai 2014 10:40 To: the joj Cc: BaseX Subject: Re: [basex-talk] multi-core performance of BaseX
Hi Roy,
I was wondering if anyone has some information about BaseX multi-core performance.
Does BaseX utilize multi-core architecture to parallel-process the Xqueries?
There has been some discussion on the list in the past; see e.g.:
http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03076.ht... http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg03159.ht...
http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg02 018.html
A general note: as BaseX is a full programming language, it is a non-trivial problem to compute arbitrary queries in parallel, and I'm not aware of any other XQuery processor that provides such capabilities. Disk access is often slower when done in parallel (in particular on conventional hard-disks), as parallel, uncoordinagted queries may lead to random access pattern.
Potential solutions could be to provide...
- functions for explicitly creating and joining threads
- optimize special XQuery expressions for parallel executions.
Hope this helps, Christian