Hi Erol,
it's true that one database and one single XQuery call won't be enough to process data of that size. As BaseX does not natively support distributed storage and parallel queries, I would suggest the classical (manual) way: split the data to multiple databases, run the query on each instance and eventually merge the textual results. This could be done in a few lines of code, provided that the queries are not sophisticated, the database contents are disjunct, no concurrency issues have to be considered, etc.
Hope this helps, Christian
On Tue, Jan 12, 2010 at 3:22 AM, Erol Akarsu eakarsu@gmail.com wrote:
Hi Christian,
I appreciate a new release of BaseX.
I have one question on storing a huge data,say 10-50 petabytes data, with Basex. I expect current version may not handle it properly. Can we distribute XML data across clusters of nodes and have a parallel xquery to be run across nodes and get result? I would like to know how it can be implemented such a big clustered xml database and have efficient xquery on this cluster.
Regards
Erol Akarsu
On Sun, Jan 10, 2010 at 4:49 PM, Christian Grün christian.gruen@gmail.com wrote:
Dear all,
we are excited to finally announce the release of BaseX 6.0! It offers a bunch of new features, such as:
- a full implementation of XQuery Update, reaching 100% in the W3C Test
Suite.
- an enhanced client/server architecture with ACID-safe transactions,
user and password management and transaction logging.
- extended XQuery Full Text features, supporting TF/IDF scoring, stop
word lists, fuzzy querying, etc.
- the improved creation of index structures, using constant memory.
- optimizations to increase the use of index structures and iterative
query processing.
As usual, please find more information at http://basex.org/
All the best,
Christian Grün BaseX Team _______________________________________________ BaseX-Announce mailing list BaseX-Announce@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-announce