Hi Anastasiou,
When adding many big documents, I usually set the ADDCACHE option [1], and add files sequentially (for example in a BaseX command script). So, when I hit the db size limit, no work is lost and I can continue adding the remaining files in a new db.
Best regards, Fabrice Etanchaud
[1] http://docs.basex.org/wiki/Options#ADDCACHE
De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Anastasiou A. Envoyé : mardi 12 septembre 2017 11:01 À : 'basex-talk@mailman.uni-konstanz.de' Objet : [basex-talk] FW: A few general questions about BaseX
I am sorry, turns out the error is probably due to malformed input in one of the files which I will have to look into, not BaseX, would however still appreciate some indication regarding the rest of the questions.
All the best
From: Anastasiou A. Sent: 12 September 2017 09:54 To: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Subject: A few general questions about BaseX
Hello everyone
I am trying to load BaseX with a large number of XML files (~500), each one a few hundreds of MBs big. BaseX fails with a message along the lines "This is too big for one database".
Can I please ask:
1) Are there any logs, beyond the DB logs? If yes, where can I find them?
a. The reason I am asking is because once basexgui gives the message, there is no indication about the error. Ideally, I would like to know if this is a limitation on memory amount or number of items (?).
2) The parser options include reading XML files from archives, which is very convenient, but once the file has been parsed, does BaseX require the "originals" for queries / returning results?
3) Is it possible to do federation with BaseX? In other words, let's say I split a database in two large parts (as per #1), is it possible to launch two baseX servers and then have them talk to each other so that ultimately I just query one of them and get back unified results?
All the best