Hi Anastasiou,

Hopefully some of these answers are somewhat helpful.

On Tue, Sep 12, 2017 at 4:54 AM, Anastasiou A. <a.anastasiou@swansea.ac.uk> wrote:

Hello everyone

I am trying to load BaseX with a large number of XML files (~500), each one a few hundreds of MBs big.

BaseX fails with a message along the lines “This is too big for one database”.

Can I please ask:

1) Are there any logs, beyond the DB logs? If yes, where can I find them?

a. The reason I am asking is because once basexgui gives the message, there is no indication about the error.
Ideally, I would like to know if this is a limitation on memory amount or number of items (?).

I'm not sure how to enable more verbose logging with the GUI -- hopefully one of the devs or power users can weigh in on this.

2) The parser options include reading XML files from archives, which is very convenient, but once the file has been
parsed, does BaseX require the “originals” for queries / returning results?

AFAIK, no it does not. BaseX will query and return results from the internal database(s).

3) Is it possible to do federation with BaseX? In other words, let’s say I split a database in two large parts (as per #1),
is it possible to launch two baseX servers and then have them talk to each other so that ultimately I just query one of
them and get back unified results?

AFAIK, the preferred method is to split your files across many databases, then query multiple databases from a single expression[1]. Others will be able to speak to this better, but I don't think there's a straightforward way to run multiple BaseX servers in a single JVM.

All the best

Best,

Bridger

[1] http://docs.basex.org/wiki/Databases