Java heap space at C:\Users\lourduswamym\oracle_scripts/BaseXClient.pm line 213.
How much memory have you assigned to BaseX [1]?
Actually the error on the server log. The insert operation was taking 1+minutes due to slow network and then the error happened. I have been running this server instance continuously for 1 day with continuous insert from a per script.
If you create a new database, you can specify a directory with the files that are to be initially added to your database. This might save you a lot of time.
So I think I should be careful to run only one single process of client at a time or make sure memory is enough to handle multiple client operations, please let me know your thoughts,
If your documents are small, you shouldn’t usually encounter any errors. But it might have to do with the large number of documents you are dealing with. One more option is to distribute your docs across multiple databases (you can access all of them via a single XQuery expression).
Best, Christian
[1] http://docs.basex.org/wiki/Start_Scripts
Thanks again, Regards Martin Lourduswamy
On Sun, May 20, 2018 at 9:48 AM, Martin Lourduswamy martin.louis@gmail.com wrote:
Hi,
Thanks for the clarifications. My database is
DB Size:
- each XML node => 500 bytes
- total size of XML DB => 10million nodes(which will grow continuously as
new files are added). 3. I will be having multiple DB's of same size and more size as the DB continuously grows
I have to do a update of the nodes as I need to replace them. I chose delete + insert instead of replace for each of the node through perl, as I will make sure there are no duplicate nodes left in the database.(just a precaution that even if some nodes get duplicated, through some other processes I will be able to remove them by delete)
DB Speed:
I will disable logging as it might speed up the database operations.
DB Architecture
1 server(windows or Linux) running baseX multiple client computers trying to query the same machine for select through fn:doc(...) API to get the data through perl then do a delete + insert from perl
I would like to know if you think this might work for production and any suggestion for data retrieval and update Any changes to the way I architecture it might help
Also, I am thinking of indexing on the node id, as that is like the primary key on which I base my operations, any suggestions might also be helpful, Thanks again, Regards Martin Lourduswamy
On Sun, May 20, 2018 at 9:28 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Martin,
I am new to BaseX, I would like ot speed up XQuery of insert and delete and replace through options.
Welcome to the list. As there are numerous ways to do updates in BaseX, feel free to give us more information on your insert and delete operations. Do you work with large single documents or many small documents? How large is your database?
While I query basex through perl, and try to connect through GUI, the perl connection aborts. Is there a parameters for parallel connections, please let me know.
I can’t tell why your perl connection is interrupted by opening the GUI (because they should be completely independent from each other). A step-by-step description on how you proceeded might be helpful.
Because there is no coupling between GUI and the client/server architecture, however, you should avoid running updates outside the GUI. Please check out [1] for more information.
Best, Christian
[1] http://docs.basex.org/wiki/Startup#Concurrent_Operations
Thanks, Regards Martin Lourduswamy