Hello Max,
thanks for the input.
I had similar thoughts. My idea was to have completely separated systems (and databases). One system to manage the updates and one for readonly access for the end users.
And periodically we will update (replace) the readonly databases with the new updated ones.
Best regards (also from Lindau :-)) Johannes
Am 21.02.2019 um 10:14 schrieb Maximilian Gärber:
Hello Johannes,
our best practice for similar scenarios is to have two instances of basex running. One that is mainly collecting data from various sources (http api, ftp, filesystem) and another instance which then gets the collected infos all at once.
BaseX really shines at processing larger chunks (and querying big documents) - think batching. But it is not well suited for hundreds of updating requests at the same time. Especially not if you have a user interface that should stay responsive at all times.
Greetings from Lindau,
Max Gärber
Am Di., 19. Feb. 2019 um 14:53 Uhr schrieb Johannes Bauer johannes.bauer@tanner.de:
Hi,
I need some input on how best to implement an updating database operation.
I use BaseX within a Web Application and have RESTXQ service endpoints.
One such service collects information from multiple databases and creates a new document (with the collected infos) in another database. The list of affected databases is dynamic and will be determined by the RESTXQ module.
I'm afraid that, due to the missing static database information in the query, the service will lock *all* databases and because I have an updating operation the locks will be write locks. Depending on the amount of databases and their size, the operation might take some time (maybe even up to some minutes). And I don't want to have write locks for such a long period of time.
I've also tried another approach:
I use a non updating service that collects the data and then calls another RESTXQ service per HTTP request that makes the db update. This way I only have read locks on *all* databases and then a short single write lock on the modified db.
But this did not work. The write call deadlocked. The first service probably has not released the read locks yet when the second service is called.
I also tried to set the QUERYINFO option to see some lock information for my module, but I didn't get any output.
Do you have any best practice on how to approach such a task?
Best regards Johannes