I'm currently at work and my setup is at home. In about 7 hours I'll get home and I will send the stack trace.
Meanwhile, is there any way to write a FLWOR, a loop, in a batched style?
Like for example in my case, this approach I described to migrate data from BaseX to PostgreSQL makes use of BaseX as an XQuery processor and transfers the full-text indexing to PostgreSQL, this is what I'm trying to do.
However, in order to avoid OOM, I am thinking of batching the transfer into chunks, and potentially restart the BaseX server in between the migration of each chunk.
That's why I am asking how I could do that in BaseX. My hope is that the OOM could be avoided in this way, because not all the data would pass through main memory and there would be less chances of the JVM GC having to deal with this data.
Restarting the BaseX server between each chunk transfer would help making sure that whatever memory was used is released.
So I wonder if something like
(<insert-big-FLWOR-here)[position() = <start> to <end>]
would work here. Of course, some count would have to be done beforehand to know how many batches there will be. Or maybe even without knowing how many batches there will be, a while-type loop could be written in Bash with the stop conditon being to check if the current chunk is empty.
Would an approach like this work to mitigate the OOM? Are there alternatives or work-arounds to this kind of OOM?
Thanks