Hi Marco,
Finally some feedback. As you may already have guessed, there is no obvious answer to your question. A straightforward query could look as follows..
let $docs-a := db:open('A1')[ ..condition.. ] let $paths := $docs-a ! db:path(.) let $docs-b := $paths ! db:open('A1', .) return ( db:delete('A1', $paths), db:add('A2', $docs-a), db:delete('B1', $paths), db:add('B2', $docs-b) )
..but it probably leads to out of memory exceptions. An alternative could be to serialize the documents to disk and try to add it in a second step. If this doesn't work out neither, I would indeed propose to move the documents in chunks. A BaseX command script for that could be generated via XQuery.
I will think about the implications of db:copy-doc and db:move-doc functions. It's right that such scripts may not be obvious candidates for OOM errors, because we already have database representations for the documents to be moved/copied. Another solution to avoid OOM would be to write the data structures of copied documents to disk instead of keeping them in main memory. Mostly a matter of when we find enough time to make it happen..
Cheers, Christian
- 2 dbs (let's say A and B) with around 200K documents each
- every document in B refers to exactly one document in A (many to one is
possible)
- I have to apply a filter on every document of A and depending on the
boolean outcome "move" the document to anoter db A'.
- According to this the correlated documents of B need to be "moved" to a
database B'.
- The solution should be made automatic in the mid term
What is the best design pattern and tool (I use the GUI by now) for this finding the right compromise for limiting "Out of memory (huge PUL)", "too much time", "complexity introduced by chunking" and "absence of a _db:move-document_ operation for moving one document in one single operation?
Every suggestion is more then warmly welcome!
Thanks and best regards, Marco