…more input would be welcome ;)
Marco, I've faced the same problem and frustration, and I agree that it would be great if BaseX could be used for such tasks. Strengthening its potential role as a data integration engine. Kind regards, Hans
Marco Lettere m.lettere@gmail.com schrieb am 10:57 Freitag, 11.November 2016:
Hi all,
I'm currently facing the issue of exporting a large table stored in a MySQL instance to a csv that has then to be zipped and served through a web service.
I'd love to use XQuery and the Basex functionality for doing it because it feels perfectly natural.
The problem is that the XQuery script that performs the SQL Query and transformation to CSV crashes with an Out of Memory exception.
I know how to increase memory and so on but I'd like to find a stable solution that is as less as possible dependent from the system requirements.
Currently the only idea is to page the operation and (maybe exploiting parallel execution) build an intermediate result on disk files that will be assembled before delivery.
Isn't there any more elegant solution based somehow on the streaming functionality or anything like this?
Thanks for any advice you can provide.
Regards,
Marco.