Hi,
I too have a question about database updates, in particular Fabrice's remark that "The GUI may not be the best place to run updates, did you try the basex command line tools ?"
I have an update that is working okay from the GUI, but I wouldn't mind it if it were faster. :-)
Is simply invoking the same XQuery using RUN from the command line all I have to do to get the benefits of using the CL?
In addition: my update query is configured with a bit of XML, something like
declare variable $build-db := <build db="new"> <source db="old1"/> <source db="old2"/> <source db="old3"/> </build>;
(Of course it's more complicated than this but you get the essence.)
But I can't pass this XML into a function to perform the updates, since I can't put update operations into a function at all. (So the XML is piled into a global variable, which the update operations reference.)
Do you have any suggestions for how I can encapsulate the updating logic and also make it configurable externally like this? And run it from the command line or a script for speed and efficiency?
Thanks, Wendell
-- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^
Dear Wendell,
It's just a simple practical hint : when I run several queries in the GUI, memory grows, And at the end a simple query can fail because memory is exhausted...
I read your xml example, Do you need to copy data from a collection to another ? I have exactly the same need.
So I ask again the BaseX users for advice, concerning this functionality.
Maybe the CREATE and ADD commands/functions could accept path from another collection as input, like doc() and collection() ?
For example :
<create-db name='target_collection'>source_collection/path/to/(mydoc)?</create-db>
or
Db:add('target_collection', 'source_collection/path/to/(mydoc)?')
Could give the same result that something like :
<create-db name='target_collection'>{collection('source_collection/path/to/(mydoc)?')}</create-db>
or
Db:add('target_collection', collection('source_collection/path/to/(mydoc)?'), db:list('source_collection/path/to/(mydoc)?'))
This would avoid the precious cost of an useless intermediate data extraction.
Best regards, Fabrice Questel-Orbit
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Wendell Piez Envoyé : mardi 16 avril 2013 17:46 À : BaseX Objet : [basex-talk] Running and configuring update operations outside the GUI
Hi,
I too have a question about database updates, in particular Fabrice's remark that "The GUI may not be the best place to run updates, did you try the basex command line tools ?"
I have an update that is working okay from the GUI, but I wouldn't mind it if it were faster. :-)
Is simply invoking the same XQuery using RUN from the command line all I have to do to get the benefits of using the CL?
In addition: my update query is configured with a bit of XML, something like
declare variable $build-db := <build db="new"> <source db="old1"/> <source db="old2"/> <source db="old3"/> </build>;
(Of course it's more complicated than this but you get the essence.)
But I can't pass this XML into a function to perform the updates, since I can't put update operations into a function at all. (So the XML is piled into a global variable, which the update operations reference.)
Do you have any suggestions for how I can encapsulate the updating logic and also make it configurable externally like this? And run it from the command line or a script for speed and efficiency?
Thanks, Wendell
-- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^ _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Fabrice,
Actually in my case it isn't as simple as copying it. The data is filtered along the way (both culled and enhanced).
Of course this doesn't contradict anything you've said. But there is a lot of data, so performance is an issue, and since we do this operation in batches, we don't really need the interactivity of the GUI for it, so the command line or a script is reasonable especially if it gives us speed.
(Mind you, the GUI has been great for developing this but it's basically working now so we're running it at scale.)
Cheers, Wendell
On Wed, Apr 17, 2013 at 3:51 AM, Fabrice Etanchaud fetanchaud@questel.com wrote:
Dear Wendell,
It's just a simple practical hint : when I run several queries in the GUI, memory grows, And at the end a simple query can fail because memory is exhausted...
I read your xml example, Do you need to copy data from a collection to another ? I have exactly the same need.
So I ask again the BaseX users for advice, concerning this functionality.
Maybe the CREATE and ADD commands/functions could accept path from another collection as input, like doc() and collection() ?
For example :
<create-db name='target_collection'>source_collection/path/to/(mydoc)?</create-db>
or
Db:add('target_collection', 'source_collection/path/to/(mydoc)?')
Could give the same result that something like :
<create-db name='target_collection'>{collection('source_collection/path/to/(mydoc)?')}</create-db>
or
Db:add('target_collection', collection('source_collection/path/to/(mydoc)?'), db:list('source_collection/path/to/(mydoc)?'))
This would avoid the precious cost of an useless intermediate data extraction.
Best regards, Fabrice Questel-Orbit
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Wendell Piez Envoyé : mardi 16 avril 2013 17:46 À : BaseX Objet : [basex-talk] Running and configuring update operations outside the GUI
Hi,
I too have a question about database updates, in particular Fabrice's remark that "The GUI may not be the best place to run updates, did you try the basex command line tools ?"
I have an update that is working okay from the GUI, but I wouldn't mind it if it were faster. :-)
Is simply invoking the same XQuery using RUN from the command line all I have to do to get the benefits of using the CL?
In addition: my update query is configured with a bit of XML, something like
declare variable $build-db :=
<build db="new"> <source db="old1"/> <source db="old2"/> <source db="old3"/> </build>;
(Of course it's more complicated than this but you get the essence.)
But I can't pass this XML into a function to perform the updates, since I can't put update operations into a function at all. (So the XML is piled into a global variable, which the update operations reference.)
Do you have any suggestions for how I can encapsulate the updating logic and also make it configurable externally like this? And run it from the command line or a script for speed and efficiency?
Thanks, Wendell
-- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^ _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Is simply invoking the same XQuery using RUN from the command line all I have to do to get the benefits of using the CL?
You could as well try to pack your code into some BaseX command scripts:
http://docs.basex.org/wiki/Commands#Command_Scripts
If you write your updating queries in XQuery, you are much more flexible, but (due to the underlying semantics) all update operations will first need to be cached before they can be executed, which means you may soon hit memory limits. You can elude those limits by creating a custom command script that contains all the update operations. In some scenarios, the creation of those scripts is directly done via XQuery, and deleted after having been successfully executed.
But I can't pass this XML into a function to perform the updates, since I can't put update operations into a function at all.
Do you refer to XQuery functions? This can be mode possible by adding the %updating annotation to your function declaration:
http://docs.basex.org/wiki/XQuery_Update#Function_Declaration
If this doesn’t help, feel free to send us a small XQuery snippet that demonstrates the issue.
Christian
basex-talk@mailman.uni-konstanz.de