Also, you can have the best of both worlds, saving results in a pipeline if it's helpful for debugging, just running the updates in place once you know it all works well. This is particularly helpful if your data is complex and your datasets are largish.
I am currently merging various sources that were not originally meant to work together, creating a common reference system for fairly complex data. I find it helpful to write a bunch of simple updates that can be executed sequentially, running them with a .bxs file. If I want to see the output after any of these updates, I can do so with an OUTPUT statement. Most of these stages are only useful for debugging purposes, so once a pipeline is working well, I get rid of the OUTPUT statement.
If your data is large, it can also be helpful to have two different statements for importing the data, one that imports everything, another that imports only a small test subset that runs quickly. Comment one of them out and use the other. For rapid development, use the small, quick subset, then run it on the whole dataset once it works.
Jonathan
On Sun, Feb 13, 2022 at 6:19 PM Christian Grün christian.gruen@gmail.com wrote:
Hi Eric,
That’s perfectly feasible. You can bind the dynamic query string to an external variable and invoke it on extracted data via xquery:eval. Here’s one way to do it:
(: db.xml :) <path><to><source><data/></source></to></path>
(: query.xq :) declare variable $query external;
for $data in db:open('db')/path/to/source/data let $updated := xquery:eval($query, map { '': $data }) return replace node $data with $updated
The query could e.g. be invoked via
basex -b query="<new>{ . }</new>" query.xq
Hope this helps, Christian
On Sun, Feb 13, 2022 at 11:26 PM Eric Levy contact@ericlevy.name wrote:
I am completely new to BaseX, and trying to conceive of a way to use it to process a compound query that combines an external query with a static one.
The core idea is a very simple application, expressed as an XQuery file, that results in the following sequence of actions:
- Extraction of data from a database, by some query.
- Modification of the results of (1), by some query.
- Persistence of the results from (2), back into the database from which the data was originally extracted.
The modifications given in (2) would be expressed external to the application, supplied as input per invocation, in a separate file or in a command-line parameter.
Thus, the overall operation would entail static components, the fixed parts of the application, represented by (1) and (3), and dynamic components, passed by the invocation, represented by (2).
I realize such a processing pipeline may be achieved through queries supplied in a full application accessing the database by one of the language-specific APIs, but I am trying to consider an approach that requires use only of XQuery statements, without involving a separate programming language or formal build process.
Does the design of BaseX accommodate an application having the structure as described?