Hi list,
I am currently looking into BaseX to find out if it meets my requirements for an XML database with XQuery implementation and REST interface. I already got the data imported, now I'm looking into the queries I need. I just stumbled upon the following problem, and could not find any documentation to overcome it, just a bug report that I think is related (https://github.com/BaseXdb/jax-rx/issues/3).
The problem is that I need to work on XML data that is POSTed to the query, the data must be available during query execution. However I could not find a way to do so, the jax-rx parameters seem to be restricted to atomic values. Did I miss anything?
As work-around, I tried to construct the entire query in the database client (which in my case is Orbeon, the runtime for the XForms application that serves as user interface), serializing the XML into a string and making it part of the query, i.e. something like let $data = <xsl:value-of select="saxon:serialize($data)"/> The query is created correctly, however, I can not get it to execute on the server as BaseX requires a content-type of application/query+xml, whereas XForms submissions (the only way to issue POST requests from Orbeon) have a content-type of application/xml.
Is there any way to pass an XML data parameter, or at least to send a custom query without using a special mime-type?
Any help would be appreciated, Ralf
Dear Ralf,
thanks for your feedback. With BaseX 6.8 (which is to be expected beginning of October), we're switching to our own, native REST implementation, which will allow us to put in new features much more easily, and enable a tighter integration of XQuery and other database features. The documentation [1] and snapshots [2] are already online.
This means, on the other hand, that we won't do any more feature updates on JAX-RX, so we'll probably move your request to the basex-api repository.
For now, you could try to specify your input file within the XQuery and bind it e.g. to the context item, as shown in the following example:
<query xmlns="http://jax-rx.sourceforge.net"> <text><![CDATA[
declare context item := <....YourXMLInput.../>; ...your query...
]]></text> </query>
Instead, we could introduce an <input/> element, the child of which would represent the element to be bound as context item. Would this be helpful?
Thanks, Christian
[1] http://docs.basex.org/wiki/REST [2] http://files.basex.org/releases/latest/
___________________________
On Wed, Sep 21, 2011 at 4:59 PM, Ralf Jung ralfjung-e@gmx.de wrote:
Hi list,
I am currently looking into BaseX to find out if it meets my requirements for an XML database with XQuery implementation and REST interface. I already got the data imported, now I'm looking into the queries I need. I just stumbled upon the following problem, and could not find any documentation to overcome it, just a bug report that I think is related (https://github.com/BaseXdb/jax-rx/issues/3).
The problem is that I need to work on XML data that is POSTed to the query, the data must be available during query execution. However I could not find a way to do so, the jax-rx parameters seem to be restricted to atomic values. Did I miss anything?
As work-around, I tried to construct the entire query in the database client (which in my case is Orbeon, the runtime for the XForms application that serves as user interface), serializing the XML into a string and making it part of the query, i.e. something like let $data = <xsl:value-of select="saxon:serialize($data)"/> The query is created correctly, however, I can not get it to execute on the server as BaseX requires a content-type of application/query+xml, whereas XForms submissions (the only way to issue POST requests from Orbeon) have a content-type of application/xml.
Is there any way to pass an XML data parameter, or at least to send a custom query without using a special mime-type?
Any help would be appreciated, Ralf _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Christian,
thanks for the quick reply!
thanks for your feedback. With BaseX 6.8 (which is to be expected beginning of October), we're switching to our own, native REST implementation, which will allow us to put in new features much more easily, and enable a tighter integration of XQuery and other database features. The documentation [1] and snapshots [2] are already online.
The release is planned so soon - so, can I assume that current git master is reasonably stable? I could then directly start using it, then I do not have to switch the REST interface later.
For now, you could try to specify your input file within the XQuery and bind it e.g. to the context item, as shown in the following example:
<query xmlns="http://jax-rx.sourceforge.net"> <text><![CDATA[
declare context item := <....YourXMLInput.../>; ...your query...
]]></text>
</query>
Unfortunately, I can not send any <query> to the database due to the mime type restriction: I can not freely set the mime type, all I can do are POST queries with a mime-type of application/xml (at least I found no way to change that, I'll ask on the Orbeon mailing list). From all I read, that means I can not send POST queries to BaseX at all. A possible fox would be, to interpret a POST query to a URL like /rest?query with a mime-type of application/xml the same way as a query to /rest with a mime type of application/query+xml . Actually I was quite surprised to see that POST can also be used to store documents, which I'd usually expect to be the behaviour of PUT - it seems the both are almost the same in BaseX?
Kind regards, Ralf
On 09/21/2011 05:55 PM, Ralf Jung wrote:
Actually I was quite surprised to see that POST can also be used to store documents, which I'd usually expect to be the behaviour of PUT - it seems the both are almost the same in BaseX?
I think POST should be used if the server decides how to name the resource and PUT if the user decides how to name the resource. The nice thing about PUT is that it's idempotent.
Thus I would suggest that it's possible to create resources with POST but the BaseX-server decides how to name the resource, that is the URI in a POST-request only "identifies the resource that will handle the enclosed entity"[1]. It maybe shouldn't be possible to specify the resource name in the URL using the POST method, just the database within the new resource should be created, but the resource name is up to the BaseX-server and a Location header should point to the new created resource.
kind regards, Johannes
[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html -- section 9.6 PUT
Hi Ralf,
The release is planned so soon - so, can I assume that current git master is reasonably stable? I could then directly start using it, then I do not have to switch the REST interface later.
Yes, indeed I'd recommend to use our latest code base; there shouldn't be any major changes before the next release anymore. All differences between 6.7.1 and 6.8 should be marked in our Wiki (..search for "6.8").
Unfortunately, I can not send any <query> to the database due to the mime type restriction: I can not freely set the mime type, all I can do are POST queries with a mime-type of application/xml (at least I found no way to change that, I'll ask on the Orbeon mailing list). From all I read, that means I can not send POST queries to BaseX at all.
I see. One possible fix in BaseX/REST would be to get rid of the "application/query+xml" content type (we thought about that before) and treat XML input in another way; this would introduce some inconsistencies with earlier versions, but as we're currently switching to a new implementation anyway, this would a good moment. I'll have to spend some more thoughts on that first, however; let's see what the Orbeon guys say.
Actually I was quite surprised to see that POST can also be used to store documents, which I'd usually expect to be the behaviour of PUT - it seems the both are almost the same in BaseX?
We're following the RESTful sementics, as e.g. presented in Wikipedia [1]. While PUT is used to update existing resources, POST is applied to create or add new resources.
Hope this helps, Christian
[1] http://en.wikipedia.org/wiki/Representational_state_transfer#RESTful_web_ser...
Hi Christian,
Yes, indeed I'd recommend to use our latest code base; there shouldn't be any major changes before the next release anymore. All differences between 6.7.1 and 6.8 should be marked in our Wiki (..search for "6.8").
Okay, I'm trying that.
I see. One possible fix in BaseX/REST would be to get rid of the "application/query+xml" content type (we thought about that before) and treat XML input in another way; this would introduce some inconsistencies with earlier versions, but as we're currently switching to a new implementation anyway, this would a good moment. I'll have to spend some more thoughts on that first, however; let's see what the Orbeon guys say.
Essentially, that's what I would consider the cleanest solution - making the behaviour depend on the request URI, not the content type. But then, I only started to have a look at BaseX yesterday ;-)
We're following the RESTful sementics, as e.g. presented in Wikipedia [1]. While PUT is used to update existing resources, POST is applied to create or add new resources.
Interesting... of course, you should follow the common semantics then. Thanks for the clarification.
Kind regards, Ralf
Hi Christian,
okay, I solved the mime type problem with a patch against Orbeon. I don't know if it's really a good idea, but it works for now. Back to the other problem, passing an XML document to a query.
Instead, we could introduce an <input/> element, the child of which would represent the element to be bound as context item. Would this be helpful?
So, what would the request look like? After all I also need to specify the xquery file somewhere.
Kind regards, Ralf
Instead, we could introduce an <input/> element, the child of which would represent the element to be bound as context item. Would this be helpful?
So, what would the request look like? After all I also need to specify the xquery file somewhere.
..it could look as follows (I'm using the latest REST syntax):
<query xmlns="http://www.basex.org/rest"> <text> (: example query :) /a/b </text> <input> <a> <b>example</b> </a> </input> </query>
Regarding problems with REST: Personally, I'm developing with Eclipse and the m2eclipse plugin for Maven, which is why unfortunately I cannot give you help on your Maven problem. You might as well try our latest builds..
http://files.basex.org/releases/latest/
..that you can run with the start scripts [1] and libraries [2] provided in GitHub. If you are running BaseX, you won't need any additional libraries (indeed, most external libs are needed for the new WebDAV service based on the Milton API).
Christian
[1] https://github.com/BaseXdb/basex-api/tree/master/etc [2] https://github.com/BaseXdb/basex-api/tree/master/lib
Hi Christian,
..it could look as follows (I'm using the latest REST syntax):
<query xmlns="http://www.basex.org/rest"> <text> (: example query :) /a/b </text> <input> <a> <b>example</b> </a> </input> </query>
While this would already be a step forwards, I'd also like to pass such an XML parameter to a query that is loaded from a file, so that I don't have to send it to the server again and again. Something like that, to also allow run queries via POST:
<run xmlns="http://www.basex.org/rest"> <file>query.xq</file> <!-- or maybe use "text" here as well, for consistency? sounds weird though --> <input><document/></input> <variable name="x" value="something"/> </run>
This would then allow all the three types of requests (query, run, command) both in GET and POST style.
I assume that, if the XML is set to be the context of the entire XQuery script, I can access it within the script like this?
let $param := . let $something := $param//text() ...
Regarding problems with REST: Personally, I'm developing with Eclipse and the m2eclipse plugin for Maven, which is why unfortunately I cannot give you help on your Maven problem. You might as well try our latest builds..
http://files.basex.org/releases/latest/
..that you can run with the start scripts [1] and libraries [2] provided in GitHub. If you are running BaseX, you won't need any additional libraries (indeed, most external libs are needed for the new WebDAV service based on the Milton API).
I see, the lib folder over there is useful, thanks... yeah, I know I can downloaded the precompiled jar files, but if I am using development versions I prefer to compile them myself, so that I can more easily try to change something here or there, and know exactly which version I got. I guess I will have to hope for someone to drop by and explain how to build that basex-api jar file, or for google to come up with something useful wrt Maven.
Kind regards, Ralf
<run xmlns="http://www.basex.org/rest"> <file>query.xq</file> <!-- or maybe use "text" here as well, for consistency? sounds weird though --> <input><document/></input> <variable name="x" value="something"/> </run>
..this shouldn't be any problem, as the "query" and "run" operations are very similar (see http://docs.basex.org/wiki/REST_POST_Schema).
I assume that, if the XML is set to be the context of the entire XQuery script, I can access it within the script like this?
let $param := . let $something := $param//text() ...
Exactly.
Regards, Christian
Ralf,
you can be lucky, we've just realized another one of your feature requests. Your feedback is quite helpful indeed, as you're currently the first active external tester of our integrated REST implementation ;)
The initial context node for a query can now be specified by adding an additional <context/> item:
<query xmlns="http://www.basex.org/rest"> <text>for $i in .//text() return string-length($i)</text> <context> <xml> <text>Hello</text> <text>World</text> </xml> </context> </query>
We're still pondering how to deal with the content type issue; have you got any feedback from Orbeon? Christian ___________________________
On Thu, Sep 22, 2011 at 12:38 AM, Christian Grün christian.gruen@gmail.com wrote:
<run xmlns="http://www.basex.org/rest"> <file>query.xq</file> <!-- or maybe use "text" here as well, for consistency? sounds weird though --> <input><document/></input> <variable name="x" value="something"/> </run>
..this shouldn't be any problem, as the "query" and "run" operations are very similar (see http://docs.basex.org/wiki/REST_POST_Schema).
I assume that, if the XML is set to be the context of the entire XQuery script, I can access it within the script like this?
let $param := . let $something := $param//text() ...
Exactly.
Regards, Christian
Hi Christian,
wow, that's quick :) However, after installing that update, I am now getting an error when doing a normal GET query that used to work fine:
GET http://localhost:8984/rest?run=list.xq Stopped at line 1, column 7: [XPDY0002] No context item set for 'list.xq'.
Is that a regression?
Kind regards, Ralf
On Thursday 22 September 2011 16:49:51 Christian Grün wrote:
Ralf,
you can be lucky, we've just realized another one of your feature requests. Your feedback is quite helpful indeed, as you're currently the first active external tester of our integrated REST implementation ;)
The initial context node for a query can now be specified by adding an additional <context/> item:
<query xmlns="http://www.basex.org/rest"> <text>for $i in .//text() return string-length($i)</text> <context> <xml> <text>Hello</text> <text>World</text> </xml> </context> </query>
We're still pondering how to deal with the content type issue; have you got any feedback from Orbeon? Christian ___________________________
On Thu, Sep 22, 2011 at 12:38 AM, Christian Grün
christian.gruen@gmail.com wrote:
<run xmlns="http://www.basex.org/rest"> <file>query.xq</file> <!-- or maybe use "text" here as well, for consistency? sounds weird though --> <input><document/></input> <variable name="x" value="something"/> </run>
..this shouldn't be any problem, as the "query" and "run" operations are very similar (see http://docs.basex.org/wiki/REST_POST_Schema).
I assume that, if the XML is set to be the context of the entire XQuery script, I can access it within the script like this?
let $param := . let $something := $param//text() ...
Exactly.
Regards, Christian
(forgot to CC list in first mail, sorry)
Hi again,
you can be lucky, we've just realized another one of your feature requests. Your feedback is quite helpful indeed, as you're currently the first active external tester of our integrated REST implementation ;)
I'm glad if I can help ;-) . The REST API looks good overall, the only thing I do not like about it is the special content-type it needs (I sent a pull request for a bugfix that was necessary to make it entirely work here).
I did not get any reply from Orbeon (usually, Erik checks the list once or twice a week). For now I patched Orbeon myself to support that content type when the serialization setting of the XForms submission is set accordingly. But maybe there is a better solution, one that uses some property of the request URI to distinguish POSTED data from POSTED queries, instead of the content-type? For example, I don't think POSTing to a URL like /rest/collection?query makes much sense if you want to store a file - is the ?query ignored (then it could instead be used as indicator that this is NOT a request to store data), or should it get part of the filename (which would be quite surprising). The only problem I see is that POST a <run>...</run> to ?query or ?command does not make much sense, IMHO that should be considered an invalid request. I attached a completely untested draft patch of what I meant. But then, I do not know the reasons for why the interface looks the way it does, so maybe my suggestion is completely against your intentions ;-)
Kind regards, Ralf
On Thursday 22 September 2011 20:48:54 you wrote:
Is that a regression?
Whoops.. no, that was a bug, which is now fixed.
Sorry, Christian
Ralf,
we had some internal discussions to balance pros and cons of the current approach (or, to call it differently, and somewhat bluntly: to balance RESTfulness vs. usability..).
As a result, we're pretty sure now that we won't loose any substantial functionality by stripping the POST request down to the query functionality and getting rid of its mainly redundant ADD functionality. And I just see that eXist has been doing exactly the same thing for a long time ;) Personally, I very much favor this change.
If there are any other users on the list who object to the projected changes to our REST API.. please cry out loud!
Christian
On Fri, Sep 23, 2011 at 6:03 PM, Ralf Jung ralfjung-e@gmx.de wrote:
(forgot to CC list in first mail, sorry)
Hi again,
you can be lucky, we've just realized another one of your feature requests. Your feedback is quite helpful indeed, as you're currently the first active external tester of our integrated REST implementation ;)
I'm glad if I can help ;-) . The REST API looks good overall, the only thing I do not like about it is the special content-type it needs (I sent a pull request for a bugfix that was necessary to make it entirely work here).
I did not get any reply from Orbeon (usually, Erik checks the list once or twice a week). For now I patched Orbeon myself to support that content type when the serialization setting of the XForms submission is set accordingly. But maybe there is a better solution, one that uses some property of the request URI to distinguish POSTED data from POSTED queries, instead of the content-type? For example, I don't think POSTing to a URL like /rest/collection?query makes much sense if you want to store a file - is the ?query ignored (then it could instead be used as indicator that this is NOT a request to store data), or should it get part of the filename (which would be quite surprising). The only problem I see is that POST a <run>...</run> to ?query or ?command does not make much sense, IMHO that should be considered an invalid request. I attached a completely untested draft patch of what I meant. But then, I do not know the reasons for why the interface looks the way it does, so maybe my suggestion is completely against your intentions ;-)
Kind regards, Ralf
On Thursday 22 September 2011 20:48:54 you wrote:
Is that a regression?
Whoops.. no, that was a bug, which is now fixed.
Sorry, Christian
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Johannes.Lichtenberger -
Ralf Jung