Hi,
I am working on some REST API code and I'm struggling with POSTing of JSON.
What I want to achieve is to have the function receive an XDM map (from JSON using fn:parse-json). I also want to output from such a map (in effect roundtripping some JSON).
The test POST body I'm using is:
{"foo": "bar"}
When reading the docs I'm led to believe that this should achieve it:
declare %rest:POST("{$body}") %rest:path("/json") %rest:consumes("application/json;lax=yes;format=map") %rest:produces("application/json;format=map") function api:json($body) { $body };
No matter what I do I always get this as response:
<json type="object"> <foo>bar</foo> </json>
Somehow I cannot force the function to receive a map(*) as $body argument. When I wrap $body in a trace() I see that it gets XML and not a map(*).
What am I missing? Or do I really have to go JSON > XML > map {....}?
Hi Marc,
declare %rest:POST("{$body}") %rest:path("/json") %rest:consumes("application/json;lax=yes;format=map") %rest:produces("application/json;format=map")
If I remember right, content-type parameters in rest annotations will be ignored, because these annotations only serve as filters. Instead, you will need to specify the content-type parameters client-side (in your POST or PUT request).
Hope this helps, Christian
Hi Christian,
No luck.
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map (btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
I'm seeing 400 errors like this:
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
So something is different now but still there seems to be an expectation somewhere on the post being XML at least the filename suggests so.
The documentation is a bit contradictory:
"The body of a POST or PUT request will be converted to an XQuery item. Conversion can be controlled by specifying a content type. It can be further influenced by specifying additional content-type parameters:"
The table below lists:
application/jsonJSON Optionsdocument-node()
On Mon, Aug 3, 2015 at 4:42 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
declare %rest:POST("{$body}") %rest:path("/json") %rest:consumes("application/json;lax=yes;format=map") %rest:produces("application/json;format=map")If I remember right, content-type parameters in rest annotations will be ignored, because these annotations only serve as filters. Instead, you will need to specify the content-type parameters client-side (in your POST or PUT request).
Hope this helps, Christian
Wow, clicked send before I meant to. The last line was a copy from the table in the documentation and the last column has "document-node()" which seems to suggest that getting a map(*) there isn't possible. So not sure what is supposed to happen.
Anyway I would think that correct behaviour would be, I think, to deliver a map(*) to the function instead of a document-node() in this case.
Maybe something to forward to Adam Retter / RESTXQ?
--Marc
On Mon, Aug 3, 2015 at 4:54 PM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
No luck.
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map (btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
I'm seeing 400 errors like this:
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
So something is different now but still there seems to be an expectation somewhere on the post being XML at least the filename suggests so.
The documentation is a bit contradictory:
"The body of a POST or PUT request will be converted to an XQueryitem. Conversion can be controlled by specifying a content type. It can be further influenced by specifying additional content-type parameters:"
The table below lists:
application/jsonJSON Optionsdocument-node()
On Mon, Aug 3, 2015 at 4:42 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
declare %rest:POST("{$body}") %rest:path("/json") %rest:consumes("application/json;lax=yes;format=map") %rest:produces("application/json;format=map")If I remember right, content-type parameters in rest annotations will be ignored, because these annotations only serve as filters. Instead, you will need to specify the content-type parameters client-side (in your POST or PUT request).
Hope this helps, Christian
-- --Marc
Hi Marc,
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map
I noticed that you get different result when specify 'format=basic' or 'format=direct':
curl -XPUT -H"Content-Type:application/json;format=basic" -Tinput.json "http://localhost:8984/test"
..so I guess something is going wrong here. I'll dive into this soon.
(btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
You can also use %input:json(...) if you don't want to specify the format client-side.
Maybe something to forward to Adam Retter / RESTXQ?
So far, the content-type parameter extensions have not been adopted in RESTXQ yet, but feel free to give him an update!
Christian
…fixed, and available via the latest snapshot [1].
Thanks, Christian
[1] http://files.basex.org/releases/latest
On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map
I noticed that you get different result when specify 'format=basic' or 'format=direct':
curl -XPUT -H"Content-Type:application/json;format=basic" -Tinput.json "http://localhost:8984/test"
..so I guess something is going wrong here. I'll dive into this soon.
(btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
You can also use %input:json(...) if you don't want to specify the format client-side.
Maybe something to forward to Adam Retter / RESTXQ?
So far, the content-type parameter extensions have not been adopted in RESTXQ yet, but feel free to give him an update!
Christian
Hi Christian,
Super-fast as always ;-)
I'm still struggling though. Couple of points
I have this RESTXQ function that I'm trying to get working.
declare %rest:POST("{$query}") %rest:path("/json") %rest:consumes("application/json") %rest:produces("application/json") %input:json("format=map") function api:json($query) { $query };
I'm posting this as the body (and have Content-Type=application/json)
{"foo": "bar"}
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
What I want to achieve is not go through JSON > XML parsing but get the same effect as using fn:parse-json() on the incoming JSON and fn:serialize(..., map { 'method': JSON }) on the output.
I can imagine it's the RESTXQ layer itself that preps the args that insists on turning the JSON into XML. Maybe because it hasn't been revamped to fit XQuery 3.1 where we can have native arrays and maps.
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
--Marc
On Mon, Aug 3, 2015 at 6:47 PM, Christian Grün christian.gruen@gmail.com wrote:
…fixed, and available via the latest snapshot [1].
Thanks, Christian
[1] http://files.basex.org/releases/latest
On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map
I noticed that you get different result when specify 'format=basic' or 'format=direct':
curl -XPUT -H"Content-Type:application/json;format=basic" -Tinput.json "http://localhost:8984/test"
..so I guess something is going wrong here. I'll dive into this soon.
(btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
You can also use %input:json(...) if you don't want to specify the format client-side.
Maybe something to forward to Adam Retter / RESTXQ?
So far, the content-type parameter extensions have not been adopted in RESTXQ yet, but feel free to give him an update!
Christian
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server. Isn't this a case of "The server knows best". As a client I just specify that this is JSON and leave it up to the server how to process. If I do that and put an %input:json(...) annotation on the RESTXQ handler it can still be overridden by things specified in the header by the client. But again this probably should be directed at RESTXQ rather than BaseX.
--Marc
On Tue, Aug 4, 2015 at 10:01 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
Super-fast as always ;-)
I'm still struggling though. Couple of points
I have this RESTXQ function that I'm trying to get working.
declare %rest:POST("{$query}") %rest:path("/json") %rest:consumes("application/json") %rest:produces("application/json") %input:json("format=map") function api:json($query) { $query };I'm posting this as the body (and have Content-Type=application/json)
{"foo": "bar"}I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
What I want to achieve is not go through JSON > XML parsing but get the same effect as using fn:parse-json() on the incoming JSON and fn:serialize(..., map { 'method': JSON }) on the output.
I can imagine it's the RESTXQ layer itself that preps the args that insists on turning the JSON into XML. Maybe because it hasn't been revamped to fit XQuery 3.1 where we can have native arrays and maps.
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
--Marc
On Mon, Aug 3, 2015 at 6:47 PM, Christian Grün christian.gruen@gmail.com wrote:
…fixed, and available via the latest snapshot [1].
Thanks, Christian
[1] http://files.basex.org/releases/latest
On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map
I noticed that you get different result when specify 'format=basic' or 'format=direct':
curl -XPUT -H"Content-Type:application/json;format=basic" -Tinput.json "http://localhost:8984/test"
..so I guess something is going wrong here. I'll dive into this soon.
(btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
You can also use %input:json(...) if you don't want to specify the format client-side.
Maybe something to forward to Adam Retter / RESTXQ?
So far, the content-type parameter extensions have not been adopted in RESTXQ yet, but feel free to give him an update!
Christian
-- --Marc
I got it working using this function. One requirement though is that the client must send the JSON with a text content type or no content-type. This way the function receives a string argument and I can do with the string what I want. Not 100% satisfied because the client has to lie about it's content type in order to get this to work.
declare %rest:POST("{$query}") %rest:path("/json") %rest:produces("application/json") function api:json($query) { rest:response output:serialization-parameters <output:media-type value="application/json"/> </output:serialization-parameters> </rest:response>, serialize(parse-json($query), map { 'method': 'json'}) };
BTW some background: I'm trying to implement a pattern for sending query specification via REST (this allows a much better query language instead of forcing everything through query params). Elastic Search does this too. My first instinct was to use a body with GET but, although the HTTP specs more or less allow it many tools and libraries (including BaseX/RESTXQ) do not support sending bodies with GET. So now I'm doing this with POST which feels wrong too.
--Marc
On Tue, Aug 4, 2015 at 10:16 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server. Isn't this a case of "The server knows best". As a client I just specify that this is JSON and leave it up to the server how to process. If I do that and put an %input:json(...) annotation on the RESTXQ handler it can still be overridden by things specified in the header by the client. But again this probably should be directed at RESTXQ rather than BaseX.
--Marc
On Tue, Aug 4, 2015 at 10:01 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
Super-fast as always ;-)
I'm still struggling though. Couple of points
I have this RESTXQ function that I'm trying to get working.
declare %rest:POST("{$query}") %rest:path("/json") %rest:consumes("application/json") %rest:produces("application/json") %input:json("format=map") function api:json($query) { $query };I'm posting this as the body (and have Content-Type=application/json)
{"foo": "bar"}I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
What I want to achieve is not go through JSON > XML parsing but get the same effect as using fn:parse-json() on the incoming JSON and fn:serialize(..., map { 'method': JSON }) on the output.
I can imagine it's the RESTXQ layer itself that preps the args that insists on turning the JSON into XML. Maybe because it hasn't been revamped to fit XQuery 3.1 where we can have native arrays and maps.
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
--Marc
On Mon, Aug 3, 2015 at 6:47 PM, Christian Grün christian.gruen@gmail.com wrote:
…fixed, and available via the latest snapshot [1].
Thanks, Christian
[1] http://files.basex.org/releases/latest
On Mon, Aug 3, 2015 at 5:11 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I'm using Postman. I already had Content-Type = application/json and Accept = application/json. I tried changing it to application/json;format=map
I noticed that you get different result when specify 'format=basic' or 'format=direct':
curl -XPUT -H"Content-Type:application/json;format=basic" -Tinput.json "http://localhost:8984/test"
..so I guess something is going wrong here. I'll dive into this soon.
(btw do you think it's necessary to have the basex/restxq specific pseudo attributes (format= etc) in the HTTP request headers?)
You can also use %input:json(...) if you don't want to specify the format client-side.
Maybe something to forward to Adam Retter / RESTXQ?
So far, the content-type parameter extensions have not been adopted in RESTXQ yet, but feel free to give him an update!
Christian
-- --Marc
-- --Marc
I got it working using this function. One requirement though is that the client must send the JSON with a text content type or no content-type. This way the function receives a string argument and I can do with the string what I want.
So you mean that your clients will always send JSON as text/plain or without content type? In that case, you can indeed use json:parse($input).
Not 100% satisfied because the client has to lie about it's content type in order to get this to work.
I need to ask for more details: Do you want to say that you are bothered by the implicit JSON conversion once the user specifies the content-type? Would you like to have disabled this conversion?
although the HTTP specs more or less allow it many tools and libraries (including BaseX/RESTXQ) do not support sending bodies with GET. So now I'm doing this with POST which feels wrong too.
In that case, Elasticsearch seems to have deviated from Roy Field's initial suggestions, which seem to also have been adopted by JAX-RS and, consistently, RESTXQ (see e.g. [1]).
[1] https://groups.yahoo.com/neo/groups/rest-discuss/conversations/messages/9962
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
All existing test cases seem to work, and I didn't come across this in my simple queries. Could you please provide me with a function and call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
-- --Marc
This looks a bit like errors I have seen with CACHERESTXQ=true [1] and then hot swapping the XQuery files. Requesting /.init sorts it for me. /Andy [1] http://docs.basex.org/wiki/Options#CACHERESTXQ
On 4 August 2015 at 10:05, Christian Grün christian.gruen@gmail.com wrote:
All existing test cases seem to work, and I didn't come across this in my simple queries. Could you please provide me with a function and call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:370) at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta
7f8299f.
Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
-- --Marc
Thanks, came to that conclusion too, strange things is that CACHERESTXQ=true never bit me before. First time I have issues with it. I have it set to true almost all the time.
--Marc
On Tue, Aug 4, 2015 at 11:24 AM, Andy Bunce bunce.andy@gmail.com wrote:
This looks a bit like errors I have seen with CACHERESTXQ=true [1] and then hot swapping the XQuery files. Requesting /.init sorts it for me. /Andy [1] http://docs.basex.org/wiki/Options#CACHERESTXQ
On 4 August 2015 at 10:05, Christian Grün christian.gruen@gmail.com wrote:
All existing test cases seem to work, and I didn't come across this in my simple queries. Could you please provide me with a function and call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
-- --Marc
Hi Christian,
I figured it out. The way to repro is with the function I posted earlier. Start the server but with CACHERESTXQ=true. Test the function: works fine. Change the handler function/file and save it. Retry the POST, boom. NPE. Switching CACHERESTXQ=false no fixes this.
About the conversion. I'm fine with the default being parse into XML as long as i can fix it to parse as array/map using a handler annotation (and in a way that the client cannot mess with this). No need to change defaults.
Maybe the wrong forum here but that REST discussion:
Server semantics for GET, however, are restricted such that a body, if any, has no semantic meaning to the request. The requirements on parsing are separate from the requirements on method semantics.
What is that supposed to mean (pun intended)? If with semantics they mean HTTP verb semantics. Eg. GET is idempotent etc. then my use case where the body is simply an easier way to specify a query to the server and thus fully compatible with GET semantics. I see no difference with providing params via the URL string or path. Somehow in the past somebody (not sure if it was Fielding himself, maybe the people from the HTTP spec) decided body should not be part of a GET and all that needs to be done with GET semantics can be done with query strings. Now I'm forced to use POST which HTTP semantics are create a new resource. To me this seems worse then using GET and most implementers have chosen to conflate specs with "morals" (the REST morals). I already found out that going for GET is useless as the road is riddled with tools and libraries that forbid you from doing so. And also ES had to allow the same queries to be POSTed instead of provided with GET. Maybe I get crucified for saying this but I think we should allow GET with body. And yes, it's very useful and can simplify clients that need to specify complex queries to a REST server. I stop ranting now, I probably should take this somewhere else but I'm still building up courage to do so ;-)
On Tue, Aug 4, 2015 at 11:05 AM, Christian Grün christian.gruen@gmail.com wrote:
All existing test cases seem to work, and I didn't come across this in my simple queries. Could you please provide me with a function and call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
-- --Marc
I figured it out. The way to repro is with the function I posted earlier. Start the server but with CACHERESTXQ=true. Test the function: works fine. Change the handler function/file and save it. Retry the POST, boom. NPE. Switching CACHERESTXQ=false no fixes this.
I see; so it seems to be a CACHERESTXQ bug, right? (Andy: thanks as well).
Maybe I get crucified for saying this but I think we should allow GET with body.
Maybe even Roy agrees with you today, but we should probably have intervened and fixed that 20 years ago ;) We could obviously weaken the RESTXQ constraints, but it may be advisable to first check out if there has been a similar discussion on JAX-RS.
And yes, it's very useful and can simplify
clients that need to specify complex queries to a REST server. I stop ranting now, I probably should take this somewhere else but I'm still building up courage to do so ;-)
On Tue, Aug 4, 2015 at 11:05 AM, Christian Grün christian.gruen@gmail.com wrote:
All existing test cases seem to work, and I didn't come across this in my simple queries. Could you please provide me with a function and call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
I can get this to work with %input:json format=direct/format=basic but when I change to format=map I still get
[bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
A second point regarding format=map. Not sure if this is the correct name as I could be posting "[1,2,3]" which is valid JSON. Does this mean there should be a format=array or would it be better to give a different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
A last point I would like to make is that I still find it dubious that the client can specify things in a header (Content Type) that override the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
-- --Marc
-- --Marc
Yes, I think it's a CACHERESTXQ bug. The handler I posted earlier triggers it. Not sure what exactly triggers it.
Yes, if only we had a time machine. Kidding aside, REST services/microservices/UI clients/Javascript frameworks all are pushing on REST. I was triggered by recent developments by Facebook/Netflix and recently summarized in a talk on InfoQ called "Demand Driven architecture" (https://qconnewyork.com/ny2015/presentation/demand-driven-architecture). I think that it solves issues for UI clients. I'm looking into which forum to raise the issue or if I should just do the pragmatic thing, use POST, and shut up. I'm a bit scared for "religuous fanatism". ;-)
On Tue, Aug 4, 2015 at 12:01 PM, Christian Grün christian.gruen@gmail.com wrote:
I figured it out. The way to repro is with the function I posted earlier. Start the server but with CACHERESTXQ=true. Test the function: works fine. Change the handler function/file and save it. Retry the POST, boom. NPE. Switching CACHERESTXQ=false no fixes this.
I see; so it seems to be a CACHERESTXQ bug, right? (Andy: thanks as well).
Maybe I get crucified for saying this but I think we should allow GET with body.
Maybe even Roy agrees with you today, but we should probably have intervened and fixed that 20 years ago ;) We could obviously weaken the RESTXQ constraints, but it may be advisable to first check out if there has been a similar discussion on JAX-RS.
And yes, it's very useful and can simplify
clients that need to specify complex queries to a REST server. I stop ranting now, I probably should take this somewhere else but I'm still building up courage to do so ;-)
On Tue, Aug 4, 2015 at 11:05 AM, Christian Grün christian.gruen@gmail.com wrote:
All existing test cases seem to work, and I didn't come across this in my simple queries. Could you please provide me with a function and call that triggers the exception?
On Tue, Aug 4, 2015 at 11:00 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
I've double checked the version and manually downloaded latest. I'm now on BaseX 8.3 beta e13a5f0 which I assume is correct.
Not sure what's going on but in most cases (with format=map) gives me a NPE (and {"foo": "bar"}):
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.3 beta e13a5f0 Java: Oracle Corporation, 1.7.0_25 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.http.restxq.RestXqFunction.parse(RestXqFunction.java:124) at org.basex.http.restxq.RestXqModule.process(RestXqModule.java:100) at org.basex.http.restxq.RestXqFunction.process(RestXqFunction.java:109) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724)
On Tue, Aug 4, 2015 at 10:48 AM, Marc van Grootel marc.van.grootel@gmail.com wrote:
Hi Christian,
I pulled it via the Maven repo using Gradle. It says BaseX 8.3 beta 7f8299f. Maybe that doesn't carry the latest?
Re format name suggestions: format=item (not good, includes XML nodes), format=function (correct per http://www.w3.org/TR/xpath-datamodel-31/#types-representation as it includes array and map) but confusing. format=native (maybe?)
I agree fully that the server should have the last say in how to process the content so I would definitely be in favor of prioritizing server parameters. Not sure if that breaks something out there though.
--Marc
On Tue, Aug 4, 2015 at 10:31 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Marc,
> I can get this to work with %input:json format=direct/format=basic but > when I change to format=map I still get > > [bxerr:BASX0003] Input could not be converted: "POST.xml" (Line 1): No > text allowed before root element.
Hm, it seems to work on my machine. Here is again the minimized version (I also tried the function you sent to me; I removed the %rest:consumes and %rest:produces annotations, because they only serve as filters, and do not influence the conversion of $query):
declare %rest:POST("{$query}") %rest:path("/json") function local:json($query) { $query };
curl -XPOST -H"Content-Type:application/json;format=map" -Tinput.json "http://localhost:8984/json"
Are you sure you tried the latest snapshot?
> A second point regarding format=map. Not sure if this is the correct > name as I could be posting "[1,2,3]" which is valid JSON. Does this > mean there should be a format=array or would it be better to give a > different name for this format.
This is a valid point. It stems from the beginnings of that feature, where the top object was always a map. Do you possibly have a suggestion for a better name?
> A last point I would like to make is that I still find it dubious that > the client can specify things in a header (Content Type) that override > the way it's porcessed on the server.
I think it's basically a good idea to allow a client to specify content-type parameters. But it's true that the server parameter should probably be priorized and overwrite a client parameter, instead of the other way round. Do you agree?
Christian
-- --Marc
-- --Marc
-- --Marc
basex-talk@mailman.uni-konstanz.de
-
Andy Bunce -
Christian Grün -
Marc van Grootel