The Zorba HTTP spec is very interesting. I think the original EXPath HTTP spec (and others) may have taken a more map-and-array form had those data structures been available in XQuery at the time. It would be interesting to reconsider some of these specs in a future major revision to them.

On Thu, Jul 12, 2018 at 4:33 PM Christian Grün <christian.gruen@gmail.com> wrote:

Hi Bogdan,

> I have one doubt why when we don't define body in http request third parameter is ignored ?

It was defined like this in the EXPath spec [1]:

“When a body element has an empty content (i.e. it has no child node
at all) its content is given by the parameter $bodies. In a single
part request, this param must have at most one item. If the body is
empty, the param cannot be the empty sequence. In a multipart request,
$bodies must have as many items as there are empty body elements. If
there are three empty body elements, the content of the first of them
is $bodies[1], and so on. The number of empty body elements must be
equal to the number of items in $bodies.”

I guess it would have been more user-friendly if an error was raised
if these requirements are not met by the function call.

Anyway, as indicated, I’d be happy if we had functions with
unambiguous function arguments in the future. The Zorba HTTP Client
Module could serve as inspiration [2]. I have never tried it in
practice, but it looks pretty well-designed (an additional options
argument should probably added to the simple functions).

Best,
Christian

[1] http://expath.org/spec/http-client#d2e362
[2] http://www.zorba.io/documentation/3.0/modules/zorba/io/http-client/



> Only when body element is provided parseBody with bodies is invoked.
> I have to check source code to figure out how to pass body payload via third argument.
>
> if(body != null) {
>         final QNm pl = body.qname();
>         // single part request
>         if(pl.eq(Q_BODY)) {
>           parseBody(body, bodies, hr.payloadAtts, hr.payload);
>           hr.isMultipart = false;
>           // multipart request
>         } else if(pl.eq(Q_MULTIPART)) {
>           parseMultipart(body, bodies.iter(), hr.payloadAtts, hr.parts);
>           hr.isMultipart = true;
>         } else {
>           throw HC_REQ_X.get(info, "Unknown payload element: " + body.qname());
>         }
> }
>
> Best Regards
> Bogdan Bogucki
>
>
> -----Wiadomość oryginalna-----
> Od: Christian Grün [mailto:christian.gruen@gmail.com]
> Wysłano: 12 lipca 2018 15:36
> Do: Bogdan Bogucki
> DW: BaseX
> Temat: Re: [basex-talk] http:send-request - problem with form data
>
> Hi Bogdan,
>
> I finally had some time to get to the bottom of the conversion inconsistencies:
>
> If the value is supplied as child of the http:body node, it will be serialized as text node. If it’s supplied as extra parameter, it will be serialized as string. If text nodes are serialized, entities will be encoded. Strings will be returned without encoded entities:
>
>   <a><![CDATA[&]]></a>   →   <a>&amp;</a>
>   ``[&]``   →   &
>
> I agree this is not very intuitive, and I have now rewritten this as
> follows: If the supplied output media-type does not result in one of the default serialization methods of the spec (xml, xhtml, html, json, text, adaptive), the text node will be atomized (i.e., converted to an item of type xs:untypedAtomic), which will then be serialized as plain string. With the new snapshot [1], it should make no difference if you supply your value as child of http:body or as extra argument.
>
> As a general guide, I would recommended everyone to use the third argument for body items. It will be both safer and faster (in particular if binary data can be streamed). As the rich syntax of http:send-request is regularly confusing users on the list, I have reopened an old Github issue, which is about providing additional and simpler functions for the most widely used HTTP methods [2].
>
> Best,
> Christian
>
> [1] http://files.basex.org/releases/latest/
> [2] https://github.com/BaseXdb/basex/issues/914
>
>
>
> On Wed, Jul 4, 2018 at 10:05 PM Christian Grün <christian.gruen@gmail.com> wrote:
> >
> > And thanks for the confirmation. I will try to find out why the two alternatives make a difference.
> >
> >
> >
> > Bogdan Bogucki <bbogucki@hussar.pl> schrieb am Mi., 4. Juli 2018, 21:56:
> >>
> >> Hello,
> >>
> >> It works with body as a third parameter. Body has to be defined in http request because without that third parameter is ignored.
> >>
> >> Working example:
> >>
> >> for $x in http:send-request(
> >>   <http:request method='post'>
> >>     <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/>
> >>     <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'/>
> >>   </http:request>
> >> ,'http:localhost:3333', 'start=0&amp;length=100&amp;
> >> ')[2]/data/results/content return $x
> >>
> >> Thank you Christian for help.
> >>
> >> Best Regards
> >> Bogdan Bogucki
> >>
> >>
> >> -----Wiadomość oryginalna-----
> >> Od: Christian Grün [mailto:christian.gruen@gmail.com]
> >> Wysłano: 4 lipca 2018 18:31
> >> Do: bbogucki@hussar.pl
> >> DW: BaseX
> >> Temat: Re: [basex-talk] http:send-request - problem with form data
> >>
> >> Hi Bogdan,
> >>
> >> I didn’t try it, but what happens if you specify the body as third argument?
> >>
> >> Cheers,
> >> Christian
> >>
> >>
> >> On Tue, Jul 3, 2018 at 10:14 PM Bogdan Bogucki <bbogucki@hussar.pl> wrote:
> >> >
> >> > Hello,
> >> >
> >> > I don't want pass values in query string. Server side expect parameters in payload.
> >> > HTML Form parameters in POST method are passed in payload not in URI.
> >> > This is the reason why I put parameters in body. Problem is
> >> > escaping & character with &amp; How can I force BaseX to not escape characters in CDATA element ?
> >> >
> >> > This is my code:
> >> >   <http:request method='post' href='http://localhost:3333'>
> >> >     <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"/>
> >> >     <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'>
> >> >     <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]>
> >> >     </http:body>
> >> >   </http:request>
> >> >
> >> > On server side I received following request:
> >> >
> >> > POST / HTTP/1.1
> >> > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64)
> >> > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99
> >> > Safari/537.36
> >> > Content-Type: application/x-www-form-urlencoded; charset=UTF-8
> >> >
> >> > Host: copart.com:3333
> >> >
> >> > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
> >> >
> >> > Connection: keep-alive
> >> > Content-Length: 75
> >> >
> >> > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100
> >> >
> >> > I expect that CDATA will be not escaped, or I am wrong ?
> >> >
> >> >
> >> > Pozdrawiam
> >> > Bogdan Bogucki
> >> >
> >> > HUSSAR SYSTEMS
> >> >
> >> > NIP: 532-170-12-00
> >> > Regon: 142624070
> >> >
> >> > e-mail: bbogucki@hussar.pl
> >> > tel. kom.: +48 607 409 301
> >> > www: www.hussar.pl
> >> >
> >> > -----Wiadomość oryginalna-----
> >> > Od: Christian Grün [mailto:christian.gruen@gmail.com]
> >> > Wysłano: 3 lipca 2018 12:11
> >> > Do: bbogucki@hussar.pl
> >> > DW: BaseX
> >> > Temat: Re: [basex-talk] http:send-request - problem with form data
> >> >
> >> > Hi Bogdan,
> >> >
> >> > If you want your arguments parsed as HTTP query parameters, it should suffice to attach them to the URL:
> >> >
> >> >   let $url := 'http://localhost:3333'
> >> >   let $params := ``[start=0&length=100&sort=date_type&page=0&size=100]``
> >> >   return http:send-request(
> >> >     <http:request method='post' href='{ $url || '?' || $params }'> ...
> >> >
> >> > If this is not what you need, you might need to give us more information on how the request will be handled server-side (for example, do you use RESTXQ?).
> >> >
> >> > Best,
> >> > Christian
> >> >
> >> >
> >> >
> >> >
> >> > On Mon, Jul 2, 2018 at 11:11 PM Bogdan Bogucki <bbogucki@hussar.pl> wrote:
> >> > >
> >> > > Hello,
> >> > >
> >> > >
> >> > >
> >> > > I have problem with http:send-request. I want to send post method request with form data but I encountered problem with characters encoding.
> >> > >
> >> > > Example:
> >> > >
> >> > >
> >> > >
> >> > > Request:
> >> > >
> >> > > for $x in http:send-request(
> >> > >
> >> > >   <http:request method='post' href=
> >> > >
> >> > > 'http://localhost:3333'>
> >> > >
> >> > >     <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT
> >> > > 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
> >> > > Chrome/67.0.3396.99 Safari/537.36"/>
> >> > >
> >> > >     <http:body media-type='application/x-www-form-urlencoded;
> >> > > charset=UTF-8'>
> >> > >
> >> > >     <![CDATA[start=0&length=100&sort=date_type&page=0&size=100]]>
> >> > >
> >> > >     </http:body>
> >> > >
> >> > >   </http:request>
> >> > >
> >> > > )[2]/json/data/results/content
> >> > >
> >> > > return $x
> >> > >
> >> > >
> >> > >
> >> > > Optimized Query:
> >> > >
> >> > > util:item-at(http:send-request(element Q{http://expath.org/ns/http-client}request { (attribute method { ("post") }, attribute href { ("http://localhost:3333") }, element Q{http://expath.org/ns/http-client}header { (attribute name { ("User-Agent") }, attribute value { ("Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36") }) }, element Q{http://expath.org/ns/http-client}body { (attribute media-type { ("application/x-www-form-urlencoded; charset=UTF-8") }, "&#xA;    start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100&#xA;    ") }) }), 2)/json/data/results/content
> >> > >
> >> > >
> >> > >
> >> > > Request send:
> >> > >
> >> > > POST / HTTP/1.1
> >> > >
> >> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64)
> >> > > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99
> >> > > Safari/537.36
> >> > >
> >> > > Content-Type: application/x-www-form-urlencoded; charset=UTF-8
> >> > >
> >> > > Host: copart.com:3333
> >> > >
> >> > > Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
> >> > >
> >> > > Connection: keep-alive
> >> > >
> >> > > Content-Length: 75
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > start=0&amp;length=100&amp;sort=date_type&amp;page=0&amp;size=100
> >> > >
> >> > >
> >> > >
> >> > > Payload is not correctly encoded it should be:
> >> > >
> >> > > start=0&length=100&sort=date_type&page=0&size=100
> >> > >
> >> > >
> >> > >
> >> > > How can I force function http:send-request to serialize CDATA without escaping ?
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Best Regards
> >> > >
> >> > > Bogdan Bogucki
> >> > >
> >> > >
> >> >
> >>
>

--

Sent from my iPhone