Hi Bram, hi Christian,

I am surprised, I would have expected you to point to the upcoming websocket implementation (see https://github.com/BaseXdb/basex/issues/1449)? To me the scenario described by Bram seems like a perfect fit for websockets, where you simply subscribe to such a generator and the server then delivers the results by notifying every subscriber.

Cheers Dirk

Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht Frankfurt am Main - Reg.-Nr.: HRB 105546 Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: Daniel Grözinger

On 19. Apr 2018, at 23:01, Christian Grün <christian.gruen@gmail.commailto:christian.gruen@gmail.com> wrote:

Hi Bram,

There is no comparable feature in BaseX right now. But maybe it’s already helpful for you to know that all server-side results will always be streamed to the output stream of the HTTP response. As a result, it can easily happen that the client has received the first 10 results while the server is still evaluating your query.

If you want to have more control, you can define a result window via variables (e.g. $start and $length). Usually, if your query succeeds in quickly delivering the result 1-10, it shouldn’t take much longer to deliver result 11-20, 21-30, etc., so you can send multiple iterative requests to the server to receive the subsequent results.

Best, Christian

On Thu, Apr 19, 2018 at 11:02 AM, Bram Vanroy <Bram.Vanroy@ugent.bemailto:Bram.Vanroy@ugent.be> wrote: Dear list

I am curious to find out whether it is possible to stream results of a query, i.e. where a submitted query behaves as a generator. Especially for web interfaces this is useful: a user is presented with a result as soon as it is found, and doesn’t have to wait for all results to be gathered. Especially useful with large databases of course.

Preferably a way that works with the nodejs, PHP client but I am open to other suggestions.

Thanks in advance

Bram

Hi Bram,

I can see the feature implementation is tagged as 9.1. Any idea when this version will roll-out? Or is an implementation already available in the latest snapshott?

We are still investigating what are the best options to integrate WebSockets in BaseX, as we mostly want to avoid building a solution that cannot be properly enhanced once new feature requests are coming in from our users.

To be more specific: We’ll probably build our solution on top of the Jetty WebSocket Server API. While this will give us not as much freedom as when implementing WebSockets completely on our own, and while creating a new Jetty dependency (it won’t work with Tomcat, etc.), we believe it frees us from the need to take care of all minor details, and it allows us to benefit from various optimizations for concurrent requests and heavy traffic that will be available out of the box.

One drawback is that the Jetty servlet approach might hinder us from including support for the 9.0 permissions layer [1], but this is still work in progress. Just follow our Github issue to stay up-to-date.

By the way, this is a good opportunity to thank

Nubisware and Marco Lettere

for sponsoring parts of this new feature! If anyone out there is interested in speeding up the implementation process, we invite you to get connected with us.

Best, Christian

[1] http://docs.basex.org/wiki/Permissions [2] https://github.com/BaseXdb/basex/issues/1449#issuecomment-372127950

Van: Kirsten, Dirk Dirk.Kirsten@senacor.com Verzonden: vrijdag 20 april 2018 9:13 Aan: Christian Grün christian.gruen@gmail.com CC: Bram Vanroy Bram.Vanroy@UGent.be; BaseX basex-talk@mailman.uni-konstanz.de Onderwerp: Re: [basex-talk] Streaming results

Hi Bram, hi Christian,

I am surprised, I would have expected you to point to the upcoming websocket implementation (see https://github.com/BaseXdb/basex/issues/1449)?

To me the scenario described by Bram seems like a perfect fit for websockets, where you simply subscribe to such a generator and the server then delivers the results by notifying every subscriber.

Cheers

Dirk

Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht Frankfurt am Main - Reg.-Nr.: HRB 105546 Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: Daniel Grözinger

On 19. Apr 2018, at 23:01, Christian Grün christian.gruen@gmail.com wrote:

Hi Bram,

There is no comparable feature in BaseX right now. But maybe it’s already helpful for you to know that all server-side results will always be streamed to the output stream of the HTTP response. As a result, it can easily happen that the client has received the first 10 results while the server is still evaluating your query.

If you want to have more control, you can define a result window via variables (e.g. $start and $length). Usually, if your query succeeds in quickly delivering the result 1-10, it shouldn’t take much longer to deliver result 11-20, 21-30, etc., so you can send multiple iterative requests to the server to receive the subsequent results.

Best, Christian

On Thu, Apr 19, 2018 at 11:02 AM, Bram Vanroy Bram.Vanroy@ugent.be wrote:

Dear list

I am curious to find out whether it is possible to stream results of a query, i.e. where a submitted query behaves as a generator. Especially for web interfaces this is useful: a user is presented with a result as soon as it is found, and doesn’t have to wait for all results to be gathered. Especially useful with large databases of course.

Preferably a way that works with the nodejs, PHP client but I am open to other suggestions.

Thanks in advance

Bram