We have successfully implemented the technique described by Christian. We have a REST end point that takes a database and node ID of a DITA element. If an HTML preview of the element is present in our cache ( a file system directory organized by node ID), it is retrieved and returned, otherwise the preview is constructed, cached and returned.
This REST endpoint is called from server-side code that also checks for a cached preview and just returns it (avoiding the overhead of the REST call), otherwise it calls the endpoint.
This reduced the time to construct a preview from 50ms to about 17ms to fetch the cached preview. We initially used a database for the cache but that didn’t not offer any significant time savings, so we realized using the file system was better. We had no other reason to need a database for the cached previews.
We are no now exploring upping our JavaScript game to use the REST API from the browser, treating our table previews like the browser treats images, fetching them asynchronously on page load.
The main complexity in our implementation is we had to enable the non-use of the cache in the tree walk code that makes the previews (it’s a simple XQuery tree walk of the incoming XML to make the HTML).
Cheers,
E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow
From: Christian Grün christian.gruen@gmail.com Date: Saturday, July 15, 2023 at 9:04 AM To: Eliot Kimber eliot.kimber@servicenow.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] How best to cache an intermediate result in the context of a larger query? [External Email]
Hi Eliot,
The following RESTXQ function demonstrates how you can cache RESTXQ results in the store:
declare %rest:path('cache/{$a}') function local:convert($a) { store:get-or-put($a, function() { <result ts='{ current-dateTime() }'>{ $a * $a }</result> }) };
If the endpoint is called for the first time, the function argument is executed and the result is returned & cached in the store. If the endpoint is called a second time, the cached value is returned (you’ll see this by reading the timestamp).
Here’s one solution for storing cached results in a secondary database:
declare %updating %rest:path('cache/{$name}') function local:convert($name) { if(db:exists('db-cache', $name)) then ( update:output(db:get('db-cache', $name)) ) else ( let $updated := db:get('db', $name) update { insert node attribute ts { current-dateTime() } into * } return ( db:put('db-cache', $updated, $name), update:output($updated) ) ) };
Hope this helps, Christian
On Thu, Jul 13, 2023 at 9:44 PM Eliot Kimber eliot.kimber@servicenow.com wrote:
In the context of a RESTXQ handler that is processing a stored document to generate HTML From it, I need to do some expensive processing and then cache the result for the next time the same document is rendered. It doesn’t make sense to preprocess all the documents to cache the data as part of my overall ingestion process because only a fraction of documents will ever be rendered by this page.
I think the answer is to turn on MIXUPDATES so I can invoke an updating function in the process of the larger RESTXQ handling query, but I’m wondering if there’s some less drastic or better-architected approach? For example, should be I be invoking a separate REST end point to do the calculation, blocking on the request, and then fetch the cached result?
I read through the update module and the various pages on XQuery update and feel like I still don’t quite understand what my updating options are.
Thanks,
Eliot
Eliot Kimber
Sr Staff Content Engineer
O: 512 554 9368
M: 512 554 9368
servicenow.com
LinkedIn | Twitter | YouTube | Facebook