Christian,
Thanks for your prompt reply.
I had tried db:attribute for quicker look-up but the bottle-neck for me is the usage part and not the retrieval part. Retrieval is quick but good to know that it has been taken care of in latest stable version of BaseX.
There are several joins due to complexity of data happening in the query that use this data coming from foo:get-node-v1(all the three parameters that are passed to this function are small objects.) To give you a quick overview of the query I have:
: : let $aNodes := foo:get-a($context) let $bNodes := foo:get-b($context) let $scopeValues := foo:get-node-v1($context, $project, $scopeName) for $aNode in $aNodes for $bNode in $bNodes[<predicate>] : : let $val := $foo:another-function($scopeValues, $bNode)
foo:another-function is basically filtering $scopeValues relevant to $bNode. We have a high number of $aNodes and $bNodes.
If I use the second variation wherein the data content is exactly same (only difference is that is local copy now) then my xquery performs way better.
Is there a way to get a local copy of the database node so that we do not reference the node in the database for further usage in the xquery (any other cleaner way instead of iterating over the data and creating a local copy that foo:get-node-v2 is doing)?
Thanks, Teena
On Thu, Dec 12, 2013 at 12:24 PM, Christian Grün christian.gruen@gmail.comwrote:
Hi Teena (cc @basex-talk),
first of all, thanks for giving us details on your data and queries. To complete this: do you have a complete version of your query that shows us what are the values of $context, $project and $scopeName? As an example, I’m wondering what are the ancestor nodes of the <scopes> elements.
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { if (exists($project) = true()) then ( $project/scopes/scope[@name=$scopeName]/item ) else () };
The following query is equivalent, by the way (this is the beauty of XPath):
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { { $project/scopes/scope[@name=$scopeName]/item };
If all your data is stored in the same database, it may be faster to open the database within the function. One more optimization could be to directly access the text index:
declare function foo:get-node-v1($context as element()?, $project as element()?, $scopeName as xs:string) as element()? { { db:attribute("name-of-your-db", $scopeName, "name")/parent::scope };
You could also check out the latest stable snapshot of BaseX 7.8 [1], which inlines functions and may thus do this automatically.
When we call the first function in another function and use it for further computation, it is always slower by 40 seconds (when compared to second variation of the function). What could be causing this?
Difficult to say; we probably need to see your complete function call.
Are we still referencing to baseX document when we do following instead of getting a local copy? let $items := foo:get-node-v1($context, $project, $scopeName)
Yes, the returned nodes still reference nodes in the database.
Hope this helps, Christian