Follow up: I wrote a function to construct the bundle-to-docs index as an element. That function, operating over the previously-construct doc-to-bundle index document, takes 0.2 seconds to run!

So it seems like the answer is to build the index you need when you need it (and then persist or not depending on how dynamic your data is) rather than trying to do a relational-style lookup against the first document.

This is the function that builds the index. It doesn’t do any lookups, just iterates over the entries in the doc-to-bundle index, which is very fast:

declare function linkrk:constructBundleToDocsIndex($database as xs:string) as element(bundle-to-docs-index) {

  let $lrcDatabase := linkrk:getRecordKeepingDbName($database)

  let $dtbIndex := collection($lrcDatabase)/doc-to-bundle-index

  let $bundlesToIndexKey as map(*) := map:merge(

    for $entry in $dtbIndex/doc-to-bundle-index-entry

    let $bundles as xs:string* := $entry/bundles/bundle ! string(.)

    for $bundle in $bundles

    return map{ $bundle : string($entry/@key)}

    , map{'duplicates' : 'combine'}

  )

  let $index as element(bundle-to-docs-index) :=

  element{'bundle-to-docs-index'} {

    for $bundle in map:keys($bundlesToIndexKey)

    return

    element {'bundle-to-docs-index-entry'} {

      attribute{'bundle'}{$bundle},

      for $key in $bundlesToIndexKey($bundle)

      return element{'doc-key'}{$key}

    }

  }

  return $index

};

_____________________________________________

Eliot Kimber

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com

LinkedIn | Twitter | YouTube | Facebook

From: Christian Grün <christian.gruen@gmail.com>
Date: Thursday, February 3, 2022 at 8:11 AM
To: Eliot Kimber <eliot.kimber@servicenow.com>
Cc: basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Optimizing Lookup from Custom Indexes

[External Email]


>     for $key in $keysForDocs
>     return $dtbIndex/doc-to-bundle-index-entry[@key eq $key]/bundles/bundle ! string(.)

You can probably save time by omitting the loop:

  $dtbIndex/doc-to-bundle-index-entry
    [@key = $keysForDocs]/bundles/bundle ! string(.)

Did you check if $dtbIndex is inlined at compile time?