Hi Sebastian,

I agree, it is kind of impossible to simplify your query. The reason is function inlining:

Original query:
  declare function local:greet($name) { 'hi ' || $name }; local:greet('john')

Optimized query (intermediate):
  let $name_1 := 'john'
  return 'hi ' || $name_1

Optimized query (final):
  'hi john'

If a function body is too large, it won’t be inlined (otherwise, recursive functions can easily blow up main memory). The inline limit in BaseX is a static value, it can be controlled via the INLINELIMIT option [1].

Coming back to your query, test.xql won’t be optimized for index access because…
1. The function body is too large
2. The passed on JSON string cannot be statically converted to a map
3. As a consequence, it cannot statically be determined if search will be fuzzy or not.
4. As a consequence, the predicate with the full-text expression contains multiple options, and none of them will be rewritten for index access.

You can e.g.b tackle this by defining a larger inline limit for this function in your function declaration (see the attached link). As stated before, you can also call ft:search directly and pass on the fuzzy option as parameter.

A general hint: With BaseX 9.1, your map lookup can be simplified to…

  $map?key2?key3 ?: ("1", "2")

…or…

  util:or($map?key2?key3, ("1", "2"))

If you want to use standard XQuery 3.1 language, you can write:

  let $key := $map?key2?key3
  return if(exists($key)) then $key else ("1", "2")

This will ensure that your map will only be looked up once.

Best,
Christian

[1] http://docs.basex.org/wiki/Options#INLINELIMIT



On Mon, Nov 12, 2018 at 6:11 PM Sebastian Zimmer <sebastian.zimmer@uni-koeln.de> wrote:

Hi Christian,

thanks for your help attempt.  I understand that the query is not really simple yet and that it's hard to see what's going on.

Attached is a database document on which you can run the query.

To your questions:

* It does make no difference if the namespace prefixes are replaced with *. Thus you can simplify the query in that manner and the issue persists.
* Yes, the issue does only occur if the code is within the local:search function. If the code is in the global scope of the file, the ft-index is applied. Thus, I cannot simplify the query that way.
* If I replace the functx function with its body, the ft-index is applied. Thus, I cannot simplify the query that way.

I'm happy to help further if I can.

Best,
Sebastian


Am 12.11.2018 um 17:24 schrieb Christian Grün:
Hi Sebastian,

It’s difficult to guess what the query optimizer does by just looking at the query. Could you pass us on a database document that allows us to run the query?

Any attempt to simplify your query is appreciated. Here are some more things that you could try:
• Does it make a difference if you work with namespaces, or could all prefixes in the location steps be replaced with *?
• Does the issue only occur if your code is defined in the local:search function?
• What happens if you replace the functx function call with its body ($value = $seq)?

Best,
Christian



On Mon, Nov 12, 2018 at 9:54 AM Sebastian Zimmer <sebastian.zimmer@uni-koeln.de> wrote:

Hi,

I'm working on a query where the full-text index is not applied, even though it could be (see attached test.xql).

I'm sorry for the somewhat complicated look of the example with data-specific XPaths and so on. I really have tried to simplify it as much as I could, but when I further simplify it, the fulltext index actually will be applied or the query is optimized to () due to cached evaluations.

But if you take the expression of the variable $number and inline it into below where the variable is referenced, you may see what I mean. The result is an equivalent query (see attached test-ft.xql) to the original one, but now the fulltext index *is* applied.

I'm using the BaseX 9.1 snapshot from 2018-11-05.

Best,
Sebastian

--
Sebastian Zimmer
sebastian.zimmer@uni-koeln.de
CCeH Logo

Cologne Center for eHumanities
DH Center at the University of Cologne
 Twitter Logo@CCeHum


--
Sebastian Zimmer
sebastian.zimmer@uni-koeln.de
CCeH Logo

Cologne Center for eHumanities
DH Center at the University of Cologne
 Twitter Logo@CCeHum