On my mac running 10.7, I get about 1.5ms per query.

On my production sever, I get values like 0.08ms per query.

So there must be something going on with my CSV content or how I’m querying that is leading to the much slower lookup times…

Cheers,

E

_____________________________________________

Eliot Kimber

Sr Staff Content Engineer

Digital Content & Design

O: 512 554 9368

M: 512 554 9368

servicenow.com

LinkedIn | Twitter | YouTube | Facebook

From: Christian Grün <christian.gruen@gmail.com>
Date: Tuesday, July 16, 2024 at 11:16 AM
To: Eliot Kimber <eliot.kimber@servicenow.com>
Cc: basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.de>
Subject: Re: [basex-talk] Query optimization: What can I check or measure?

[External Email]

---

I tried to create some code that we can use for joint testing. With the following snippet, you can create a database (sized around 400 MB) with 1 million value attributes and around 300.0000 distinct values:

db:create(
  'test',
  <xml>{
    for $i in 1 to 1000000
    let $value := codepoints-to-string(
      random:seeded-integer($i mod 300000, 256, 26) ! (. + 97)
    )
    return <item value='{ $value }'/>
  }</xml>,
  'test.xml',
  map { 'maxlen': 256, 'tokenindex': true() }
)

The following query chooses a random entry and returns all elements that contain the attribute with this value:

let $count := count(index:tokens('test'))
let $pos := (abs(random:integer()) mod $count) + 1
return db:token('test', index:tokens('test')[$pos])

The second query takes around 3 ms in my tests. Do you get similar performance?