Hi Menashè,
I am not sure if I can propose any way out, because there are too many factors that would need to be looked at right now (automatically composed queries, no node ids, gigabytes of data, ...).
So let's maybe go back to your original observation:
Once I specific XPath, it seems that the ids have no meaning. The resultset is always empty once they are being used.
Can you give us more details on that? Can you give me a *simple* example that worked, and another example that does not work anymore for you? Did you think about using db:node-pre($db/root()) ?
I attached two simplified versions of your query. How do they perform on your database instance? Maybe we should first try to get these queries fast before looking at more complex examples (even if these are the ones that are composed in practice).
Christian
On Mon, Jun 22, 2015 at 4:48 PM, Menashè Eliezer meliezer@ogs.trieste.it wrote:
Hi Christian, I'm have again performance problems. I have BaseX 8.2.1. As you may remember, you've recommended changing 'for $x in collection("CDI")' to 'for $x in collection("CDI")/gmd:MD_Metadata/gmd:identificationInfo/sdn:SDN_DataIdentification'. However, I've discovered I cannot specify XPath while working with IDs (db:node-pre). It's a multi-step process: client program sends to the server the search filter defined by end-user and get IDs. Then there are several queries for getting different information about this specific subset. Instead of redefining the filters, the only condition is where db:node-pre($x)=$ids for having a better performance. Once I specific XPath, it seems that the ids have no meaning. The resultset is always empty once they are being used. So, I've returned to use 'for $x in collection("CDI")' in the first query of getting all IDs, but the performance is extremely low.
**I'm attaching the query and its related info using BaseXGUI (local server) with much smaller database. The performance seems ok.
I'm using your BaseXClient.java, however I see the delay already in the BaseX server logs: QUERY[0] xquery version "3.0"; declare namespace queryName ='GetIDS'; declare namespace gco = "http://www.isotc211.org/2005/gco"; declare namespace gmd = "http://www.isotc211.org/2005/gmd"; declare namespace gml = "http://www.opengis.net/gml"; declare namespace gmx="http://www.isotc211.org/2005/gmx"; declare namespace sdn = "http://www.seadatanet.org"; dec lare namespace fn = "http://www.w3.org/2005/xpath-functions"; declare namespace xs = "http://www.w3.org/2001/XMLSchema"; declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:method 'xml';declare option output:item-separator ","; let $db := db:open("CDI") for $x in $db where $x/gmd:MD_Metadata/gmd:identificationInfo/sdn:SDN_DataIdentification/gmd:extent/gmd:EX_Exte nt/gmd:geographicElement/gmd:EX_GeographicBoundingBox/gmd:westBoundLongitude/gco:Decimal>="-5.8447265625" and $x/gmd:MD_Metadata/gmd:identificationInfo/sdn:SDN_DataIdentification/gmd:extent/gmd:EX_Extent/gmd :geographicElement... 0.17 ms 110 16:36:09.713 192.168.155.30:39211 admin OK RESULTS[0] 25957.11 ms
Then I have other slow queries, but I would like to focus in this phase on the biggest delay. Server: Java 1.7.0_79, VM="-XX:MaxPermSize=512m -Xms3096m -Xmx3096m" The network layer between client and server is very fast.
P.S. Id there an undocumented way to log the full xquery in BaseX server logs? I've seen the -V option, but I don't use the standalone version, but: java -cp /usr/share/java/basex.jar org.basex.BaseXServer -d doesn't give me extra query info.
With kind regards, Menashè