Hi,
your query only reads data, right? With a db that small you could make sure it is all in memory by doing let $db := db:open('foo') update { }
Then, at least you don't have a penalty for disk/index access. As well, If I want to make sure that some data is availabe really fast (and to avoid n*n cost), I use the map:merge [1] function to build a quick look-up structure.
[1] http://docs.basex.org/wiki/Map_Module#map:merge
Regards,
Max
2016-08-03 19:06 GMT+02:00 Bondeson, Carl Carl.Bondeson@ct.gov:
I am looking at creating a deliverable jar, in the not too distant future, which illustrates the issues. I just don't have the time right now. It is on my to-do list. Thanks again for your support!
Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Wednesday, August 03, 2016 1:02 PM To: Bondeson, Carl Carl.Bondeson@ct.gov Cc: Luiz Matos luizmatos@ic.uff.br; basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x
Sorry to hear that. If you can provide us your data, we could have another look.
On Wed, Aug 3, 2016 at 7:00 PM, Bondeson, Carl Carl.Bondeson@ct.gov wrote:
I have not been able to get back to the same performance as was seen in 7.9. I attempted a change in architecture to use separate databases rather than having them combined in one. I created classes to support them, and updates to the XSLT tier to address the namespace additions. So rather than having 1 database with 14 MB of data, I now have 4 databases served by their own classes. In the end the performance was once again much worse than 7.9 with a single database. I have to look at the cost of calling up to 3 constructors in the XSLT transform, while looking at the gains, if any, or working with smaller individual databases.I will save this branch for another day. I have too many deliverables to afford any more resources at this time.
Carl R Bondeson IT Analyst 3 Information Technology Connecticut Department of Public Health 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Friday, July 29, 2016 7:45 AM To: Bondeson, Carl Carl.Bondeson@ct.gov Cc: Luiz Matos luizmatos@ic.uff.br; basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x
I have put together a set of queries, the output, and corresponding xml.
I have run the first query; it yields no results. Is this expected? If you want, you can send me some more data in private.
Could you please pass me on the query info output for both 8.5.2 and 7.9?
I appreciate your support.
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson@ct.gov
From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Friday, July 29, 2016 5:07 AM To: Bondeson, Carl Carl.Bondeson@ct.gov; Luiz Matos luizmatos@ic.uff.br Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] BaseX performance 7.x vs 8.x
Hi Carl, hi Luiz,
With every version of BaseX, we are adding new query optimizations (usually, this can be experienced if you run newer code with older version of BaseX). Existing optimization should of course be preserved. To be able to understand why your queries run slower than before, I would be more than glad if you could both provide me with little examples that we can try by ourselves. The Query Info output for both versions of BaseX would be helpful as well; it indicates which optimizations are performed or not.
Thanks in advance
Christian
On Thu, Jul 28, 2016 at 8:36 PM, Bondeson, Carl Carl.Bondeson@ct.gov wrote:
As a follow up for performance a few queries are listed with execution times
declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='Gender'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/>
Version: 7.9 execution time: 3.82 ms
Version: 8.5.1 execution time: 138.8 ms
declare option output:item-separator "
"; for $h in //ValidationDictionary/Validation let $g := $h/Parent[@Element='PatientDefinition'] let $i := $g/Child[@Element='PartyDefinition'] let $j := $i/Item[@Attribute='FirstName'] for $k in $j/@Type where $h/ProductCode='ZIKA' return <BaseXValidation Parent='{$g/@Element}' Child='{$i/@Element}' Action='{$k/../@Action}' Attribute='{$k/../@Attribute}' DataType='{$k/..//@DataType}' Effect='{$k/../@Effect}' Name='{$k/../@Name}' Type='{$k}' LogEvent='{$k/../@LogEvent}' Extra1='{$k/../@Extra1}' Extra2='{$k/../@Extra2}' Extra3='{$k/../@Extra3}'/>
Version: 7.9 execution time: 3.3 ms
Version: 8.5.1 execution time: 133.8 ms
Query:declare option output:item-separator "
";for $g in //ValidationDictionary//Validation let $h := $g//Parent[@Element='PatientDefinition']let $i := $h//Child[@Element='PartyDefinition'] where $g//ProductCode='ZIKA' return <BaseXValidation Parent='{$h/@Element}' Child='{$i/@Element}'/>
Version: 7.9 execution time: 2.2 ms
Version: 8.5.1 execution time: 156.1 ms
These queries are called from an XSLT transformer via all callout and are performed in very type loops. I was wondering why the transforms were never completing. 40-70x decrease in performance would slow any process down.
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson@ct.gov
From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, July 28, 2016 1:36 PM To: basex-talk@mailman.uni-konstanz.de Subject: [basex-talk] BaseX performance 7.x vs 8.x
I am seeing huge differences in the performance of a multi-threaded JBoss application recently upgraded from 7.9 to 8.4. The problem is that 8.4 is much slower than 7.9. Any ideas? Do I need to add indices or something along those lines?
Carl R Bondeson
IT Analyst 3
Information Technology
Connecticut Department of Public Health
410 Capitol Ave
Hartford, CT 06134
Phone: 860-509-7434
carl.bondeson@ct.gov