Very good news ! Since I know how to do my request with XPath and not use direct path to file anymore, the performance are excellent !
database size | direct path | XPath 2.4 Go | 1.5 sec 160% CPU | 0.25 sec 6% CPU 224 Ko | 0.24 sec 2% CPU | 0.24 sec 2% CPU
It looks that path query have poor performances with a huge number of files in the database, what seems fair to me since BaseX is optimized for XMLish uses.
Best regards
Florent
On Thu, Dec 18, 2014 at 4:50 PM, Florent Gallaire fgallaire@gmail.com wrote:
On Thu, Dec 18, 2014 at 3:35 PM, Pascal Heus pascal.heus@gmail.com
wrote:
We run BaseX on EC2 micro and small instances without significant issue
(though AWS servers, particularly micro/small, are not known >for their high performance). Have you tried on a different instance type? Not sure how heavily loaded your instance is but understand >that the t2 instance type works well for servers that are not under constant pressure (they're Burstable Performance Instances). See > http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/t2-instances.html. You essentially accumulate CPU "credits" while your >instance is idle, which then allows your CPU to burst to 100% when the server gets hit. If you're out of credits, your CPU may be >running at as low as 10% capacity.
Thanks a lot Pascal, I think you have the explanation I was searching for
!
I have only tested EC2 micro instance because there is a 1 year free offer. And the problem is CPU bound (at the beginning I was thinking at a RAM/swap problem).
Now you're also workign with a 2.5Gb database on a servers with 1Gb of
memory, so disk access almost inevitably comes into play. I >assume you have SSD EBS volume under the hood since your on t2, but you'll be swapping, and this is more likely the problem. How >much Java memory do you allocate to BaseX (-Xmx)? So I would say in your case a t2.small or preferably t2.medium would be a better >choice a better choice (of course more pricy).
Yes I use a 30 Go (max for the free offer) SSD EBS, Amazon says it performs 90 IOPS. I use the -Xmx default of 512Mo and I run that in a docker container (CoreOS powered).
Talking about swap, be aware also that Amazon Linux instances don't come
configured with a disk swap by default, so the 1Gb in all you >get. I always add a 1Gb swap file to our micro instances using:
sudo dd if=/dev/zero of=/swapfile bs=1M count=1024 sudo mkswap /swapfile sudo swapon /swapfile which you can verify with sudo swapon -s or free -k Also add the line below to the /etc/fstab file (so it's survives through
reboot)
/swapfile swap swap defaults 0
Thank for the tip !
Florent
-- FLOSS Engineer & Lawyer