I’m still seeing performance issues that I’m not sure how to diagnose. The symptom is:
1. Go to a page that does a query that takes 20 seconds to complete in order to build a report. This is acceptable performance. 2. Navigate to a different page. 3. Return to the first page—it either takes a very long time (minutes) to respond or never returns. In this case, going to a different browser I can get to the page with the same good initial response time
This suggests that maybe it’s an issue with browser cache or the server not invalidating a cache or something or just an implementation mistake on my part.
I will start adding more diagnostic messages in my server code to see what’s happening when but this really feels like I just have some setting not set correctly.
I take it as a given that BaseX, properly configured and using queries that are not poorly implemented should perform acceptably as a concurrent web server with not too much load (10s or 100s of users, not 1000s of users).
Cheers,
E.
_____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow
From: Eliot Kimber eliot.kimber@servicenow.com Date: Sunday, February 6, 2022 at 12:31 AM To: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Managing/Debugging Server Load and Performance Liam,
Here’s what free -h tells me:
/data/basex/git/dita-build-tools/src/main$ free -h total used free shared buff/cache available Mem: 15G 1.4G 8.3G 529M 6.0G 13G Swap: 0B 0B 0B
I read that as there being no swap. But note that this is a virtual machine (from AWS as far as I know)—I don’t know if that makes a difference.
If I understand what cpuinfo is telling me, I have 4 CPUs, each with 4 cores, so should be able to service 16 threads, yes?
I have DTD parsing turned off (which is the default)—if I didn’t do that it would take hours to load our content into the database, so that *shouldn’t* be a problem (I hope).
I did find prof:runtime() in the docs and verified that it has 4GB of RAM available.
_____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow
From: Liam R. E. Quin liam@fromoldbooks.org Date: Sunday, February 6, 2022 at 12:08 AM To: Eliot Kimber eliot.kimber@servicenow.com, basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Managing/Debugging Server Load and Performance [External Email]
On Sun, 2022-02-06 at 03:34 +0000, Eliot Kimber wrote:
- Using the JRE provided with Oxygen, allocated with 4GB (we are
also using this server to run Oxygen via scripting and it needs 8GB to handle our insanely huge DITA maps)
Make sure you have e.g. 64 gigabytes or more of swap configures; free -h will tell you this
- Set parallel to 4 (to match the number of cores, but just
guessing that this is a useful setting based on the docs)
check /proc/cpuinfo (e.g, less /proc/cpuinfo) and you'll prolly find it can run 8 threads
I’m seeing some apparent occasional slowness on pages that should not be slow (don’t reflect long-running queries or huge data volumes)
make sure there are no xml catalogs or DTDs to be fetched externally - or, if there are catalogs, e.g. used with fn:transform(), that those catalog files do NOT start with a doctype that causes a network fetch of a dtd...
but I’m not really sure how to diagnose it or even verify that I’ve succeeded in giving BaseX all the resources it needs.
maybe in an ssh/terminal window, keep "top" running while you fetch a page, and see if the system gets really busy. Note also the centos system is probably using a hard drive, not an SSD, so file access may be slower - make sure you have indexes!
hope this helps at least a little,
-- Liam Quin, https://urldefense.com/v3/__https://www.delightfulcomputing.com/__;!!N4vogdj...https://urldefense.com/v3/__https:/www.delightfulcomputing.com/__;!!N4vogdjhuJM!U0hEyXTeoTf2B18jzQmUco2XDS97VUqRet5HS3OjWp_cIEbY9gMS9UJKn8aBFs61KYmFdA$ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: https://urldefense.com/v3/__http://www.fromoldbooks.org__;!!N4vogdjhuJM!U0hE...https://urldefense.com/v3/__http:/www.fromoldbooks.org__;!!N4vogdjhuJM!U0hEyXTeoTf2B18jzQmUco2XDS97VUqRet5HS3OjWp_cIEbY9gMS9UJKn8aBFs5ZXch2EA$