All,
I am facing some issues with BaseX (9.3.1) performance when running cold, i.e. immediately after the server is started.
The problem I am seeing is that the performance of moderately complex queries is terrible for a some time after the database is opened, but then improves dramatically after a few query evaluations have taken place. For instance, a certain query takes around 7 minutes to when the server has just been started, but then performance improves to consistently to take around 18 seconds. This happens for most queries, it is not a characteristic of any particular query, as far as I can tell. This is annoying, of course, and users are complaining that the system is not responsive.
Here are some observations:
1. The first time the query is submitted the java process does not consume any CPU cycles and no footprint is added to the 10GB that is allocated when the database is opened (it takes around 1 minute to open the database. You can see the database info below). I would have expected a surge of activity as soon as the query was submitted, as all the 16 cores contribute to evaluating the result. 2. If I use the BaseX GUI (with an allowed 26GB but consuming around 10GB in this case) I never get any delays - The query performance is consistently around 18 secs, immediately after opening the database. 3. Both the HTTP server or TCP server exhibit this anomaly. 4. After the initial sluggishness, when the database is running warm, the performance stays satisfactory, at around 18 seconds. I have never seen it return to the bad situation. 5. The first time the query is run, the footprint rises about 1GB to around 11GB. It does not increase with subsequent query evaluations. 6. I think that the system remains warm even if I close down the java server process and start it again.
Is this an accepted behavior or an anomaly? Why is the behavior different if the database is queried in the stand-alone situation (the BaseX UI client)? Is there some setting that I need to provide to the Java process to ameliorate the cold performance? Any indexes that I have to put in place? Any suggestions are welcome.
Here are some stats describing the hardware setup: The machine hosting both the server and the client is a powerful 16 core box running a AMD EPYC 7452 Processor with 64GB RAM running in the cloud; since they both the client and the server are running on the same box, there should be negligible network latency in this scenario. The machine (running windows server 2016 datacenter) is doing nothing else at the time of these tests: The Idle process consumes 99% of the CPU time and almost no disk or network access takes place. The machine does not have hardware disks - They are hosted on another box, I believe.
The database information: <database> <databaseproperties> <name>generated</name> <size>16 GB</size> <nodes>705422480</nodes> <documents>136673</documents> <binaries>0</binaries> <timestamp>2020-09-17T22:24:15.037Z</timestamp> <uptodate>false</uptodate> </databaseproperties> <resourceproperties> <inputpath>C:/Users/pvillads/Desktop/generated/</inputpath> <inputsize>16 GB</inputsize> <inputdate>2020-09-17T22:24:14.943Z</inputdate> </resourceproperties> <indexes> <textindex>false</textindex> <attrindex>false</attrindex> <tokenindex>false</tokenindex> <ftindex>false</ftindex> <textinclude/> <attrinclude/> <tokeninclude/> <ftinclude/> <language>English</language> <stemming>false</stemming> <casesens>false</casesens> <diacritics>false</diacritics> <stopwords/> <updindex>false</updindex> <autooptimize>false</autooptimize> <maxcats>100</maxcats> <maxlen>96</maxlen> <splitsize>0</splitsize> </indexes> </database>
$ java -version java version "1.8.0_271" Java(TM) SE Runtime Environment (build 1.8.0_271-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.271-b09, mixed mode)
Thanks,
Best Regards
Peter Villadsen Principal Architect Microsoft Business Applications Group