Re: [basex-talk] [EXTERNAL] - Re: Basex huge performance hit in linux (Basex query execution is too slow in linux)

Hi Nikhil,

Thanks for your reply.

As the XQJ driver was not implemented by us, but an external contributor (Charles Foster): Do you encounter similar issues with the standard client [1]?

Best, Christian

[1] https://docs.basex.org/wiki/Clients

On Wed, Aug 12, 2020 at 4:25 PM Nikhil Sethi nsethi@opentext.com wrote:

Hi Christian,

Please find the response

1. Can you reproduce this behavior on another independent Linux system?

Yes it can be replicated in any Linux Machine. 2. What happens with a different Java version? We are supposed to change the version of java which we support. 3. Do you store all data on a local disk (is it an HDD or SSD, same on both systems?) In window and in linux both are hdd and in Linux it is using Ext4 storage type, we have provided sufficient heap size too.

On further analysis we found out

When BaseX Server is in Windows machine and Java client program is on same windows machine it is taking 3-15(no network latency) ms to execute Query. When BaseX Server is in Linux machine and Java client program is on Other windows machine it is taking 54-60(with network latency) ms to execute Query. When BaseX Server is in Linux machine and Java client program is on same linux machine it is taking 126 ms(no network latency) to execute Query.

From the above observation it seems xqj api seems to be slow in the linux machine.

We are using following jars while executing the queries

xqj-api-1.0.jar xqj2-0.2.0.jar basex-9.2.4.jar basex-xqj-9.0.jar javax.xml-1.3.4.jar

And below is the logic to create connection and execute the query : System.setProperty("user.home", db_dir); BaseXXQDataSource ds = new BaseXXQDataSource(); ds.setProperty("serverName", "localhost"); ds.setProperty("port", "1984"); ds.setProperty("databaseName", "mydbname"); XQConnection2 connection = (XQConnection2) ds.getConnection("admin", "myDbPassword"); XQStaticContext xqsc = connection.getStaticContext(); xqsc.setScrollability(XQConstants.SCROLLTYPE_SCROLLABLE); connection.setStaticContext(xqsc); XQPreparedExpression expr = connection.prepareExpression(query); XQResultSequence result = expr.executeQuery(); XQSequence scrollableCopy = connection.createSequence(result);

Is there any configurations required in linux machine to overcome from this lag or Is the any other way to execute the basex queries and get the response with data in Node (like XQResultSequence getNode() returns data as node object)

Thanks, Nikhil

---

From: Christian Grün christian.gruen@gmail.com Sent: Monday, August 10, 2020 2:32 PM To: Nikhil Sethi nsethi@opentext.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de; Devi Anantharaman danantha@opentext.com Subject: [EXTERNAL] - Re: [basex-talk] Basex huge performance hit in linux (Basex query execution is too slow in linux)

Hi Nikhil,

If your hardware environment is similar, the difference between Windows and Linux should actually be negligible.

Some questions that might help to track the problem down:

1. Can you reproduce this behavior on another independent Linux system?
2. What happens with a different Java version?
3. Do you store all data on a local disk (is it an HDD or SSD, same on

both systems?)

Best, Christian

On Sun, Aug 9, 2020 at 10:29 AM Nikhil Sethi nsethi@opentext.com wrote:

...

Hi Basex Team,

We are facing a huge performance hit in linux machine while using basex, its working smoothly in windows machine

While executing any query : -> windows machine takes around 4 to 15 millisec -> On linux machine same query takes around 120 to 150 millisec

During some application operation execution we are selecting/inserting the xml nodes in different docs in basex and there are around 13000 xml nodes which needs to be selected/imported in this operation.

Linux => rhel or cents of

Based on the logs of query execution we have analysed and found : Windows worst case(15 ms) query execution time => 13000 x 15ms => 65000ms => 65 sec = around 1 min Linux average case(130 ms(120+150)/2) query execution time => 13000 x 130ms => 1690000ms => 1690 sec = around 28 min to execute same operation

we are using BaseX941.zip ZIP Package

eg queries :

for $alias in collection(mydbname)//* where (($alias/@id="4249f0807da81000a734525400c54430018") and ($alias/@deleted="false") and ($alias/object-type/@uid="com.MyObjectName")) return $alias

or

insert node <myxmlNode>…….</myxmlNode> into (for $alias in doc('MyDoc') return $alias)

I am using basex with java

code to get the result

System.setProperty("user.home", db_dir); BaseXXQDataSource ds = new BaseXXQDataSource(); ds.setProperty("serverName", "localhost"); ds.setProperty("port", "1984"); ds.setProperty("databaseName", "mydbname"); XQConnection2 connection = (XQConnection2) ds.getConnection("admin", "myDbPassword"); XQStaticContext xqsc = connection.getStaticContext(); xqsc.setScrollability(XQConstants.SCROLLTYPE_SCROLLABLE); connection.setStaticContext(xqsc); XQPreparedExpression expr = connection.prepareExpression(query); XQResultSequence result = expr.executeQuery(); XQSequence scrollableCopy = connection.createSequence(result);

getting result from the xqsequnce for 13000 queries is taking another 10 mins Node node = scrollableCopy.getNode()

we have tried to increase xms and xmx in basexserver but it didn't help, Can you please suggest any solution/configuration required for linux machine to increase the performance.

Thanks, Nikhil