Hi Christian,

Please find the response 1. Can you reproduce this behavior on another independent Linux system? Yes it can be replicated in any Linux Machine. 2. What happens with a different Java version? We are supposed to change the version of java which we support. 3. Do you store all data on a local disk (is it an HDD or SSD, same on both systems?) In window and in linux both are hdd and in Linux it is using Ext4 storage type, we have provided sufficient heap size too.

On further analysis we found out

1. When BaseX Server is in Windows machine and Java client program is on same windows machine it is taking 3-15(no network latency) ms to execute Query. 2. When BaseX Server is in Linux machine and Java client program is on Other windows machine it is taking 54-60(with network latency) ms to execute Query. 3. When BaseX Server is in Linux machine and Java client program is on same linux machine it is taking 126 ms(no network latency) to execute Query.

From the above observation it seems xqj api seems to be slow in the linux machine.

We are using following jars while executing the queries

1. xqj-api-1.0.jar 2. xqj2-0.2.0.jar 3. basex-9.2.4.jar 4. basex-xqj-9.0.jar 5. javax.xml-1.3.4.jar

And below is the logic to create connection and execute the query : System.setProperty("user.home", db_dir); BaseXXQDataSource ds = new BaseXXQDataSource(); ds.setProperty("serverName", "localhost"); ds.setProperty("port", "1984"); ds.setProperty("databaseName", "mydbname"); XQConnection2 connection = (XQConnection2) ds.getConnection("admin", "myDbPassword"); XQStaticContext xqsc = connection.getStaticContext(); xqsc.setScrollability(XQConstants.SCROLLTYPE_SCROLLABLE); connection.setStaticContext(xqsc); XQPreparedExpression expr = connection.prepareExpression(query); XQResultSequence result = expr.executeQuery(); XQSequence scrollableCopy = connection.createSequence(result);

Is there any configurations required in linux machine to overcome from this lag or Is the any other way to execute the basex queries and get the response with data in Node (like XQResultSequence getNode() returns data as node object)

Thanks, Nikhil

________________________________ From: Christian Grün christian.gruen@gmail.com Sent: Monday, August 10, 2020 2:32 PM To: Nikhil Sethi nsethi@opentext.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de; Devi Anantharaman danantha@opentext.com Subject: [EXTERNAL] - Re: [basex-talk] Basex huge performance hit in linux (Basex query execution is too slow in linux)

Hi Nikhil,

If your hardware environment is similar, the difference between Windows and Linux should actually be negligible.

Some questions that might help to track the problem down:

1. Can you reproduce this behavior on another independent Linux system? 2. What happens with a different Java version? 3. Do you store all data on a local disk (is it an HDD or SSD, same on both systems?)

Best, Christian

On Sun, Aug 9, 2020 at 10:29 AM Nikhil Sethi nsethi@opentext.com wrote:

Hi Basex Team,

We are facing a huge performance hit in linux machine while using basex, its working smoothly in windows machine

While executing any query : -> windows machine takes around 4 to 15 millisec -> On linux machine same query takes around 120 to 150 millisec

During some application operation execution we are selecting/inserting the xml nodes in different docs in basex and there are around 13000 xml nodes which needs to be selected/imported in this operation.

Linux => rhel or cents of

Based on the logs of query execution we have analysed and found : Windows worst case(15 ms) query execution time => 13000 x 15ms => 65000ms => 65 sec = around 1 min Linux average case(130 ms(120+150)/2) query execution time => 13000 x 130ms => 1690000ms => 1690 sec = around 28 min to execute same operation

we are using BaseX941.zip ZIP Package

eg queries :

for $alias in collection(mydbname)//* where (($alias/@id="4249f0807da81000a734525400c54430018") and ($alias/@deleted="false") and ($alias/object-type/@uid="com.MyObjectName")) return $alias

or

insert node <myxmlNode>…….</myxmlNode> into (for $alias in doc('MyDoc') return $alias)

I am using basex with java

code to get the result

System.setProperty("user.home", db_dir); BaseXXQDataSource ds = new BaseXXQDataSource(); ds.setProperty("serverName", "localhost"); ds.setProperty("port", "1984"); ds.setProperty("databaseName", "mydbname"); XQConnection2 connection = (XQConnection2) ds.getConnection("admin", "myDbPassword"); XQStaticContext xqsc = connection.getStaticContext(); xqsc.setScrollability(XQConstants.SCROLLTYPE_SCROLLABLE); connection.setStaticContext(xqsc); XQPreparedExpression expr = connection.prepareExpression(query); XQResultSequence result = expr.executeQuery(); XQSequence scrollableCopy = connection.createSequence(result);

getting result from the xqsequnce for 13000 queries is taking another 10 mins Node node = scrollableCopy.getNode()

we have tried to increase xms and xmx in basexserver but it didn't help, Can you please suggest any solution/configuration required for linux machine to increase the performance.

Thanks, Nikhil

Hi Christian

I have tried using the slandered client which seems to be quite fast in linux during retrieval , But i am facing issue with inserting data inside the database in windows and linux both when i am using it in my application :.

I am creating connection as below :

My database name is xms and to make database portable(so that i can copy paste my db anywhere along with my application ) i am setting the 3 properties(user.dir,user.home,org.basex.path) to basex server folder home (db_dir) location:

BaseXClient obtainSession() { BaseXClient connection = null; System.setProperty("user.dir", db_dir); System.setProperty("user.home", db_dir); System.setProperty("org.basex.path", db_dir); BaseXClient connection = null; try { connection = new BaseXClient("localhost", 1984, "admin", "mypassword"); connection.execute("open xms"); } catch (NumberFormatException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } return connection; }

From my application using BaseXClient connection i am able to create the Table/document under my xms db (like different tables user, environment,..) to keep different type of data :

public void createLibrary(String libraryName) { BaseXClient connection = datasourceFactory.getConnection(); try { InputStream bais = new ByteArrayInputStream("<root></root>".getBytes()); connection.add(libraryName, bais); } catch (IOException e) { logger.error("Error occured while creating library : " + libraryName); }

Creating document is working fine but is unable to insert the data inside the db while using it from springboot application :

We are using the below method to execute the queries private Query executeQuery(String query) { Query result = null; BaseXClient connection = obtainSession(); try { result = connection.query(query); System.out.println("connection.info() : " + connection.info()); } catch (Exception e) { e.printStackTrace(); } return result; }

while inserting the node we are seeing connection is opened but no data is inserted when we are verifying through gui,

The same program works from standalone java program but doesn't work in Sprinboot application: 2020/08/16 18:03:56,160 DEBUG [main][com.documentum.xms.persistence.impl.BaseXTemplate] - insert node <user created-date="2020-08-16T18:03:56.006+05:30" deleted="false" enabled="true" id="eab02f7c7da910008891a4c3f0d6e12801" modified-date="2020-08-16T18:03:55.987+05:30" number-changes="1" password="AAAAEDS/EsTWFnWo8ZvwR20m2hwGXpjYh0lI8f4QyTwZPochfdOE5Ih8sjI+b+O5SwcjVw==" role="ROLE_ADMIN" username="admin"> <object-type name="User" uid="com.emc.xms.resources.User"/> <resource-state> <currentState>WIP</currentState> </resource-state> <vdc>*</vdc> </user> into (for $alias in doc('xms/User') return $alias) 2020/08/16 18:03:56,161 INFO [main][com.documentum.xms.persistence.impl.BaseXTemplate] - connection.info() : Database 'xms' was opened in 0.03 ms.

Is there any configuration required to execute queries while using it in springboot?

Thanks, Nikhil

________________________________ From: Christian Grün christian.gruen@gmail.com Sent: Wednesday, August 12, 2020 8:09 PM To: Nikhil Sethi nsethi@opentext.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de; Devi Anantharaman danantha@opentext.com Subject: Re: [EXTERNAL] - Re: [basex-talk] Basex huge performance hit in linux (Basex query execution is too slow in linux)

Hi Nikhil,

Thanks for your reply.

As the XQJ driver was not implemented by us, but an external contributor (Charles Foster): Do you encounter similar issues with the standard client [1]?

Best, Christian

[1] https://urldefense.com/v3/__https://docs.basex.org/wiki/Clients__;!!Obbck6kT...

On Wed, Aug 12, 2020 at 4:25 PM Nikhil Sethi nsethi@opentext.com wrote:

Hi Christian,

Please find the response

1. Can you reproduce this behavior on another independent Linux system?

Yes it can be replicated in any Linux Machine. 2. What happens with a different Java version? We are supposed to change the version of java which we support. 3. Do you store all data on a local disk (is it an HDD or SSD, same on both systems?) In window and in linux both are hdd and in Linux it is using Ext4 storage type, we have provided sufficient heap size too.

On further analysis we found out

When BaseX Server is in Windows machine and Java client program is on same windows machine it is taking 3-15(no network latency) ms to execute Query. When BaseX Server is in Linux machine and Java client program is on Other windows machine it is taking 54-60(with network latency) ms to execute Query. When BaseX Server is in Linux machine and Java client program is on same linux machine it is taking 126 ms(no network latency) to execute Query.

From the above observation it seems xqj api seems to be slow in the linux machine.

We are using following jars while executing the queries

xqj-api-1.0.jar xqj2-0.2.0.jar basex-9.2.4.jar basex-xqj-9.0.jar javax.xml-1.3.4.jar

And below is the logic to create connection and execute the query : System.setProperty("user.home", db_dir); BaseXXQDataSource ds = new BaseXXQDataSource(); ds.setProperty("serverName", "localhost"); ds.setProperty("port", "1984"); ds.setProperty("databaseName", "mydbname"); XQConnection2 connection = (XQConnection2) ds.getConnection("admin", "myDbPassword"); XQStaticContext xqsc = connection.getStaticContext(); xqsc.setScrollability(XQConstants.SCROLLTYPE_SCROLLABLE); connection.setStaticContext(xqsc); XQPreparedExpression expr = connection.prepareExpression(query); XQResultSequence result = expr.executeQuery(); XQSequence scrollableCopy = connection.createSequence(result);

Is there any configurations required in linux machine to overcome from this lag or Is the any other way to execute the basex queries and get the response with data in Node (like XQResultSequence getNode() returns data as node object)

Thanks, Nikhil

---

From: Christian Grün christian.gruen@gmail.com Sent: Monday, August 10, 2020 2:32 PM To: Nikhil Sethi nsethi@opentext.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de; Devi Anantharaman danantha@opentext.com Subject: [EXTERNAL] - Re: [basex-talk] Basex huge performance hit in linux (Basex query execution is too slow in linux)

Hi Nikhil,

If your hardware environment is similar, the difference between Windows and Linux should actually be negligible.

Some questions that might help to track the problem down:

1. Can you reproduce this behavior on another independent Linux system?
2. What happens with a different Java version?
3. Do you store all data on a local disk (is it an HDD or SSD, same on

both systems?)

Best, Christian

On Sun, Aug 9, 2020 at 10:29 AM Nikhil Sethi nsethi@opentext.com wrote:

...

Hi Basex Team,

We are facing a huge performance hit in linux machine while using basex, its working smoothly in windows machine

While executing any query : -> windows machine takes around 4 to 15 millisec -> On linux machine same query takes around 120 to 150 millisec

During some application operation execution we are selecting/inserting the xml nodes in different docs in basex and there are around 13000 xml nodes which needs to be selected/imported in this operation.

Linux => rhel or cents of

Based on the logs of query execution we have analysed and found : Windows worst case(15 ms) query execution time => 13000 x 15ms => 65000ms => 65 sec = around 1 min Linux average case(130 ms(120+150)/2) query execution time => 13000 x 130ms => 1690000ms => 1690 sec = around 28 min to execute same operation

we are using BaseX941.zip ZIP Package

eg queries :

for $alias in collection(mydbname)//* where (($alias/@id="4249f0807da81000a734525400c54430018") and ($alias/@deleted="false") and ($alias/object-type/@uid="com.MyObjectName")) return $alias

or

insert node <myxmlNode>…….</myxmlNode> into (for $alias in doc('MyDoc') return $alias)

I am using basex with java

code to get the result

System.setProperty("user.home", db_dir); BaseXXQDataSource ds = new BaseXXQDataSource(); ds.setProperty("serverName", "localhost"); ds.setProperty("port", "1984"); ds.setProperty("databaseName", "mydbname"); XQConnection2 connection = (XQConnection2) ds.getConnection("admin", "myDbPassword"); XQStaticContext xqsc = connection.getStaticContext(); xqsc.setScrollability(XQConstants.SCROLLTYPE_SCROLLABLE); connection.setStaticContext(xqsc); XQPreparedExpression expr = connection.prepareExpression(query); XQResultSequence result = expr.executeQuery(); XQSequence scrollableCopy = connection.createSequence(result);

getting result from the xqsequnce for 13000 queries is taking another 10 mins Node node = scrollableCopy.getNode()

we have tried to increase xms and xmx in basexserver but it didn't help, Can you please suggest any solution/configuration required for linux machine to increase the performance.

Thanks, Nikhil