I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
Exception: Unknown Query ID:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.govmailto:carl.bondeson@ct.gov [cid:image001.jpg@01CFD286.0424E090]http://www.ct.gov/dph
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
I forgot that since the server was started and the database constructed in the singleton, any new instances will still have access to it. I was using the client session as a permanent object rather than recreating these for each class instance.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, September 18, 2014 7:25 AM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
Can I create a BaseX database in the singleton, add the necessary documents to it, and then reference this database in any thread wishing to create connect client session to it? I don't see this discussed anywhere.
Thanks
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, September 18, 2014 9:27 AM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I forgot that since the server was started and the database constructed in the singleton, any new instances will still have access to it. I was using the client session as a permanent object rather than recreating these for each class instance.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, September 18, 2014 7:25 AM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
I figured it out. I create the database in the singleton, adding all the necessary XML documents. When the thread needs to perform a query I just create a new instance of the client session and open the database. The performance is excellent!!
-----Original Message----- From: Bondeson, Carl Sent: Tuesday, September 23, 2014 12:58 PM To: Bondeson, Carl; Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Multi threaded application
Can I create a BaseX database in the singleton, add the necessary documents to it, and then reference this database in any thread wishing to create connect client session to it? I don't see this discussed anywhere.
Thanks
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, September 18, 2014 9:27 AM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I forgot that since the server was started and the database constructed in the singleton, any new instances will still have access to it. I was using the client session as a permanent object rather than recreating these for each class instance.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, September 18, 2014 7:25 AM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
My next question. Do I need to create indices for attributes within the XML documents? Are these created by default? Will I get better performance in my multithreaded environment?
Thanks.. Great product!
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Tuesday, September 23, 2014 2:57 PM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I figured it out. I create the database in the singleton, adding all the necessary XML documents. When the thread needs to perform a query I just create a new instance of the client session and open the database. The performance is excellent!!
-----Original Message----- From: Bondeson, Carl Sent: Tuesday, September 23, 2014 12:58 PM To: Bondeson, Carl; Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Multi threaded application
Can I create a BaseX database in the singleton, add the necessary documents to it, and then reference this database in any thread wishing to create connect client session to it? I don't see this discussed anywhere.
Thanks
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, September 18, 2014 9:27 AM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I forgot that since the server was started and the database constructed in the singleton, any new instances will still have access to it. I was using the client session as a permanent object rather than recreating these for each class instance.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, September 18, 2014 7:25 AM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
Hi Carl,
if you want to know more about index structures in BaseX, I invite you to check out the related article in our Wiki [1].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Index
On Tue, Sep 23, 2014 at 9:14 PM, Bondeson, Carl Carl.Bondeson@ct.gov wrote:
My next question. Do I need to create indices for attributes within the XML documents? Are these created by default? Will I get better performance in my multithreaded environment?
Thanks.. Great product!
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Tuesday, September 23, 2014 2:57 PM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I figured it out. I create the database in the singleton, adding all the necessary XML documents. When the thread needs to perform a query I just create a new instance of the client session and open the database. The performance is excellent!!
-----Original Message----- From: Bondeson, Carl Sent: Tuesday, September 23, 2014 12:58 PM To: Bondeson, Carl; Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Multi threaded application
Can I create a BaseX database in the singleton, add the necessary documents to it, and then reference this database in any thread wishing to create connect client session to it? I don't see this discussed anywhere.
Thanks
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, September 18, 2014 9:27 AM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I forgot that since the server was started and the database constructed in the singleton, any new instances will still have access to it. I was using the client session as a permanent object rather than recreating these for each class instance.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, September 18, 2014 7:25 AM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
FYI. I have designed and implemented a multithreaded ELR (Electronic Laboratory Reporting) message transformation, translation, and conversion system using BaseX as the API for all canonical and hierarchical translations. I originally wrote it using an API from Apelon which had many issues, most notably performance. When I started doing performance testing I was noticing the largest performance hits were from the BaseX queries. This warranted looking into using a read-only/static database in a singleton which is accessed synchronously by any of the threads. The difference in performance was stunning. Queries that originally took 100s of millisecond are now executing in 5 or less milliseconds. This system is being used to receive medical lab reports from various hospitals, laboratories, etc. etc. It then transforms these messages from various formats to a common schema. These objects are then passed through various levels of translation, validation, and transformation. They are then delivered in various formats to data systems that are used by epidemiologists to track outbreaks, and the like. A test case delivered 50 payloads with 100s of messages in each to the system. It originally took 16 minutes to process. It now takes under 3. I just wanted to pass on a success story when most of the dialogs are only relegated to problems or issues.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Tuesday, September 23, 2014 5:41 PM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
if you want to know more about index structures in BaseX, I invite you to check out the related article in our Wiki [1].
Hope this helps, Christian
[1] http://docs.basex.org/wiki/Index
On Tue, Sep 23, 2014 at 9:14 PM, Bondeson, Carl Carl.Bondeson@ct.gov wrote:
My next question. Do I need to create indices for attributes within the XML documents? Are these created by default? Will I get better performance in my multithreaded environment?
Thanks.. Great product!
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Tuesday, September 23, 2014 2:57 PM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I figured it out. I create the database in the singleton, adding all the necessary XML documents. When the thread needs to perform a query I just create a new instance of the client session and open the database. The performance is excellent!!
-----Original Message----- From: Bondeson, Carl Sent: Tuesday, September 23, 2014 12:58 PM To: Bondeson, Carl; Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: RE: [basex-talk] Multi threaded application
Can I create a BaseX database in the singleton, add the necessary documents to it, and then reference this database in any thread wishing to create connect client session to it? I don't see this discussed anywhere.
Thanks
-----Original Message----- From: basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Bondeson, Carl Sent: Thursday, September 18, 2014 9:27 AM To: Christian Grün Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
I forgot that since the server was started and the database constructed in the singleton, any new instances will still have access to it. I was using the client session as a permanent object rather than recreating these for each class instance.
-----Original Message----- From: Christian Grün [mailto:christian.gruen@gmail.com] Sent: Thursday, September 18, 2014 7:25 AM To: Bondeson, Carl Cc: basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Multi threaded application
Hi Carl,
Exception: Unknown Query ID:
The exception indicates that you are launching concurrent queries with a single client session. I surely know too less about your concrete implementation, but are the some specific reasons for avoiding the creation of multiple ClientSession objects?
Best, Christian
I have using BaseX in a canonical manner to handle the many translations used in ELR systems. I won't get into specifics but it has been working well. I started looking into the use of a singleton database object in place of the many ClientSession objects created in the constructor of the associated classes. In a single thread the performance varied between a 2x and 5x decrease. That would be expected since the amount of i/o required to load the XML would be replaced by simply executing queries against the database. The application that is utilizing BaseX is multithreaded. Once I deployed the application in JBoss I continue to receive exceptions:
These class methods are "synchronized" and are being invoked via Xalan callouts from XSLT.
I am currently using version 7.9 from the Maven repository.
Is BaseX thread safe in this manner?
Carl R Bondeson Systems Developer Department of Public Health Data Processing 410 Capitol Ave Hartford, CT 06134 Phone: 860-509-7434 carl.bondeson@ct.gov
basex-talk@mailman.uni-konstanz.de
-
Bondeson, Carl -
Christian Grün