Hi All,
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP. at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680) Exception in thread "Thread-1" java.lang.RuntimeException: java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164) at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP. at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680) Exception in thread "Thread-1" java.lang.RuntimeException: java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164) at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP. at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680) Exception in thread "Thread-1" java.lang.RuntimeException: java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164) at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
..difficult to guess what might be the problem. Just try if your solution works with a single thread, etc. ___________________________
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP. at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680) Exception in thread "Thread-1" java.lang.RuntimeException: java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164) at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680)Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR:com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK, value.getCollectionName())); InputStream is = XMLUtils.nodeToInputStream(value.getDocument()); baseXClient.replace(value.getId(), is); } catch (Exception e) { throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } } ...
What you mean "single writer"?
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov < dimitar.popov@uni-konstanz.de> wrote:
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting
all
documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680)Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR:com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection
and as
soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Using a single thread it works fine..All documents are inserted..
Regards
On Sun, Jun 24, 2012 at 11:47 AM, Willian Barcella wbarcella@gmail.comwrote:
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK, value.getCollectionName())); InputStream is = XMLUtils.nodeToInputStream(value.getDocument()); baseXClient.replace(value.getId(), is); } catch (Exception e) { throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } } ...
What you mean "single writer"?
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov < dimitar.popov@uni-konstanz.de> wrote:
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting
all
documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 =
10K,
however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680)Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR:com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection
and as
soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
I'm a little bit confused about your code. Some examples:
- after the execution of "baseXClient.execute(CLOSE)", baseXClient can never be null; however, it may actually be null before the call of execute() - what is hidden behind the CHECK and CLOSE variables?
At least for debugging purposes, I would recommend to create new client instead of getting them from the pool. Your connection should then be terminated via baseXClient.close().
Christian ___________________________
On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella wbarcella@gmail.com wrote:
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK, value.getCollectionName())); InputStream is = XMLUtils.nodeToInputStream(value.getDocument()); baseXClient.replace(value.getId(), is); } catch (Exception e) { throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } } ...
What you mean "single writer"?
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov dimitar.popov@uni-konstanz.de wrote:
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
On Sun, Jun 24, 2012 at 12:08 PM, Christian Grün christian.gruen@gmail.comwrote:
I'm a little bit confused about your code. Some examples:
- after the execution of "baseXClient.execute(CLOSE)", baseXClient can
never be null; however, it may actually be null before the call of execute()
*baseXClient can be null because can happen an exception taking this from queue.*
- what is hidden behind the CHECK and CLOSE variables?
*private static final String CHECK = Commands.Cmd.CHECK + " %s"; * *private static final String OPEN = Commands.Cmd.OPEN + " %s"; *
At least for debugging purposes, I would recommend to create new client instead of getting them from the pool. Your connection should then be terminated via baseXClient.close().
*For every call to method save() should I create a new BaseXClient?*
Christian ___________________________
On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella wbarcella@gmail.com wrote:
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK,
value.getCollectionName()));
InputStream is =XMLUtils.nodeToInputStream(value.getDocument()); baseXClient.replace(value.getId(), is); } catch (Exception e) { throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } } ...
What you mean "single writer"?
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov dimitar.popov@uni-konstanz.de wrote:
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not
persisting
all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class,
I
simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680)Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR:com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection and as soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
baseXClient can be null because can happen an exception taking this from queue.
If that’s true, I’d call this somewhat spectacular ;) Note that baseXClient is a local variable, which cannot be invalidated by any other thread. But it may well be that takeConnection() returns a null reference, in which case it won't make sense to call baseXClient.execute() just after that.
For every call to method save() should I create a new BaseXClient?
Exactly. Either way, please check your code once more, or try to provide us with an sscce [1]. I have doubts that the erroneous inserting behavior is really rooted in the BaseX core classes.
Christian
On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella wbarcella@gmail.com wrote:
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK, value.getCollectionName())); InputStream is = XMLUtils.nodeToInputStream(value.getDocument()); baseXClient.replace(value.getId(), is); } catch (Exception e) { throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } } ...
What you mean "single writer"?
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov dimitar.popov@uni-konstanz.de wrote:
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
> I builded a kind of connection pool using BaseXClient.java class, > I > simulated several users trying insert documents in BaseX, > the result was a disaster. > ERROR: data.BaseXDataAccess - Error closing db > java.io.IOException: Stopped at line 1, column 19: > Unknown command: 1340477810253095000. Try HELP. > > at data.BaseXClient.execute(BaseXClient.java:99) > at data.BaseXClient.execute(BaseXClient.java:110) > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > at data.TestInsert$Insert.run(TestInsert.java:88) > at java.lang.Thread.run(Thread.java:680) > > Exception in thread "Thread-1" java.lang.RuntimeException: > > java.io.IOException: > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > I started several threads each one takes a BaseXClient connection > and as > soon as it finishs its job BaseXClient is put in a > queue again to be took by other thread. > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk@mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
you are right there was an error in my code: replaced by: if (baseXClient != null) { baseXClient.execute(CLOSE); this.giveBack(baseXClient); } but it is not persisting all documents..All package sscce. On Sun, Jun 24, 2012 at 12:45 PM, Christian Grün christian.gruen@gmail.comwrote:
baseXClient can be null because can happen an exception taking this from queue.
If that’s true, I’d call this somewhat spectacular ;) Note that baseXClient is a local variable, which cannot be invalidated by any other thread. But it may well be that takeConnection() returns a null reference, in which case it won't make sense to call baseXClient.execute() just after that.
For every call to method save() should I create a new BaseXClient?
Exactly. Either way, please check your code once more, or try to provide us with an sscce [1]. I have doubts that the erroneous inserting behavior is really rooted in the BaseX core classes.
Christian
On Sun, Jun 24, 2012 at 4:47 PM, Willian Barcella wbarcella@gmail.com wrote:
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK, value.getCollectionName())); InputStream is = XMLUtils.nodeToInputStream(value.getDocument()); baseXClient.replace(value.getId(), is); } catch (Exception e) { throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } } ...
What you mean "single writer"?
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov dimitar.popov@uni-konstanz.de wrote:
check if you close the database after each insert operation. you
should
also know that currently, basex allows only a single writer, i.e. if you
use
multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting all documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000
=
10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote: > Hi Willian, > > thanks for your mail. I would assume that one single BaseXClient > object was used for several concurrent tasks. As BaseXClient > objects > are very light-weight, and as the registration of new clients is > usually more than fast enough, the usual approach is to create a > new > BaseXClient instance for a dedicated action. > > Hope this helps, > Christian > _______________________________ > > > I builded a kind of connection pool using BaseXClient.java
class,
> > I > > simulated several users trying insert documents in BaseX, > > the result was a disaster. > > ERROR: data.BaseXDataAccess - Error closing db > > java.io.IOException: Stopped at line 1, column 19: > > Unknown command: 1340477810253095000. Try HELP. > > > > at data.BaseXClient.execute(BaseXClient.java:99) > > at data.BaseXClient.execute(BaseXClient.java:110) > > at data.BaseXDataAccess.save(BaseXDataAccess.java:169) > > at data.TestInsert$Insert.run(TestInsert.java:88) > > at java.lang.Thread.run(Thread.java:680) > > > > Exception in thread "Thread-1" java.lang.RuntimeException: > > > > java.io.IOException: > > at data.BaseXDataAccess.save(BaseXDataAccess.java:164) > > > > at data.TestInsert$Insert.run(TestInsert.java:88)ERROR: > > com.xftec.hyperion.data.BaseXDataAccess - Error closing db > > > > I started several threads each one takes a BaseXClient
connection
> > and as > > soon as it finishs its job BaseXClient is put in a > > queue again to be took by other thread. > > > > _______________________________________________ > > BaseX-Talk mailing list > > BaseX-Talk@mailman.uni-konstanz.de > > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
some hints, which may help reveal the problem:
On Sunday 24 June 2012 11:47:15 Willian Barcella wrote:
I close DB each insert, this is my code: ... public void save(String key, DocumentWrapper value) { BaseXClient baseXClient = null; try { //Open or Create database baseXClient = this.takeConnection(); baseXClient.execute(format(CHECK, value.getCollectionName())); InputStream is = XMLUtils.nodeToInputStream(value.getDocument());
is DocumentWrapper.getCollectionName() thread-safe?
baseXClient.replace(value.getId(), is); } catch (Exception e) {
you may want to log a possible exception here - otherwise, it will be silently ignored, since it occurs in a thread other than the main one.
throw new RuntimeException(e); } finally { try { baseXClient.execute(CLOSE); if (baseXClient != null) this.giveBack(baseXClient); } catch (IOException e) { logger.error("Error closing db", e); } } }...
What you mean "single writer"?
I mean that, the basex server has internal lock mechanism, which blocks all sessions willing to execute an updating query, and effectively, it executes them sequentially, i.e. one after another. this means that basex does not support parallel execution of update queries.
regards, dimitar
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov <
dimitar.popov@uni-konstanz.de> wrote:
check if you close the database after each insert operation. you should also know that currently, basex allows only a single writer, i.e. if you use multiple threads just to insert of the documents faster, that won't happen.
regards, dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it. Now I am not getting errors on client side, but BaseX is not persisting
all
documents that I inserted. I created 20 BaseXClient's, 10 threads to take BasXClient from a LinkedBlockingQueue, each thread was responsible to insert 1000 documents, total of documents 10 x 1000 = 10K, however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
christian.gruen@gmail.comwrote:
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient object was used for several concurrent tasks. As BaseXClient objects are very light-weight, and as the registration of new clients is usually more than fast enough, the usual approach is to create a new BaseXClient instance for a dedicated action.
Hope this helps, Christian _______________________________
I builded a kind of connection pool using BaseXClient.java class, I simulated several users trying insert documents in BaseX, the result was a disaster. ERROR: data.BaseXDataAccess - Error closing db java.io.IOException: Stopped at line 1, column 19: Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99) at data.BaseXClient.execute(BaseXClient.java:110) at data.BaseXDataAccess.save(BaseXDataAccess.java:169) at data.TestInsert$Insert.run(TestInsert.java:88) at java.lang.Thread.run(Thread.java:680)Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException: at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR:com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection
and as
soon as it finishs its job BaseXClient is put in a queue again to be took by other thread.
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Hi Dimitar,
On Sun, Jun 24, 2012 at 12:17 PM, Dimitar Popov < dimitar.popov@uni-konstanz.de> wrote:
**
some hints, which may help reveal the problem:
On Sunday 24 June 2012 11:47:15 Willian Barcella wrote:
I close DB each insert, this is my code:
...
public void save(String key, DocumentWrapper value)
{
BaseXClient baseXClient = null;
try
{
//Open or Create database
baseXClient = this.takeConnection();
baseXClient.execute(format(CHECK, value.getCollectionName()));
InputStream is =
XMLUtils.nodeToInputStream(value.getDocument());
is DocumentWrapper.getCollectionName() thread-safe?
*It is not, but I am creating a new object everytime. *
baseXClient.replace(value.getId(), is);
} catch (Exception e)
{
you may want to log a possible exception here - otherwise, it will be silently ignored, since it occurs in a thread other than the main one.
*I inserted a log in this point you mentioned, didnt catch nothing *
throw new RuntimeException(e);
} finally
{
try
{
baseXClient.execute(CLOSE);
if (baseXClient != null)
this.giveBack(baseXClient);
} catch (IOException e)
{
logger.error("Error closing db", e);
}
}
}
...
What you mean "single writer"?
I mean that, the basex server has internal lock mechanism, which blocks all sessions willing to execute an updating query, and effectively, it executes them sequentially, i.e. one after another. this means that basex does not support parallel execution of update queries.
*Even BaseX does not support parallel updates, in server side it shouldn't lose documents or it should throw an exception to client side saying that document was not inserted. *
regards,
dimitar
On Sun, Jun 24, 2012 at 11:38 AM, Dimitar Popov <
dimitar.popov@uni-konstanz.de> wrote:
check if you close the database after each insert operation. you should
also
know that currently, basex allows only a single writer, i.e. if you use
multiple threads just to insert of the documents faster, that won't
happen.
regards,
dimitar
On Sunday 24 June 2012 11:24:00 Willian Barcella wrote:
There was an error in my code, I fixed it.
Now I am not getting errors on client side, but BaseX is not
persisting
all
documents that I inserted.
I created 20 BaseXClient's, 10 threads to take BasXClient from a
LinkedBlockingQueue, each thread was
responsible to insert 1000 documents, total of documents 10 x 1000 =
10K,
however was inserted 9801.
Regards
On Sun, Jun 24, 2012 at 10:48 AM, Christian Grün
Hi Willian,
thanks for your mail. I would assume that one single BaseXClient
object was used for several concurrent tasks. As BaseXClient
objects
are very light-weight, and as the registration of new clients is
usually more than fast enough, the usual approach is to create a
new
BaseXClient instance for a dedicated action.
Hope this helps,
Christian
I builded a kind of connection pool using BaseXClient.java
class, I
simulated several users trying insert documents in BaseX,
the result was a disaster.
ERROR: data.BaseXDataAccess - Error closing db
java.io.IOException: Stopped at line 1, column 19:
Unknown command: 1340477810253095000. Try HELP.
at data.BaseXClient.execute(BaseXClient.java:99)
at data.BaseXClient.execute(BaseXClient.java:110)
at data.BaseXDataAccess.save(BaseXDataAccess.java:169)
at data.TestInsert$Insert.run(TestInsert.java:88)
at java.lang.Thread.run(Thread.java:680)
Exception in thread "Thread-1" java.lang.RuntimeException:
java.io.IOException:
at data.BaseXDataAccess.save(BaseXDataAccess.java:164)
at data.TestInsert$Insert.run(TestInsert.java:88)ERROR:
com.xftec.hyperion.data.BaseXDataAccess - Error closing db
I started several threads each one takes a BaseXClient connection
and as
soon as it finishs its job BaseXClient is put in a
queue again to be took by other thread.
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
BaseX-Talk mailing list
BaseX-Talk@mailman.uni-konstanz.de
On Sunday 24 June 2012 12:30:08 Willian Barcella wrote:
What you mean "single writer"?
I mean that, the basex server has internal lock mechanism, which blocks all sessions willing to execute an updating query, and effectively, it executes them sequentially, i.e. one after another. this means that basex does not support parallel execution of update queries.
*Even BaseX does not support parallel updates, in server side it shouldn't lose documents or it should throw an exception to client side saying that document was not inserted. *
agreed. can you put a static counter in your class, which is incremented each time the save method is called. this can help isolate the cause: if the value of the counter is different from 10000, then the problem is somewhere in the thread management code; otherwise, it has to be somewhere in the baseX code.
regards, dimitar
I am using CountDownLatch, and I decrement each call to save()..
On Sun, Jun 24, 2012 at 2:12 PM, Dimitar Popov < dimitar.popov@uni-konstanz.de> wrote:
On Sunday 24 June 2012 12:30:08 Willian Barcella wrote:
What you mean "single writer"?
I mean that, the basex server has internal lock mechanism, which blocks all sessions willing to execute an updating query, and effectively, it executes them sequentially, i.e. one after another. this means that
basex
does not support parallel execution of update queries.
*Even BaseX does not support parallel updates, in server side it
shouldn't
lose documents or it should throw an exception to client side saying that document was not inserted. *
agreed. can you put a static counter in your class, which is incremented each time the save method is called. this can help isolate the cause: if the value of the counter is different from 10000, then the problem is somewhere in the thread management code; otherwise, it has to be somewhere in the baseX code.
regards, dimitar
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Dimitar Popov -
Willian Barcella