Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads. Am I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession => T): T = { withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result } }
then:
withDbSession { session => // many calls to session.add(...) }
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote:
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads. Am I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession => T): T = { withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result } }
then:
withDbSession { session => // many calls to session.add(...) }
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
The fact that it's scala should be insignificant, since I'm using the Java API.
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion times session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no problems "in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün christian.gruen@gmail.com wrote:
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote:
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads. Am I
wrong
about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession => T): T =
{
withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result }}
then:
withDbSession { session => // many calls to session.add(...) }
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
The fact that it's scala should be insignificant, since I'm using the Java API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1] https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba... [2] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b... [3] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion times session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no problems "in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün christian.gruen@gmail.com wrote:
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote:
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads. Am I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession => T): T = { withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result } }
then:
withDbSession { session => // many calls to session.add(...) }
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
Thanks Christian. I will try to find out how I can possibly cause an IllegalMonitorStateException.
Looking at these tests, it appears that none of them do large numbers of ADD commands. Did I miss something?
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote:
The fact that it's scala should be insignificant, since I'm using the
Java
API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1] https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba... [2] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b... [3] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion times session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no
problems
"in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote:
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I
would
expect it to be possible to add documents from multiple threads. Am I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the
same
time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295"
java.lang.IllegalMonitorStateException
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession => T):
T =
{ withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result } }
then:
withDbSession { session => // many calls to session.add(...) }
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
Looking at these tests, it appears that none of them do large numbers of ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote:
The fact that it's scala should be insignificant, since I'm using the Java API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1] https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba... [2] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b... [3] https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion times session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no problems "in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün christian.gruen@gmail.com wrote:
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote:
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads. Am I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion times session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession => T): T = { withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result } }
then:
withDbSession { session => // many calls to session.add(...) }
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
Can I start up the basexserver with some kind of logging so I can see what leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün christian.gruen@gmail.com wrote:
Looking at these tests, it appears that none of them do large numbers of
ADD
commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
The fact that it's scala should be insignificant, since I'm using the Java API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1]
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba...
[2]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
[3]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion times session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no problems "in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün christian.gruen@gmail.com wrote:
...difficult to tell, as we don't have too much experience with
Scala.
Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote:
Dear BaseX experts,
I'm wondering if anybody can see something that I'm doing wrong. I would expect it to be possible to add documents from multiple threads.
Am
I wrong about that??
When I set two of my datasets saving documents to BaseX 7.9 at the same time, I get this exception:
$ basexserver BaseX 7.9 [Server] Server was started (port: 1984) Exception in thread "Thread-2295" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104) at org.basex.core.Command.execute(Command.java:117) at org.basex.server.ClientListener.execute(ClientListener.java:384) at org.basex.server.ClientListener.add(ClientListener.java:355) at org.basex.server.ClientListener.run(ClientListener.java:100)
All I am doing is making concurrent calls in two parallel sessions:
session = new ClientSession(...) session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) session.add([path string], [document-bytes]) // a zillion
times
session.execute(new Flush())
During the execution of the parallel add commands, it crashes.
The relevant code:
def withSession[T](block: ClientSession => T): T = { val session = new ClientSession(host, port, user, pass) try { block(session) } finally { session.close() } }
def withDbSession[T](database: String)(block: ClientSession =>
T):
T = { withSession { session => session.execute(new Open(database)) session.execute(new Set("autoflush", "false")) val result = block(session) session.execute(new Flush()) result } }
then:
withDbSession { session => // many calls to session.add(...) }
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
You can set the debugging flag and check out the database log files to get more insight into what's going on.
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong gerald@delving.eu wrote:
Can I start up the basexserver with some kind of logging so I can see what leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün christian.gruen@gmail.com wrote:
Looking at these tests, it appears that none of them do large numbers of ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote:
The fact that it's scala should be insignificant, since I'm using the Java API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1]
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba... [2]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b... [3]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
What's happening is that two (or more) threads are doing this at the same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion times session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no problems "in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün christian.gruen@gmail.com wrote:
...difficult to tell, as we don't have too much experience with Scala. Could you provide this example as Java code?
On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong gerald@delving.eu wrote: > Dear BaseX experts, > > I'm wondering if anybody can see something that I'm doing wrong. I > would > expect it to be possible to add documents from multiple threads. > Am > I > wrong > about that?? > > When I set two of my datasets saving documents to BaseX 7.9 at the > same > time, I get this exception: > > $ basexserver > BaseX 7.9 [Server] > Server was started (port: 1984) > Exception in thread "Thread-2295" > java.lang.IllegalMonitorStateException > at > > > > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) > at > > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) > at > > > > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) > at org.basex.core.DBLocking.release(DBLocking.java:219) > at org.basex.core.Context.unregister(Context.java:249) > at org.basex.core.Command.execute(Command.java:104) > at org.basex.core.Command.execute(Command.java:117) > at > org.basex.server.ClientListener.execute(ClientListener.java:384) > at org.basex.server.ClientListener.add(ClientListener.java:355) > at org.basex.server.ClientListener.run(ClientListener.java:100) > > All I am doing is making concurrent calls in two parallel > sessions: > > session = new ClientSession(...) > session.execute(new Open(database)) > session.execute(new Set("autoflush", "false")) > session.add([path string], [document-bytes]) // a zillion > times > session.execute(new Flush()) > > During the execution of the parallel add commands, it crashes. > > The relevant code: > > def withSession[T](block: ClientSession => T): T = { > val session = new ClientSession(host, port, user, pass) > try { > block(session) > } > finally { > session.close() > } > } > > def withDbSession[T](database: String)(block: ClientSession => > T): > T = > { > withSession { > session => > session.execute(new Open(database)) > session.execute(new Set("autoflush", "false")) > val result = block(session) > session.execute(new Flush()) > result > } > } > > then: > > withDbSession { > session => > // many calls to session.add(...) > } > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
It happened again. Could I not argue that nothing I can do as a client to BaseX should be able to freeze it? I mean, even if, for some reason, things are getting muddled up.
Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104)
On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün christian.gruen@gmail.com wrote:
You can set the debugging flag and check out the database log files to get more insight into what's going on.
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong gerald@delving.eu wrote:
Can I start up the basexserver with some kind of logging so I can see
what
leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Looking at these tests, it appears that none of them do large numbers
of
ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote:
The fact that it's scala should be insignificant, since I'm using
the
Java API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1]
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba...
[2]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
[3]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
What's happening is that two (or more) threads are doing this at
the
same time:
ClientSession session = new ClientSession(...); session.execute(new Open(database)); session.execute(new Set("autoflush", "false")); session.add([path string], [document-bytes]); // a zillion
times
session.execute(new Flush()); session.close();
My first question is of course if this should work parallel with no problems "in theory".
On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün christian.gruen@gmail.com wrote: > > ...difficult to tell, as we don't have too much experience with > Scala. > Could you provide this example as Java code? > > > > On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <
gerald@delving.eu>
> wrote: > > Dear BaseX experts, > > > > I'm wondering if anybody can see something that I'm doing
wrong. I
> > would > > expect it to be possible to add documents from multiple threads. > > Am > > I > > wrong > > about that?? > > > > When I set two of my datasets saving documents to BaseX 7.9 at
the
> > same > > time, I get this exception: > > > > $ basexserver > > BaseX 7.9 [Server] > > Server was started (port: 1984) > > Exception in thread "Thread-2295" > > java.lang.IllegalMonitorStateException > > at > > > > > > > >
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
> > at > > > > > > > >
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
> > at > > > > > > > >
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
> > at org.basex.core.DBLocking.release(DBLocking.java:219) > > at org.basex.core.Context.unregister(Context.java:249) > > at org.basex.core.Command.execute(Command.java:104) > > at org.basex.core.Command.execute(Command.java:117) > > at > > org.basex.server.ClientListener.execute(ClientListener.java:384) > > at org.basex.server.ClientListener.add(ClientListener.java:355) > > at org.basex.server.ClientListener.run(ClientListener.java:100) > > > > All I am doing is making concurrent calls in two parallel > > sessions: > > > > session = new ClientSession(...) > > session.execute(new Open(database)) > > session.execute(new Set("autoflush", "false")) > > session.add([path string], [document-bytes]) // a zillion > > times > > session.execute(new Flush()) > > > > During the execution of the parallel add commands, it crashes. > > > > The relevant code: > > > > def withSession[T](block: ClientSession => T): T = { > > val session = new ClientSession(host, port, user, pass) > > try { > > block(session) > > } > > finally { > > session.close() > > } > > } > > > > def withDbSession[T](database: String)(block: ClientSession => > > T): > > T = > > { > > withSession { > > session => > > session.execute(new Open(database)) > > session.execute(new Set("autoflush", "false")) > > val result = block(session) > > session.execute(new Flush()) > > result > > } > > } > > > > then: > > > > withDbSession { > > session => > > // many calls to session.add(...) > > } > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
Hello Gerald,
I completely agree. I guess, one could consider this a bug (but of course, by definition, BaseX does not has bugs; it's hidden features). Seems like something is going wrong when releasing the write lock. Could you please test it with our latest snapshot, to see if the issue was already fixed?
Christian is currently on vacation, so an actual bug fix could take some more time than usual. If I have time, I'll try to look into it.
Cheers, Dirk
On 09/24/2014 05:41 PM, Gerald de Jong wrote:
It happened again. Could I not argue that nothing I can do as a client to BaseX should be able to freeze it? I mean, even if, for some reason, things are getting muddled up.
Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104)
On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün christian.gruen@gmail.com wrote:
You can set the debugging flag and check out the database log files to get more insight into what's going on.
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong gerald@delving.eu wrote:
Can I start up the basexserver with some kind of logging so I can see
what
leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Looking at these tests, it appears that none of them do large numbers
of
ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote:
> The fact that it's scala should be insignificant, since I'm using
the
> Java > API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1]
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba...
[2]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
[3]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
> > What's happening is that two (or more) threads are doing this at
the
> same > time: > > ClientSession session = new ClientSession(...); > session.execute(new Open(database)); > session.execute(new Set("autoflush", "false")); > session.add([path string], [document-bytes]); // a zillion
times
> session.execute(new Flush()); > session.close(); > > My first question is of course if this should work parallel with no > problems > "in theory". > > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > christian.gruen@gmail.com > wrote: >> >> ...difficult to tell, as we don't have too much experience with >> Scala. >> Could you provide this example as Java code? >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <
gerald@delving.eu>
>> wrote: >>> Dear BaseX experts, >>> >>> I'm wondering if anybody can see something that I'm doing
wrong. I
>>> would >>> expect it to be possible to add documents from multiple threads. >>> Am >>> I >>> wrong >>> about that?? >>> >>> When I set two of my datasets saving documents to BaseX 7.9 at
the
>>> same >>> time, I get this exception: >>> >>> $ basexserver >>> BaseX 7.9 [Server] >>> Server was started (port: 1984) >>> Exception in thread "Thread-2295" >>> java.lang.IllegalMonitorStateException >>> at >>> >>> >>> >>>
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
>>> at >>> >>> >>> >>>
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
>>> at >>> >>> >>> >>>
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
>>> at org.basex.core.DBLocking.release(DBLocking.java:219) >>> at org.basex.core.Context.unregister(Context.java:249) >>> at org.basex.core.Command.execute(Command.java:104) >>> at org.basex.core.Command.execute(Command.java:117) >>> at >>> org.basex.server.ClientListener.execute(ClientListener.java:384) >>> at org.basex.server.ClientListener.add(ClientListener.java:355) >>> at org.basex.server.ClientListener.run(ClientListener.java:100) >>> >>> All I am doing is making concurrent calls in two parallel >>> sessions: >>> >>> session = new ClientSession(...) >>> session.execute(new Open(database)) >>> session.execute(new Set("autoflush", "false")) >>> session.add([path string], [document-bytes]) // a zillion >>> times >>> session.execute(new Flush()) >>> >>> During the execution of the parallel add commands, it crashes. >>> >>> The relevant code: >>> >>> def withSession[T](block: ClientSession => T): T = { >>> val session = new ClientSession(host, port, user, pass) >>> try { >>> block(session) >>> } >>> finally { >>> session.close() >>> } >>> } >>> >>> def withDbSession[T](database: String)(block: ClientSession => >>> T): >>> T = >>> { >>> withSession { >>> session => >>> session.execute(new Open(database)) >>> session.execute(new Set("autoflush", "false")) >>> val result = block(session) >>> session.execute(new Flush()) >>> result >>> } >>> } >>> >>> then: >>> >>> withDbSession { >>> session => >>> // many calls to session.add(...) >>> } >>> >>> -- >>> Delving BV, Vasteland 8, Rotterdam >>> http://www.delving.eu >>> http://twitter.com/fluxe >>> skype: beautifulcode >>> +31629339805 > > > > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
I'm glad you agree! I am not able to reproduce it consistently, and I will certainly try out the latest snapshot as much as possible.
On Fri, Sep 26, 2014 at 12:59 PM, Dirk Kirsten dk@basex.org wrote:
Hello Gerald,
I completely agree. I guess, one could consider this a bug (but of course, by definition, BaseX does not has bugs; it's hidden features). Seems like something is going wrong when releasing the write lock. Could you please test it with our latest snapshot, to see if the issue was already fixed?
Christian is currently on vacation, so an actual bug fix could take some more time than usual. If I have time, I'll try to look into it.
Cheers, Dirk
On 09/24/2014 05:41 PM, Gerald de Jong wrote:
It happened again. Could I not argue that nothing I can do as a client
to
BaseX should be able to freeze it? I mean, even if, for some reason, things are getting muddled up.
Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104)
On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
You can set the debugging flag and check out the database log files to get more insight into what's going on.
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong gerald@delving.eu
wrote:
Can I start up the basexserver with some kind of logging so I can see
what
leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Looking at these tests, it appears that none of them do large numbers
of
ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote: > >> The fact that it's scala should be insignificant, since I'm using
the
>> Java >> API. > > Well, it might be theoretically insignificant, but we cannot give a > quick answer on what's your problem. In general, parallel client > requests are no problem, we have various JUnit tests doing exactly > that (see e.g. [1], [2] or [3]). > > [1] > >
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba...
> [2] > >
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
> [3] > >
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
> > >> >> What's happening is that two (or more) threads are doing this at
the
>> same >> time: >> >> ClientSession session = new ClientSession(...); >> session.execute(new Open(database)); >> session.execute(new Set("autoflush", "false")); >> session.add([path string], [document-bytes]); // a zillion
times
>> session.execute(new Flush()); >> session.close(); >> >> My first question is of course if this should work parallel with no >> problems >> "in theory". >> >> On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün >> christian.gruen@gmail.com >> wrote: >>> >>> ...difficult to tell, as we don't have too much experience with >>> Scala. >>> Could you provide this example as Java code? >>> >>> >>> >>> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong <
gerald@delving.eu>
>>> wrote: >>>> Dear BaseX experts, >>>> >>>> I'm wondering if anybody can see something that I'm doing
wrong. I
>>>> would >>>> expect it to be possible to add documents from multiple threads. >>>> Am >>>> I >>>> wrong >>>> about that?? >>>> >>>> When I set two of my datasets saving documents to BaseX 7.9 at
the
>>>> same >>>> time, I get this exception: >>>> >>>> $ basexserver >>>> BaseX 7.9 [Server] >>>> Server was started (port: 1984) >>>> Exception in thread "Thread-2295" >>>> java.lang.IllegalMonitorStateException >>>> at >>>> >>>> >>>> >>>>
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
>>>> at >>>> >>>> >>>> >>>>
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
>>>> at >>>> >>>> >>>> >>>>
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
>>>> at org.basex.core.DBLocking.release(DBLocking.java:219) >>>> at org.basex.core.Context.unregister(Context.java:249) >>>> at org.basex.core.Command.execute(Command.java:104) >>>> at org.basex.core.Command.execute(Command.java:117) >>>> at >>>> org.basex.server.ClientListener.execute(ClientListener.java:384) >>>> at org.basex.server.ClientListener.add(ClientListener.java:355) >>>> at org.basex.server.ClientListener.run(ClientListener.java:100) >>>> >>>> All I am doing is making concurrent calls in two parallel >>>> sessions: >>>> >>>> session = new ClientSession(...) >>>> session.execute(new Open(database)) >>>> session.execute(new Set("autoflush", "false")) >>>> session.add([path string], [document-bytes]) // a zillion >>>> times >>>> session.execute(new Flush()) >>>> >>>> During the execution of the parallel add commands, it crashes. >>>> >>>> The relevant code: >>>> >>>> def withSession[T](block: ClientSession => T): T = { >>>> val session = new ClientSession(host, port, user, pass) >>>> try { >>>> block(session) >>>> } >>>> finally { >>>> session.close() >>>> } >>>> } >>>> >>>> def withDbSession[T](database: String)(block: ClientSession => >>>> T): >>>> T = >>>> { >>>> withSession { >>>> session => >>>> session.execute(new Open(database)) >>>> session.execute(new Set("autoflush", "false")) >>>> val result = block(session) >>>> session.execute(new Flush()) >>>> result >>>> } >>>> } >>>> >>>> then: >>>> >>>> withDbSession { >>>> session => >>>> // many calls to session.add(...) >>>> } >>>> >>>> -- >>>> Delving BV, Vasteland 8, Rotterdam >>>> http://www.delving.eu >>>> http://twitter.com/fluxe >>>> skype: beautifulcode >>>> +31629339805 >> >> >> >> >> -- >> Delving BV, Vasteland 8, Rotterdam >> http://www.delving.eu >> http://twitter.com/fluxe >> skype: beautifulcode >> +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22
Hi Gerald,
did you manage to write some Java code that causes the same problems? I am afraid that this is currently all I can offer.
Christian
On Wed, Sep 24, 2014 at 5:41 PM, Gerald de Jong gerald@delving.eu wrote:
It happened again. Could I not argue that nothing I can do as a client to BaseX should be able to freeze it? I mean, even if, for some reason, things are getting muddled up.
Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException at java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104)
On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün christian.gruen@gmail.com wrote:
You can set the debugging flag and check out the database log files to get more insight into what's going on.
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong gerald@delving.eu wrote:
Can I start up the basexserver with some kind of logging so I can see what leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün christian.gruen@gmail.com wrote:
Looking at these tests, it appears that none of them do large numbers of ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote:
> The fact that it's scala should be insignificant, since I'm using > the > Java > API.
Well, it might be theoretically insignificant, but we cannot give a quick answer on what's your problem. In general, parallel client requests are no problem, we have various JUnit tests doing exactly that (see e.g. [1], [2] or [3]).
[1]
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba... [2]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b... [3]
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
> > What's happening is that two (or more) threads are doing this at > the > same > time: > > ClientSession session = new ClientSession(...); > session.execute(new Open(database)); > session.execute(new Set("autoflush", "false")); > session.add([path string], [document-bytes]); // a zillion > times > session.execute(new Flush()); > session.close(); > > My first question is of course if this should work parallel with > no > problems > "in theory". > > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > christian.gruen@gmail.com > wrote: >> >> ...difficult to tell, as we don't have too much experience with >> Scala. >> Could you provide this example as Java code? >> >> >> >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong >> gerald@delving.eu >> wrote: >> > Dear BaseX experts, >> > >> > I'm wondering if anybody can see something that I'm doing >> > wrong. I >> > would >> > expect it to be possible to add documents from multiple >> > threads. >> > Am >> > I >> > wrong >> > about that?? >> > >> > When I set two of my datasets saving documents to BaseX 7.9 at >> > the >> > same >> > time, I get this exception: >> > >> > $ basexserver >> > BaseX 7.9 [Server] >> > Server was started (port: 1984) >> > Exception in thread "Thread-2295" >> > java.lang.IllegalMonitorStateException >> > at >> > >> > >> > >> > >> > java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374) >> > at >> > >> > >> > >> > >> > java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260) >> > at >> > >> > >> > >> > >> > java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131) >> > at org.basex.core.DBLocking.release(DBLocking.java:219) >> > at org.basex.core.Context.unregister(Context.java:249) >> > at org.basex.core.Command.execute(Command.java:104) >> > at org.basex.core.Command.execute(Command.java:117) >> > at >> > >> > org.basex.server.ClientListener.execute(ClientListener.java:384) >> > at org.basex.server.ClientListener.add(ClientListener.java:355) >> > at org.basex.server.ClientListener.run(ClientListener.java:100) >> > >> > All I am doing is making concurrent calls in two parallel >> > sessions: >> > >> > session = new ClientSession(...) >> > session.execute(new Open(database)) >> > session.execute(new Set("autoflush", "false")) >> > session.add([path string], [document-bytes]) // a zillion >> > times >> > session.execute(new Flush()) >> > >> > During the execution of the parallel add commands, it crashes. >> > >> > The relevant code: >> > >> > def withSession[T](block: ClientSession => T): T = { >> > val session = new ClientSession(host, port, user, pass) >> > try { >> > block(session) >> > } >> > finally { >> > session.close() >> > } >> > } >> > >> > def withDbSession[T](database: String)(block: ClientSession >> > => >> > T): >> > T = >> > { >> > withSession { >> > session => >> > session.execute(new Open(database)) >> > session.execute(new Set("autoflush", "false")) >> > val result = block(session) >> > session.execute(new Flush()) >> > result >> > } >> > } >> > >> > then: >> > >> > withDbSession { >> > session => >> > // many calls to session.add(...) >> > } >> > >> > -- >> > Delving BV, Vasteland 8, Rotterdam >> > http://www.delving.eu >> > http://twitter.com/fluxe >> > skype: beautifulcode >> > +31629339805 > > > > > -- > Delving BV, Vasteland 8, Rotterdam > http://www.delving.eu > http://twitter.com/fluxe > skype: beautifulcode > +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
Thanks, Christian. I haven't seen it crash recently, but I'm watching closely.
I think I will also take another approach to getting the bulk of the data into BaseX. The slowdown that I was talking about earlier was a little annoying (haven't tried yet with the latest snapshot, sorry), and my approach of storing one document per "record" (there are millions) might not be the smartest.
So now I've built a module which builds a single big text file of those million records, and when I load that with a single BaseX command it's much much faster. The only thing I'm missing is a nice progress bar, but I can probably live without that.
On Mon, Oct 6, 2014 at 2:34 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Gerald,
did you manage to write some Java code that causes the same problems? I am afraid that this is currently all I can offer.
Christian
On Wed, Sep 24, 2014 at 5:41 PM, Gerald de Jong gerald@delving.eu wrote:
It happened again. Could I not argue that nothing I can do as a client
to
BaseX should be able to freeze it? I mean, even if, for some reason,
things
are getting muddled up.
Exception in thread "Thread-56391" java.lang.IllegalMonitorStateException at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
at org.basex.core.DBLocking.release(DBLocking.java:219) at org.basex.core.Context.unregister(Context.java:249) at org.basex.core.Command.execute(Command.java:104)
On Tue, Sep 23, 2014 at 2:17 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
You can set the debugging flag and check out the database log files to get more insight into what's going on.
On Tue, Sep 23, 2014 at 2:15 PM, Gerald de Jong gerald@delving.eu
wrote:
Can I start up the basexserver with some kind of logging so I can see what leads to the IllegalMonitorStateException?
On Tue, Sep 23, 2014 at 1:53 PM, Christian Grün christian.gruen@gmail.com wrote:
Looking at these tests, it appears that none of them do large
numbers
of ADD commands. Did I miss something?
You may need to check the other tests. But as the (updating) add command will never be executed in parallel, it shouldn't cause any problems with parallelization (feel free to tell me if I'm wrong).
On Tue, Sep 23, 2014 at 1:34 PM, Christian Grün christian.gruen@gmail.com wrote: > > > The fact that it's scala should be insignificant, since I'm
using
> > the > > Java > > API. > > Well, it might be theoretically insignificant, but we cannot give
a
> quick answer on what's your problem. In general, parallel client > requests are no problem, we have various JUnit tests doing exactly > that (see e.g. [1], [2] or [3]). > > [1] > > >
https://github.com/BaseXdb/basex/blob/master/basex-core/src/test/java/org/ba...
> [2] > > >
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
> [3] > > >
https://github.com/BaseXdb/basex/blob/master/basex-tests/src/test/java/org/b...
> > > > > > What's happening is that two (or more) threads are doing this at > > the > > same > > time: > > > > ClientSession session = new ClientSession(...); > > session.execute(new Open(database)); > > session.execute(new Set("autoflush", "false")); > > session.add([path string], [document-bytes]); // a zillion > > times > > session.execute(new Flush()); > > session.close(); > > > > My first question is of course if this should work parallel with > > no > > problems > > "in theory". > > > > On Tue, Sep 23, 2014 at 12:46 PM, Christian Grün > > christian.gruen@gmail.com > > wrote: > >> > >> ...difficult to tell, as we don't have too much experience with > >> Scala. > >> Could you provide this example as Java code? > >> > >> > >> > >> On Tue, Sep 23, 2014 at 12:10 PM, Gerald de Jong > >> gerald@delving.eu > >> wrote: > >> > Dear BaseX experts, > >> > > >> > I'm wondering if anybody can see something that I'm doing > >> > wrong. I > >> > would > >> > expect it to be possible to add documents from multiple > >> > threads. > >> > Am > >> > I > >> > wrong > >> > about that?? > >> > > >> > When I set two of my datasets saving documents to BaseX 7.9
at
> >> > the > >> > same > >> > time, I get this exception: > >> > > >> > $ basexserver > >> > BaseX 7.9 [Server] > >> > Server was started (port: 1984) > >> > Exception in thread "Thread-2295" > >> > java.lang.IllegalMonitorStateException > >> > at > >> > > >> > > >> > > >> > > >> >
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryRelease(ReentrantReadWriteLock.java:374)
> >> > at > >> > > >> > > >> > > >> > > >> >
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1260)
> >> > at > >> > > >> > > >> > > >> > > >> >
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.unlock(ReentrantReadWriteLock.java:1131)
> >> > at org.basex.core.DBLocking.release(DBLocking.java:219) > >> > at org.basex.core.Context.unregister(Context.java:249) > >> > at org.basex.core.Command.execute(Command.java:104) > >> > at org.basex.core.Command.execute(Command.java:117) > >> > at > >> > > >> >
org.basex.server.ClientListener.execute(ClientListener.java:384)
> >> > at
org.basex.server.ClientListener.add(ClientListener.java:355)
> >> > at
org.basex.server.ClientListener.run(ClientListener.java:100)
> >> > > >> > All I am doing is making concurrent calls in two parallel > >> > sessions: > >> > > >> > session = new ClientSession(...) > >> > session.execute(new Open(database)) > >> > session.execute(new Set("autoflush", "false")) > >> > session.add([path string], [document-bytes]) // a
zillion
> >> > times > >> > session.execute(new Flush()) > >> > > >> > During the execution of the parallel add commands, it
crashes.
> >> > > >> > The relevant code: > >> > > >> > def withSession[T](block: ClientSession => T): T = { > >> > val session = new ClientSession(host, port, user, pass) > >> > try { > >> > block(session) > >> > } > >> > finally { > >> > session.close() > >> > } > >> > } > >> > > >> > def withDbSession[T](database: String)(block: ClientSession > >> > => > >> > T): > >> > T = > >> > { > >> > withSession { > >> > session => > >> > session.execute(new Open(database)) > >> > session.execute(new Set("autoflush", "false")) > >> > val result = block(session) > >> > session.execute(new Flush()) > >> > result > >> > } > >> > } > >> > > >> > then: > >> > > >> > withDbSession { > >> > session => > >> > // many calls to session.add(...) > >> > } > >> > > >> > -- > >> > Delving BV, Vasteland 8, Rotterdam > >> > http://www.delving.eu > >> > http://twitter.com/fluxe > >> > skype: beautifulcode > >> > +31629339805 > > > > > > > > > > -- > > Delving BV, Vasteland 8, Rotterdam > > http://www.delving.eu > > http://twitter.com/fluxe > > skype: beautifulcode > > +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
-- Delving BV, Vasteland 8, Rotterdam http://www.delving.eu http://twitter.com/fluxe skype: beautifulcode +31629339805
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Dirk Kirsten -
Gerald de Jong