Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these sources and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection. The metadata of all collections are stored into a common collection. Each source is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant, could be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real application and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote:
Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these sources and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection. The metadata of all collections are stored into a common collection. Each source is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant, could be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real application and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
Hi Christian,
Thanks for your answer. I am back in my office only today, and try to implement your recommendation: making sure that every sessions is created and used by only one thread.
It seems that the java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
does not occur anymore.
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find
is still happening.
What I did is basically, in the test application, replace all session creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
} });
Not very elegant but should be enough to make sure every session is executed in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote:
Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these
sources
and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection. The metadata of all collections are stored into a common collection. Each
source
is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant,
could
be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real
application
and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find is still happening.
I am pretty sure that any of the concurrent command calls are not moved to their own thread yet.
But feel free to pass me on your code (ideally, a version that's as small as possible).
Best, Christian
What I did is basically, in the test application, replace all session creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
}});
Not very elegant but should be enough to make sure every session is executed in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote:
Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these sources and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection. The metadata of all collections are stored into a common collection. Each source is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant, could be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real application and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
Hello Christian,
I checked again my code, but I am quite sure that each LocalSession is created and used by only one thread. I attach the test application code.
Each LocalSession is used to execute several commands, but that shouldn't be a problem, right ?
I also had a look in BaseX code, especially in the org.basex.core.users.User class.
What happens in my test application is that I create several Writer that will add XML document each in a specific collection, but using the same user. Each time I create a new Writer it will first create the collection then grant read and write permission to this collection to the user "USER", and then start adding XML document in this collection.
My suspicion is that the problem occurs when a new Writer is created, and it is granting permission to the newly created collection while another Writer (in another thread) is adding a new XML document in its own collection. If, during the execution of the Grant command, the method User#perm(final Perm prm, final String pattern) is called while the execution of a concurrent Write command is at the same time executing the User#find(final String db) method, then we have a ConcurrentModificationException as perm method will put a new entry in locals field while the find method is iterating on the locals entrySet of the same User, the same User instance being shared by all subcontext created when opening a new LocalSession. For test purposes, I synchronized the access to locals in the perm(), find() and remove() methods of the class User, in that case I do not have the ConcurrentModificationException any more.
What do you think, is this possible ?
Regards
Simon
On Mon, Apr 13, 2015 at 11:04 AM, Christian Grün christian.gruen@gmail.com wrote:
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find is still happening.
I am pretty sure that any of the concurrent command calls are not moved to their own thread yet.
But feel free to pass me on your code (ideally, a version that's as small as possible).
Best, Christian
What I did is basically, in the test application, replace all session creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
}});
Not very elegant but should be enough to make sure every session is
executed
in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote:
Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these sources and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection.
The
metadata of all collections are stored into a common collection. Each source is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant, could be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real application and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
Hi Simon,
Thanks for the updated code. It's still quite a lot code.. Is all of it required to reproduce the bug? E.g., does the bug only occur with the specified options etc? Could you possibly reduce it even more? Idealle, we can later rewrite it to a JUnit test once the problem is fixed (and we try to keep our tests as concise as possible).
But I'll have some thoughts about your helpful assumptions in parallel.
Thanks in advance, Christian
On Tue, Apr 14, 2015 at 1:10 PM, Simon Chatelain schatela@gmail.com wrote:
Hello Christian,
I checked again my code, but I am quite sure that each LocalSession is created and used by only one thread. I attach the test application code.
Each LocalSession is used to execute several commands, but that shouldn't be a problem, right ?
I also had a look in BaseX code, especially in the org.basex.core.users.User class.
What happens in my test application is that I create several Writer that will add XML document each in a specific collection, but using the same user. Each time I create a new Writer it will first create the collection then grant read and write permission to this collection to the user "USER", and then start adding XML document in this collection.
My suspicion is that the problem occurs when a new Writer is created, and it is granting permission to the newly created collection while another Writer (in another thread) is adding a new XML document in its own collection. If, during the execution of the Grant command, the method User#perm(final Perm prm, final String pattern) is called while the execution of a concurrent Write command is at the same time executing the User#find(final String db) method, then we have a ConcurrentModificationException as perm method will put a new entry in locals field while the find method is iterating on the locals entrySet of the same User, the same User instance being shared by all subcontext created when opening a new LocalSession. For test purposes, I synchronized the access to locals in the perm(), find() and remove() methods of the class User, in that case I do not have the ConcurrentModificationException any more.
What do you think, is this possible ?
Regards
Simon
On Mon, Apr 13, 2015 at 11:04 AM, Christian Grün christian.gruen@gmail.com wrote:
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find is still happening.
I am pretty sure that any of the concurrent command calls are not moved to their own thread yet.
But feel free to pass me on your code (ideally, a version that's as small as possible).
Best, Christian
What I did is basically, in the test application, replace all session creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
}});
Not very elegant but should be enough to make sure every session is executed in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote:
Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these sources and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection. The metadata of all collections are stored into a common collection. Each source is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant, could be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real application and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
I have minimized your example a bit (attached), and I agree it seems to be the user iterator that causes surprises.
Looking at it.. Christian
On Tue, Apr 14, 2015 at 1:48 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the updated code. It's still quite a lot code.. Is all of it required to reproduce the bug? E.g., does the bug only occur with the specified options etc? Could you possibly reduce it even more? Idealle, we can later rewrite it to a JUnit test once the problem is fixed (and we try to keep our tests as concise as possible).
But I'll have some thoughts about your helpful assumptions in parallel.
Thanks in advance, Christian
On Tue, Apr 14, 2015 at 1:10 PM, Simon Chatelain schatela@gmail.com wrote:
Hello Christian,
I checked again my code, but I am quite sure that each LocalSession is created and used by only one thread. I attach the test application code.
Each LocalSession is used to execute several commands, but that shouldn't be a problem, right ?
I also had a look in BaseX code, especially in the org.basex.core.users.User class.
What happens in my test application is that I create several Writer that will add XML document each in a specific collection, but using the same user. Each time I create a new Writer it will first create the collection then grant read and write permission to this collection to the user "USER", and then start adding XML document in this collection.
My suspicion is that the problem occurs when a new Writer is created, and it is granting permission to the newly created collection while another Writer (in another thread) is adding a new XML document in its own collection. If, during the execution of the Grant command, the method User#perm(final Perm prm, final String pattern) is called while the execution of a concurrent Write command is at the same time executing the User#find(final String db) method, then we have a ConcurrentModificationException as perm method will put a new entry in locals field while the find method is iterating on the locals entrySet of the same User, the same User instance being shared by all subcontext created when opening a new LocalSession. For test purposes, I synchronized the access to locals in the perm(), find() and remove() methods of the class User, in that case I do not have the ConcurrentModificationException any more.
What do you think, is this possible ?
Regards
Simon
On Mon, Apr 13, 2015 at 11:04 AM, Christian Grün christian.gruen@gmail.com wrote:
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find is still happening.
I am pretty sure that any of the concurrent command calls are not moved to their own thread yet.
But feel free to pass me on your code (ideally, a version that's as small as possible).
Best, Christian
What I did is basically, in the test application, replace all session creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
}});
Not very elegant but should be enough to make sure every session is executed in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote:
Hello all,
I have the following problem.
Using:
BaseX 8.1.1 beta cf713e2 (20150328.133759) Java 1.8.0_31 Windows 7
The context: I have several sources of data from which I receive XML documents. My application will create a collection for each of these sources and store each received document into the collection corresponding to the source, and then update some kind of metadata about the collection. The metadata of all collections are stored into a common collection. Each source is handled by a different thread. I am using an embedded BaseX, and access to it through LocalSession.
Now the problem is that after a certain amount of time (not constant, could be seconds or minutes) there is some exceptions. Either: java.lang.IllegalMonitorStateException at org.basex.core.locks.DBLocking.acquire and org.basex.core.locks.DBLocking.release
Or: java.util.ConcurrentModificationException at org.basex.core.users.User.find
I put the full stack trace in attachment.
I build a test application that mimic what happens in the real application and it seems to exhibit the same problem. The source of this test application is attached
My question: Am I doing something wrong, or is there a bug in BaseX?
Thanks a lot
Simon
Simon,
I am glad to report the concurrency bug in the user access code has been fixed [1,2]. BaseX 8.1.1 may already be released this week. Of course, please report back to me if you should encounter additional bugs.
Thanks for testing, Christian
[1] https://github.com/BaseXdb/basex/commit/b5d1716e662a300d484cbbb19d1bf23cbef5... [2] http://files.basex.org/releases/latest
On Tue, Apr 14, 2015 at 2:44 PM, Christian Grün christian.gruen@gmail.com wrote:
I have minimized your example a bit (attached), and I agree it seems to be the user iterator that causes surprises.
Looking at it.. Christian
On Tue, Apr 14, 2015 at 1:48 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the updated code. It's still quite a lot code.. Is all of it required to reproduce the bug? E.g., does the bug only occur with the specified options etc? Could you possibly reduce it even more? Idealle, we can later rewrite it to a JUnit test once the problem is fixed (and we try to keep our tests as concise as possible).
But I'll have some thoughts about your helpful assumptions in parallel.
Thanks in advance, Christian
On Tue, Apr 14, 2015 at 1:10 PM, Simon Chatelain schatela@gmail.com wrote:
Hello Christian,
I checked again my code, but I am quite sure that each LocalSession is created and used by only one thread. I attach the test application code.
Each LocalSession is used to execute several commands, but that shouldn't be a problem, right ?
I also had a look in BaseX code, especially in the org.basex.core.users.User class.
What happens in my test application is that I create several Writer that will add XML document each in a specific collection, but using the same user. Each time I create a new Writer it will first create the collection then grant read and write permission to this collection to the user "USER", and then start adding XML document in this collection.
My suspicion is that the problem occurs when a new Writer is created, and it is granting permission to the newly created collection while another Writer (in another thread) is adding a new XML document in its own collection. If, during the execution of the Grant command, the method User#perm(final Perm prm, final String pattern) is called while the execution of a concurrent Write command is at the same time executing the User#find(final String db) method, then we have a ConcurrentModificationException as perm method will put a new entry in locals field while the find method is iterating on the locals entrySet of the same User, the same User instance being shared by all subcontext created when opening a new LocalSession. For test purposes, I synchronized the access to locals in the perm(), find() and remove() methods of the class User, in that case I do not have the ConcurrentModificationException any more.
What do you think, is this possible ?
Regards
Simon
On Mon, Apr 13, 2015 at 11:04 AM, Christian Grün christian.gruen@gmail.com wrote:
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find is still happening.
I am pretty sure that any of the concurrent command calls are not moved to their own thread yet.
But feel free to pass me on your code (ideally, a version that's as small as possible).
Best, Christian
What I did is basically, in the test application, replace all session creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
}});
Not very elegant but should be enough to make sure every session is executed in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the detailed description of your setup and your test code.
In BaseX, every session needs to be executed in a separate Java thread. The exceptions will e.g. disappear if you pass on your admin session to the store() and updateMetaStore() functions (or if you run these operations in a separate thread).
Hope this helps, Christian
On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain schatela@gmail.com wrote: > Hello all, > > I have the following problem. > > Using: > > BaseX 8.1.1 beta cf713e2 (20150328.133759) > Java 1.8.0_31 > Windows 7 > > The context: I have several sources of data from which I receive XML > documents. My application will create a collection for each of these > sources > and store each received document into the collection corresponding to > the > source, and then update some kind of metadata about the collection. > The > metadata of all collections are stored into a common collection. Each > source > is handled by a different thread. > I am using an embedded BaseX, and access to it through LocalSession. > > Now the problem is that after a certain amount of time (not constant, > could > be seconds or minutes) there is some exceptions. > Either: java.lang.IllegalMonitorStateException > at org.basex.core.locks.DBLocking.acquire > and org.basex.core.locks.DBLocking.release > > Or: java.util.ConcurrentModificationException > at org.basex.core.users.User.find > > I put the full stack trace in attachment. > > I build a test application that mimic what happens in the real > application > and it seems to exhibit the same problem. The source of this test > application is attached > > My question: Am I doing something wrong, or is there a bug in BaseX? > > Thanks a lot > > Simon
That was fast :-) . Thank you very much. I will test it right away.
Regards
Simon
On Tue, Apr 14, 2015 at 3:28 PM, Christian Grün christian.gruen@gmail.com wrote:
Simon,
I am glad to report the concurrency bug in the user access code has been fixed [1,2]. BaseX 8.1.1 may already be released this week. Of course, please report back to me if you should encounter additional bugs.
Thanks for testing, Christian
[1] https://github.com/BaseXdb/basex/commit/b5d1716e662a300d484cbbb19d1bf23cbef5... [2] http://files.basex.org/releases/latest
On Tue, Apr 14, 2015 at 2:44 PM, Christian Grün christian.gruen@gmail.com wrote:
I have minimized your example a bit (attached), and I agree it seems to be the user iterator that causes surprises.
Looking at it.. Christian
On Tue, Apr 14, 2015 at 1:48 PM, Christian Grün christian.gruen@gmail.com wrote:
Hi Simon,
Thanks for the updated code. It's still quite a lot code.. Is all of it required to reproduce the bug? E.g., does the bug only occur with the specified options etc? Could you possibly reduce it even more? Idealle, we can later rewrite it to a JUnit test once the problem is fixed (and we try to keep our tests as concise as possible).
But I'll have some thoughts about your helpful assumptions in parallel.
Thanks in advance, Christian
On Tue, Apr 14, 2015 at 1:10 PM, Simon Chatelain schatela@gmail.com
wrote:
Hello Christian,
I checked again my code, but I am quite sure that each LocalSession is created and used by only one thread. I attach the test application code.
Each LocalSession is used to execute several commands, but that
shouldn't be
a problem, right ?
I also had a look in BaseX code, especially in the
org.basex.core.users.User
class.
What happens in my test application is that I create several Writer
that
will add XML document each in a specific collection, but using the same user. Each time I create a new Writer it will first create the collection
then
grant read and write permission to this collection to the user "USER",
and
then start adding XML document in this collection.
My suspicion is that the problem occurs when a new Writer is created,
and it
is granting permission to the newly created collection while another
Writer
(in another thread) is adding a new XML document in its own collection. If, during the execution of the Grant command, the method
User#perm(final
Perm prm, final String pattern) is called while the execution of a concurrent Write command is at the same time executing the
User#find(final
String db) method, then we have a ConcurrentModificationException as
perm
method will put a new entry in locals field while the find method is iterating on the locals entrySet of the same User, the same User
instance
being shared by all subcontext created when opening a new LocalSession. For test purposes, I synchronized the access to locals in the perm(),
find()
and remove() methods of the class User, in that case I do not have the ConcurrentModificationException any more.
What do you think, is this possible ?
Regards
Simon
On Mon, Apr 13, 2015 at 11:04 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
But the java.util.ConcurrentModificationException at org.basex.core.users.User.find is still happening.
I am pretty sure that any of the concurrent command calls are not moved to their own thread yet.
But feel free to pass me on your code (ideally, a version that's as small as possible).
Best, Christian
What I did is basically, in the test application, replace all
session
creations
try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
by
Thread t = new Thread(new Runnable() { public void run() { try (final Session session = new LocalSession(rootContext, ...)) { ... } catch {...}
}});
Not very elegant but should be enough to make sure every session is executed in its own Java thread.
Am I still missing something ?
Thank you very much for your help, it is very appreciated.
Regards
Simon
On Tue, Apr 7, 2015 at 12:16 PM, Christian Grün christian.gruen@gmail.com wrote: > > Hi Simon, > > Thanks for the detailed description of your setup and your test
code.
> > In BaseX, every session needs to be executed in a separate Java > thread. The exceptions will e.g. disappear if you pass on your
admin
> session to the store() and updateMetaStore() functions (or if you
run
> these operations in a separate thread). > > Hope this helps, > Christian > > > On Thu, Apr 2, 2015 at 1:58 PM, Simon Chatelain <
schatela@gmail.com>
> wrote: > > Hello all, > > > > I have the following problem. > > > > Using: > > > > BaseX 8.1.1 beta cf713e2 (20150328.133759) > > Java 1.8.0_31 > > Windows 7 > > > > The context: I have several sources of data from which I receive
XML
> > documents. My application will create a collection for each of
these
> > sources > > and store each received document into the collection
corresponding to
> > the > > source, and then update some kind of metadata about the
collection.
> > The > > metadata of all collections are stored into a common collection.
Each
> > source > > is handled by a different thread. > > I am using an embedded BaseX, and access to it through
LocalSession.
> > > > Now the problem is that after a certain amount of time (not
constant,
> > could > > be seconds or minutes) there is some exceptions. > > Either: java.lang.IllegalMonitorStateException > > at org.basex.core.locks.DBLocking.acquire > > and org.basex.core.locks.DBLocking.release > > > > Or: java.util.ConcurrentModificationException > > at org.basex.core.users.User.find > > > > I put the full stack trace in attachment. > > > > I build a test application that mimic what happens in the real > > application > > and it seems to exhibit the same problem. The source of this test > > application is attached > > > > My question: Am I doing something wrong, or is there a bug in
BaseX?
> > > > Thanks a lot > > > > Simon
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Simon Chatelain