Hi Christian,

Thanks for the quick reply!

No, all the ADD/REPLACE/DELETE operations are executed sequentially by a single thread. For example, even if I try to add 2000 documents the above exceptions might occur after the first 1000 have been updated successfully.

If I do manage to create a reproducible example, I will of course share it with the community.

Could this instability be simply related to NFS? What kind of handling/workaround would you suggest?

Thanks, Panagiotis

On 1 August 2016 at 13:46, Christian Grün christian.gruen@gmail.com wrote:

Hi Panagiotis,

...

Unfortunately, I do not have a clear pattern as to what causes the

above, or

...

any errors during initialization.

Difficult to say what might be the problem. Are you working with multiple threads?

If there’s any chance for you to provide us with a reproducible example, feel free to do so.

Christian

On Mon, Aug 1, 2016 at 12:37 PM, Panagiotis Bailis pmpailis@gmail.com wrote:

...

Hi all,

I am using Basex 8.4.4 through the JAVA API and I noticed quite a few occurrences of the following NPE and AIOOB exceptions in the logs:

---

java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:126) at org.basex.data.DiskData.close(DiskData.java:140) at org.basex.core.Datas.close(Datas.java:84) at org.basex.core.Context.close(Context.java:155)

---

org.basex.core.BaseXException: Improper use? Potential bug? Your

feedback is

...

welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.4.4 Java: Oracle Corporation, 1.8.0_66 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:120) at org.basex.data.DiskData.flush(DiskData.java:235) at org.basex.data.DiskData.finishUpdate(DiskData.java:225) at org.basex.core.Command.finishUpdate(Command.java:341) at org.basex.core.cmd.ACreate.finishUpdate(ACreate.java:87) at org.basex.core.cmd.Replace.run(Replace.java:59) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.core.Command.execute(Command.java:123)

---

org.basex.core.BaseXException: Improper use? Potential bug? Your

feedback is

...

welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.4.4 Java: Oracle Corporation, 1.8.0_66 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 0 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:501) at

org.basex.io.random.TableDiskAccess.setPage(TableDiskAccess.java:473)

...

at

org.basex.io.random.TableDiskAccess.readPage(TableDiskAccess.java:482) at

org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:316)

...

at org.basex.data.Data.insert(Data.java:901)
at org.basex.data.Data.insert(Data.java:727)
at org.basex.query.up.atomic.Insert.apply(Insert.java:44)
at

org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:297)

...

at

org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:281)

...

at org.basex.core.cmd.Replace.replace(Replace.java:106)
at org.basex.core.cmd.Replace.run(Replace.java:57)
at org.basex.core.Command.run(Command.java:398)
at org.basex.core.Command.execute(Command.java:100)
at org.basex.core.Command.execute(Command.java:123)

---

The collection files are stored to NFS, but the above exceptions seem to appear at random while trying to update an existing collection. Unfortunately, I do not have a clear pattern as to what causes the

above, or

...

any errors during initialization.

Thank you in advance, Panagiotis

Did you try BaseX 8.5.2?

Not yet. Planning to upgrade within the week, but if I recall correctly the same errors were also appearing (sparingly) in BaseX 8.3.1. Once I upgrade to the newest version, should the same issues arise again, I'll let you know.

On 1 August 2016 at 14:04, Christian Grün christian.gruen@gmail.com wrote:

...

Could this instability be simply related to NFS?

Well, it could be ;) I haven’t experienced a similar pattern so far (maybe someone else on the list did), so I am afraid I cannot really help. Did you try BaseX 8.5.2?

...

On 1 August 2016 at 13:46, Christian Grün christian.gruen@gmail.com

wrote:

...

...

Hi Panagiotis,

...

Unfortunately, I do not have a clear pattern as to what causes the above, or any errors during initialization.

Difficult to say what might be the problem. Are you working with multiple threads?

If there’s any chance for you to provide us with a reproducible example, feel free to do so.

Christian

On Mon, Aug 1, 2016 at 12:37 PM, Panagiotis Bailis pmpailis@gmail.com wrote:

...

Hi all,

I am using Basex 8.4.4 through the JAVA API and I noticed quite a few occurrences of the following NPE and AIOOB exceptions in the logs:

---

...

...

...

java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:126) at org.basex.data.DiskData.close(DiskData.java:140) at org.basex.core.Datas.close(Datas.java:84) at org.basex.core.Context.close(Context.java:155)

---

...

...

...

org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.4.4 Java: Oracle Corporation, 1.8.0_66 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:120) at org.basex.data.DiskData.flush(DiskData.java:235) at org.basex.data.DiskData.finishUpdate(DiskData.java:225) at org.basex.core.Command.finishUpdate(Command.java:341) at org.basex.core.cmd.ACreate.finishUpdate(ACreate.java:87) at org.basex.core.cmd.Replace.run(Replace.java:59) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.core.Command.execute(Command.java:123)

---

...

...

...

org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.4.4 Java: Oracle Corporation, 1.8.0_66 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 0 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:501) at org.basex.io.random.TableDiskAccess.setPage(TableDiskAccess.java:473) at org.basex.io.random.TableDiskAccess.readPage(TableDiskAccess.java:482) at org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:316) at org.basex.data.Data.insert(Data.java:901) at org.basex.data.Data.insert(Data.java:727) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at

org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:297)

...

...

...

at

org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:281)

...

...

...

at org.basex.core.cmd.Replace.replace(Replace.java:106)
at org.basex.core.cmd.Replace.run(Replace.java:57)
at org.basex.core.Command.run(Command.java:398)
at org.basex.core.Command.execute(Command.java:100)
at org.basex.core.Command.execute(Command.java:123)

---

...

...

...

The collection files are stored to NFS, but the above exceptions seem

to

...

...

...

appear at random while trying to update an existing collection. Unfortunately, I do not have a clear pattern as to what causes the above, or any errors during initialization.

Thank you in advance, Panagiotis

Hi Vincent,

The difference in this case is that there doesn't appear to be some sort of consistency between the number of documents or the specifics of each exception (which is rather annoying!). However, the next step could be somewhat similar to yours, in the sense that I could move from the NFS to the local FS, in order to simply see if the problem re-occurs.

Thanks, Panagiotis

On 1 August 2016 at 14:16, Lizzi, Vincent < Vincent.Lizzi@taylorandfrancis.com> wrote:

This sounds similar to a problem I ran into previously, when adding several thousand documents an exception would occur after a somewhat consistent number of documents were added. The number might have been 1000. I suspected the antivirus program might be interfering with BaseX, though couldn't confirm this was the problem, and ran the import on a different machine that had a fresh install of Windows. This was over a year ago, and I don't recall the specifics. Maybe this is helpful in some way.

Vincent

---

*From:* basex-talk-bounces@mailman.uni-konstanz.de < basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Christian Grün < christian.gruen@gmail.com> *Sent:* Monday, August 1, 2016 7:04:26 AM *To:* Panagiotis Bailis *Cc:* BaseX *Subject:* Re: [basex-talk] NPE in Basex8.4.4

...

Could this instability be simply related to NFS?

Well, it could be ;) I haven’t experienced a similar pattern so far (maybe someone else on the list did), so I am afraid I cannot really help. Did you try BaseX 8.5.2?

...

On 1 August 2016 at 13:46, Christian Grün christian.gruen@gmail.com

wrote:

...

...

Hi Panagiotis,

...

Unfortunately, I do not have a clear pattern as to what causes the above, or any errors during initialization.

Difficult to say what might be the problem. Are you working with multiple threads?

If there’s any chance for you to provide us with a reproducible example, feel free to do so.

Christian

On Mon, Aug 1, 2016 at 12:37 PM, Panagiotis Bailis pmpailis@gmail.com wrote:

...

Hi all,

I am using Basex 8.4.4 through the JAVA API and I noticed quite a few occurrences of the following NPE and AIOOB exceptions in the logs:

---

...

...

...

java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:126) at org.basex.data.DiskData.close(DiskData.java:140) at org.basex.core.Datas.close(Datas.java:84) at org.basex.core.Context.close(Context.java:155)

---

...

...

...

org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.4.4 Java: Oracle Corporation, 1.8.0_66 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.data.DiskData.write(DiskData.java:120) at org.basex.data.DiskData.flush(DiskData.java:235) at org.basex.data.DiskData.finishUpdate(DiskData.java:225) at org.basex.core.Command.finishUpdate(Command.java:341) at org.basex.core.cmd.ACreate.finishUpdate(ACreate.java:87) at org.basex.core.cmd.Replace.run(Replace.java:59) at org.basex.core.Command.run(Command.java:398) at org.basex.core.Command.execute(Command.java:100) at org.basex.core.Command.execute(Command.java:123)

---

...

...

...

org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 8.4.4 Java: Oracle Corporation, 1.8.0_66 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 0 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:501) at org.basex.io.random.TableDiskAccess.setPage(TableDiskAccess.java:473) at org.basex.io.random.TableDiskAccess.readPage(TableDiskAccess.java:482) at org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:316) at org.basex.data.Data.insert(Data.java:901) at org.basex.data.Data.insert(Data.java:727) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at

org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:297)

...

...

...

at

org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:281)

...

...

...

at org.basex.core.cmd.Replace.replace(Replace.java:106)
at org.basex.core.cmd.Replace.run(Replace.java:57)
at org.basex.core.Command.run(Command.java:398)
at org.basex.core.Command.execute(Command.java:100)
at org.basex.core.Command.execute(Command.java:123)

---

...

...

...

The collection files are stored to NFS, but the above exceptions seem

to

...

...

...

appear at random while trying to update an existing collection. Unfortunately, I do not have a clear pattern as to what causes the above, or any errors during initialization.

Thank you in advance, Panagiotis