Received the following stack trace after running a .bxs script using basexclient from the command line:
Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:390) org.basex.data.Data.insert(Data.java:931) org.basex.data.Data.insert(Data.java:820) org.basex.data.atomic.Insert.apply(Insert.java:31) org.basex.data.atomic.AtomicUpdateList.applyStructuralUpdates(AtomicUpdateList.java:297) org.basex.data.atomic.AtomicUpdateList.execute(AtomicUpdateList.java:285) org.basex.query.up.DatabaseUpdates.apply(DatabaseUpdates.java:183) org.basex.query.up.ContextModifier.apply(ContextModifier.java:90) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
What the script does is very complex (large xquery functions, db:node-pre() indexing and lookups, etc) so if you need it to isolate this issue we can discuss directly.
However I get a similar stack trace when I attempt to OPTIMIZE ALL:
open deepbills
Database 'deepbills' was opened in 114.99 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 65 org.basex.util.Compress.pull(Compress.java:156) org.basex.util.Compress.unpack(Compress.java:112) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:343) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:199) org.basex.build.Builder.parse(Builder.java:73) org.basex.build.DiskBuilder.build(DiskBuilder.java:90) org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124) org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
info database
Database Properties Name: deepbills Size: 901 MB Nodes: 25598080 Documents: 15528 Binaries: 0 Timestamp: 2013-04-08-15-02-07
Resource Properties Timestamp: 2013-04-08-15-02-07 Encoding: UTF-8 Whitespace Chopping: ON
Indexes Up-to-date: false Text Index: OFF Attribute Index: OFF Full-Text Index: OFF UPDINDEX: OFF MAXCATS: 100 MAXLEN: 96
If I restore from a backup (taken after this issue appeared), I still can't "optimize all", but the stack trace appears truncated:
restore deepbills
'deepbills-2013-04-08-15-54-32.zip' was restored in 40918.34 ms.
optimize all
No database opened.
open deepbills
Database 'deepbills' was opened in 136.22 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException
If I restore a backup that is just a little bit older (a little over an hour), it seems fine:
restore deepbills-2013-04-08-14-22-22
'deepbills-2013-04-08-14-22-22.zip' was restored in 38982.91 ms.
optimize all
Database 'deepbills' was optimized in 129658.46 ms.
So I have two similar databases, one 363MB zip, one 372MB zip, one of which is fine and the other which seems to encounter this problem. Could I be looking at database corruption here? If so, how to fix?
Dear Francis, I experienced similar problems when executing XQuery Update on collections with UPDINDEX = TRUE. Are you in that case ? You should give a try to the last stable version.
Regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Francis Avila Envoyé : lundi 8 avril 2013 18:29 À : BaseX Objet : [basex-talk] NegativeArraySizeException
Received the following stack trace after running a .bxs script using basexclient from the command line:
Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:390) org.basex.data.Data.insert(Data.java:931) org.basex.data.Data.insert(Data.java:820) org.basex.data.atomic.Insert.apply(Insert.java:31) org.basex.data.atomic.AtomicUpdateList.applyStructuralUpdates(AtomicUpdateList.java:297) org.basex.data.atomic.AtomicUpdateList.execute(AtomicUpdateList.java:285) org.basex.query.up.DatabaseUpdates.apply(DatabaseUpdates.java:183) org.basex.query.up.ContextModifier.apply(ContextModifier.java:90) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
What the script does is very complex (large xquery functions, db:node-pre() indexing and lookups, etc) so if you need it to isolate this issue we can discuss directly.
However I get a similar stack trace when I attempt to OPTIMIZE ALL:
open deepbills
Database 'deepbills' was opened in 114.99 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 65 org.basex.util.Compress.pull(Compress.java:156) org.basex.util.Compress.unpack(Compress.java:112) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:343) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:199) org.basex.build.Builder.parse(Builder.java:73) org.basex.build.DiskBuilder.build(DiskBuilder.java:90) org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124) org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
info database
Database Properties Name: deepbills Size: 901 MB Nodes: 25598080 Documents: 15528 Binaries: 0 Timestamp: 2013-04-08-15-02-07
Resource Properties Timestamp: 2013-04-08-15-02-07 Encoding: UTF-8 Whitespace Chopping: ON
Indexes Up-to-date: false Text Index: OFF Attribute Index: OFF Full-Text Index: OFF UPDINDEX: OFF MAXCATS: 100 MAXLEN: 96
If I restore from a backup (taken after this issue appeared), I still can't "optimize all", but the stack trace appears truncated:
restore deepbills
'deepbills-2013-04-08-15-54-32.zip' was restored in 40918.34 ms.
optimize all
No database opened.
open deepbills
Database 'deepbills' was opened in 136.22 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException
If I restore a backup that is just a little bit older (a little over an hour), it seems fine:
restore deepbills-2013-04-08-14-22-22
'deepbills-2013-04-08-14-22-22.zip' was restored in 38982.91 ms.
optimize all
Database 'deepbills' was optimized in 129658.46 ms.
So I have two similar databases, one 363MB zip, one 372MB zip, one of which is fine and the other which seems to encounter this problem. Could I be looking at database corruption here? If so, how to fix?
I am not using UPINDEX=true and I am already on the latest stable version (7.6).
This happens when I run OPTIMIZE, though, so perhaps it is related to indexes.
On Apr 9, 2013, at 2:06 AM, Fabrice Etanchaud wrote:
Dear Francis, I experienced similar problems when executing XQuery Update on collections with UPDINDEX = TRUE. Are you in that case ? You should give a try to the last stable version.
Regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Francis Avila Envoyé : lundi 8 avril 2013 18:29 À : BaseX Objet : [basex-talk] NegativeArraySizeException
Received the following stack trace after running a .bxs script using basexclient from the command line:
Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:390) org.basex.data.Data.insert(Data.java:931) org.basex.data.Data.insert(Data.java:820) org.basex.data.atomic.Insert.apply(Insert.java:31) org.basex.data.atomic.AtomicUpdateList.applyStructuralUpdates(AtomicUpdateList.java:297) org.basex.data.atomic.AtomicUpdateList.execute(AtomicUpdateList.java:285) org.basex.query.up.DatabaseUpdates.apply(DatabaseUpdates.java:183) org.basex.query.up.ContextModifier.apply(ContextModifier.java:90) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
What the script does is very complex (large xquery functions, db:node-pre() indexing and lookups, etc) so if you need it to isolate this issue we can discuss directly.
However I get a similar stack trace when I attempt to OPTIMIZE ALL:
open deepbills
Database 'deepbills' was opened in 114.99 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 65 org.basex.util.Compress.pull(Compress.java:156) org.basex.util.Compress.unpack(Compress.java:112) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:343) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:199) org.basex.build.Builder.parse(Builder.java:73) org.basex.build.DiskBuilder.build(DiskBuilder.java:90) org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124) org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
info database
Database Properties Name: deepbills Size: 901 MB Nodes: 25598080 Documents: 15528 Binaries: 0 Timestamp: 2013-04-08-15-02-07
Resource Properties Timestamp: 2013-04-08-15-02-07 Encoding: UTF-8 Whitespace Chopping: ON
Indexes Up-to-date: false Text Index: OFF Attribute Index: OFF Full-Text Index: OFF UPDINDEX: OFF MAXCATS: 100 MAXLEN: 96
If I restore from a backup (taken after this issue appeared), I still can't "optimize all", but the stack trace appears truncated:
restore deepbills
'deepbills-2013-04-08-15-54-32.zip' was restored in 40918.34 ms.
optimize all
No database opened.
open deepbills
Database 'deepbills' was opened in 136.22 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException
If I restore a backup that is just a little bit older (a little over an hour), it seems fine:
restore deepbills-2013-04-08-14-22-22
'deepbills-2013-04-08-14-22-22.zip' was restored in 38982.91 ms.
optimize all
Database 'deepbills' was optimized in 129658.46 ms.
So I have two similar databases, one 363MB zip, one 372MB zip, one of which is fine and the other which seems to encounter this problem. Could I be looking at database corruption here? If so, how to fix?
Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
This could point to a node ID overflow, and may be related to Fabrice’s observations, for which I’ve now created a new GitHub issue [1]. Feel free to give us an update if you should manage to make this error reproducible in some way.
[1] https://github.com/BaseXdb/basex/issues/676 ___________________________
On Tue, Apr 9, 2013 at 3:10 PM, Francis Avila favila@dancingmammoth.com wrote:
I am not using UPINDEX=true and I am already on the latest stable version (7.6).
This happens when I run OPTIMIZE, though, so perhaps it is related to indexes.
On Apr 9, 2013, at 2:06 AM, Fabrice Etanchaud wrote:
Dear Francis, I experienced similar problems when executing XQuery Update on collections with UPDINDEX = TRUE. Are you in that case ? You should give a try to the last stable version.
Regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Francis Avila Envoyé : lundi 8 avril 2013 18:29 À : BaseX Objet : [basex-talk] NegativeArraySizeException
Received the following stack trace after running a .bxs script using basexclient from the command line:
Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:390) org.basex.data.Data.insert(Data.java:931) org.basex.data.Data.insert(Data.java:820) org.basex.data.atomic.Insert.apply(Insert.java:31) org.basex.data.atomic.AtomicUpdateList.applyStructuralUpdates(AtomicUpdateList.java:297) org.basex.data.atomic.AtomicUpdateList.execute(AtomicUpdateList.java:285) org.basex.query.up.DatabaseUpdates.apply(DatabaseUpdates.java:183) org.basex.query.up.ContextModifier.apply(ContextModifier.java:90) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
What the script does is very complex (large xquery functions, db:node-pre() indexing and lookups, etc) so if you need it to isolate this issue we can discuss directly.
However I get a similar stack trace when I attempt to OPTIMIZE ALL:
open deepbills
Database 'deepbills' was opened in 114.99 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 65 org.basex.util.Compress.pull(Compress.java:156) org.basex.util.Compress.unpack(Compress.java:112) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:343) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:199) org.basex.build.Builder.parse(Builder.java:73) org.basex.build.DiskBuilder.build(DiskBuilder.java:90) org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124) org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
info database
Database Properties Name: deepbills Size: 901 MB Nodes: 25598080 Documents: 15528 Binaries: 0 Timestamp: 2013-04-08-15-02-07
Resource Properties Timestamp: 2013-04-08-15-02-07 Encoding: UTF-8 Whitespace Chopping: ON
Indexes Up-to-date: false Text Index: OFF Attribute Index: OFF Full-Text Index: OFF UPDINDEX: OFF MAXCATS: 100 MAXLEN: 96
If I restore from a backup (taken after this issue appeared), I still can't "optimize all", but the stack trace appears truncated:
restore deepbills
'deepbills-2013-04-08-15-54-32.zip' was restored in 40918.34 ms.
optimize all
No database opened.
open deepbills
Database 'deepbills' was opened in 136.22 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException
If I restore a backup that is just a little bit older (a little over an hour), it seems fine:
restore deepbills-2013-04-08-14-22-22
'deepbills-2013-04-08-14-22-22.zip' was restored in 38982.91 ms.
optimize all
Database 'deepbills' was optimized in 129658.46 ms.
So I have two similar databases, one 363MB zip, one 372MB zip, one of which is fine and the other which seems to encounter this problem. Could I be looking at database corruption here? If so, how to fix?
Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
I can reproduce this with a database backup I saved. Unfortunately I can't trim down the test case for the NegativeArraySizeException--reproducing requires a 317MB zip file (for a nearly 1GB database), an xquery module, and an xquery script. I have no trouble sharing this data and code with you since it's all public domain or open source, but it's big.
The same database produces an ArrayIndexOutOfBoundsException when OPTIMIZE ALL is run on it.
I suspected something related to nodeid overflows but:
1. As you can see below, the number of nodes reported by INFO DATABASE is only 25598080, which is not anywhere near the limit. 2. The database is larger now but this problem has not reappeared. I had to revert the DB to an earlier backup (lost about an hour of data), but the same script has been running on it for a week since then without issues.
On Apr 13, 2013, at 2:30 PM, Christian Grün wrote:
This could point to a node ID overflow, and may be related to Fabrice’s observations, for which I’ve now created a new GitHub issue [1]. Feel free to give us an update if you should manage to make this error reproducible in some way.
[1] https://github.com/BaseXdb/basex/issues/676 ___________________________
On Tue, Apr 9, 2013 at 3:10 PM, Francis Avila favila@dancingmammoth.com wrote:
I am not using UPINDEX=true and I am already on the latest stable version (7.6).
This happens when I run OPTIMIZE, though, so perhaps it is related to indexes.
On Apr 9, 2013, at 2:06 AM, Fabrice Etanchaud wrote:
Dear Francis, I experienced similar problems when executing XQuery Update on collections with UPDINDEX = TRUE. Are you in that case ? You should give a try to the last stable version.
Regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Francis Avila Envoyé : lundi 8 avril 2013 18:29 À : BaseX Objet : [basex-talk] NegativeArraySizeException
Received the following stack trace after running a .bxs script using basexclient from the command line:
Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:390) org.basex.data.Data.insert(Data.java:931) org.basex.data.Data.insert(Data.java:820) org.basex.data.atomic.Insert.apply(Insert.java:31) org.basex.data.atomic.AtomicUpdateList.applyStructuralUpdates(AtomicUpdateList.java:297) org.basex.data.atomic.AtomicUpdateList.execute(AtomicUpdateList.java:285) org.basex.query.up.DatabaseUpdates.apply(DatabaseUpdates.java:183) org.basex.query.up.ContextModifier.apply(ContextModifier.java:90) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
What the script does is very complex (large xquery functions, db:node-pre() indexing and lookups, etc) so if you need it to isolate this issue we can discuss directly.
However I get a similar stack trace when I attempt to OPTIMIZE ALL:
open deepbills
Database 'deepbills' was opened in 114.99 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 65 org.basex.util.Compress.pull(Compress.java:156) org.basex.util.Compress.unpack(Compress.java:112) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:343) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:199) org.basex.build.Builder.parse(Builder.java:73) org.basex.build.DiskBuilder.build(DiskBuilder.java:90) org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124) org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
info database
Database Properties Name: deepbills Size: 901 MB Nodes: 25598080 Documents: 15528 Binaries: 0 Timestamp: 2013-04-08-15-02-07
Resource Properties Timestamp: 2013-04-08-15-02-07 Encoding: UTF-8 Whitespace Chopping: ON
Indexes Up-to-date: false Text Index: OFF Attribute Index: OFF Full-Text Index: OFF UPDINDEX: OFF MAXCATS: 100 MAXLEN: 96
If I restore from a backup (taken after this issue appeared), I still can't "optimize all", but the stack trace appears truncated:
restore deepbills
'deepbills-2013-04-08-15-54-32.zip' was restored in 40918.34 ms.
optimize all
No database opened.
open deepbills
Database 'deepbills' was opened in 136.22 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException
If I restore a backup that is just a little bit older (a little over an hour), it seems fine:
restore deepbills-2013-04-08-14-22-22
'deepbills-2013-04-08-14-22-22.zip' was restored in 38982.91 ms.
optimize all
Database 'deepbills' was optimized in 129658.46 ms.
So I have two similar databases, one 363MB zip, one 372MB zip, one of which is fine and the other which seems to encounter this problem. Could I be looking at database corruption here? If so, how to fix?
Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
The same database produces an ArrayIndexOutOfBoundsException when OPTIMIZE ALL is run on it.
Yes, this indicates that this database is already corrupt.
- As you can see below, the number of nodes reported by INFO DATABASE is only 25598080, which is not anywhere near the limit.
True. I further analysed the stack trace code, and I believe it’s actually not related to the negative node id issue. Hard to guess, however, what else might have been the reason for the corruption. Maybe it’s got to do with the observation from Nicholas some days ago [1].
- The database is larger now but this problem has not reappeared. I had to revert the DB to an earlier backup (lost about an hour of data), but the same script has been running on it for a week since then without issues.
Good news, at least. Christian
[1] http://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg02453.html
On Tue, Apr 9, 2013 at 3:10 PM, Francis Avila favila@dancingmammoth.com wrote:
I am not using UPINDEX=true and I am already on the latest stable version (7.6).
This happens when I run OPTIMIZE, though, so perhaps it is related to indexes.
On Apr 9, 2013, at 2:06 AM, Fabrice Etanchaud wrote:
Dear Francis, I experienced similar problems when executing XQuery Update on collections with UPDINDEX = TRUE. Are you in that case ? You should give a try to the last stable version.
Regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Francis Avila Envoyé : lundi 8 avril 2013 18:29 À : BaseX Objet : [basex-talk] NegativeArraySizeException
Received the following stack trace after running a .bxs script using basexclient from the command line:
Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException org.basex.io.random.TableDiskAccess.insert(TableDiskAccess.java:390) org.basex.data.Data.insert(Data.java:931) org.basex.data.Data.insert(Data.java:820) org.basex.data.atomic.Insert.apply(Insert.java:31) org.basex.data.atomic.AtomicUpdateList.applyStructuralUpdates(AtomicUpdateList.java:297) org.basex.data.atomic.AtomicUpdateList.execute(AtomicUpdateList.java:285) org.basex.query.up.DatabaseUpdates.apply(DatabaseUpdates.java:183) org.basex.query.up.ContextModifier.apply(ContextModifier.java:90) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
What the script does is very complex (large xquery functions, db:node-pre() indexing and lookups, etc) so if you need it to isolate this issue we can discuss directly.
However I get a similar stack trace when I attempt to OPTIMIZE ALL:
open deepbills
Database 'deepbills' was opened in 114.99 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 65 org.basex.util.Compress.pull(Compress.java:156) org.basex.util.Compress.unpack(Compress.java:112) org.basex.data.DiskData.txt(DiskData.java:268) org.basex.data.DiskData.text(DiskData.java:235) org.basex.io.serial.Serializer.node(Serializer.java:343) org.basex.io.serial.Serializer.serialize(Serializer.java:99) org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:199) org.basex.build.Builder.parse(Builder.java:73) org.basex.build.DiskBuilder.build(DiskBuilder.java:90) org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:124) org.basex.core.cmd.OptimizeAll.run(OptimizeAll.java:44) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
info database
Database Properties Name: deepbills Size: 901 MB Nodes: 25598080 Documents: 15528 Binaries: 0 Timestamp: 2013-04-08-15-02-07
Resource Properties Timestamp: 2013-04-08-15-02-07 Encoding: UTF-8 Whitespace Chopping: ON
Indexes Up-to-date: false Text Index: OFF Attribute Index: OFF Full-Text Index: OFF UPDINDEX: OFF MAXCATS: 100 MAXLEN: 96
If I restore from a backup (taken after this issue appeared), I still can't "optimize all", but the stack trace appears truncated:
restore deepbills
'deepbills-2013-04-08-15-54-32.zip' was restored in 40918.34 ms.
optimize all
No database opened.
open deepbills
Database 'deepbills' was opened in 136.22 ms.
optimize all
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Sun Microsystems Inc., 1.6.0_27 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException
If I restore a backup that is just a little bit older (a little over an hour), it seems fine:
restore deepbills-2013-04-08-14-22-22
'deepbills-2013-04-08-14-22-22.zip' was restored in 38982.91 ms.
optimize all
Database 'deepbills' was optimized in 129658.46 ms.
So I have two similar databases, one 363MB zip, one 372MB zip, one of which is fine and the other which seems to encounter this problem. Could I be looking at database corruption here? If so, how to fix?
Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
-- Francis Avila Senior Developer Dancing Mammoth, Inc. (Formerly PJ Doland Web Design, Inc.) P: 703.621.0990 E: favila@dancingmammoth.com http://dancingmammoth.com
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Fabrice Etanchaud -
Francis Avila