New subject: UPDINDEX and ever growing index size

6 Aug 2014


      Hi Christian,
I’ve been following along with your snapshots and I think there’s a bug still remaining somewhere but I can’t work out what’s causing it which is why I haven’t been back to you.
If you use the GUI and the database is open in the GUI then everything works correctly. I can add, replace or a mix of the two with no issues.
However if I use the GUI but the database is closed then I can still get AOOB errors.
I’ve spent the evening gradually reducing it down so I can reproduce the error using just one element and one attribute and do it with command files.
I’ve attached three files:
AOOB-succeed.bxs will always work. The database is created and remains open during the replacements. Everything is fine.
AOOB-fail.bxs will fail. The database is closed after being created and the XQuery runs on the closed database. The first replacements (plural but fewer than the number of documents in the database - 1) are fine but subsequent ones fail.
AOOB-fail2.bxs will also fail. This is the same as AOOB-fail.bxs except that only one replacement is made. This still fails but will a different error (key does not exist).
I hope these help track down this bug - it’s working really well otherwise.
Regards, James
On 6 Aug 2014, at 11:14, Christian Grün christian.gruen@gmail.com wrote:
...
Hi James,
while writing some more tests, I stumbled upon another small bug,
causing AOOB exceptions during the insertion of new entries. The new
snapshot is online.
Christian
On Mon, Aug 4, 2014 at 7:32 PM, Christian Grün
christian.gruen@gmail.com wrote:
...
A last one for today: I have just uploaded another snapshot which
should speed up index updates.
Looking forward to your feedback,
Christian
On Mon, Aug 4, 2014 at 4:01 PM, Christian Grün
christian.gruen@gmail.com wrote:
...
The bug was hidden well [1], but it should be fixed now. Could you
check out the latest snapshot?
Christian
[1] https://github.com/BaseXdb/basex/commit/429585ce26fca98d124d78fb88216ad7317c...
On Mon, Aug 4, 2014 at 2:41 PM, Christian Grün
christian.gruen@gmail.com wrote:
...
Hi James,
I've found a little example for the bug (see attached).
Sorry for the inconvenience; I'm working on a fix.
Christian
On Mon, Aug 4, 2014 at 11:30 AM, Christian Grün
christian.gruen@gmail.com wrote:
...
James,
thanks for testing. We have a bunch of test cases that succeeded for
the rewritten index handling, but as it seems, we definitely need some
more. I'm pretty sure it's a single bug that causes all the error
messages (because the code is in itself pretty straightforward), so I
would be glad if you could compose a little, self-contained example
that provokes the error. I have attached a little (working) command
script which you can open in the gui (and execute there) and modify it
until it raises one of the reported errors.
Thanks,
Christian
On Fri, Aug 1, 2014 at 7:14 PM, James Ball basex-talk@jamesball.co.uk wrote:
...
Hi Christian,
> I'm glad to tell you that I have now implemented the projected optimizations
Thank you for providing the snapshot. I've downloaded it and begun running some tests.
Unfortunately I'm immediately finding some odd behaviour. I'm using the script I provided in my original issue report to the list.
I can use replace() to add as many documents to the database as I want as long as the documents are new (no document exists to be replaced).
If I use replace() on one document in a transaction ($files set to 1 in my script) then everything works.
However if I try to replace more than one file in a transaction ($files set to 2+) I get an error.
Error:
Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 8.0 beta 3a7d766
Java: Oracle Corporation, 1.7.0_60
OS: Mac OS X, x86_64
Stack Trace:
java.lang.RuntimeException: Key does not exist: 'Name'
       at org.basex.util.Util.notExpected(Util.java:60)
       at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:82)
       at org.basex.data.DiskData.indexDelete(DiskData.java:390)
       at org.basex.data.DiskData.indexDelete(DiskData.java:452)
       at org.basex.data.Data.delete(Data.java:632)
       at org.basex.data.atomic.Delete.apply(Delete.java:39)
       at org.basex.data.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:298)
       at org.basex.data.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:282)
       at org.basex.query.up.DataUpdates.apply(DataUpdates.java:161)
       at org.basex.query.up.ContextModifier.apply(ContextModifier.java:118)
       at org.basex.query.up.Updates.apply(Updates.java:129)
       at org.basex.query.QueryContext.iter(QueryContext.java:351)
       at org.basex.query.QueryContext.execute(QueryContext.java:605)
       at org.basex.query.QueryProcessor.execute(QueryProcessor.java:100)
       at org.basex.core.cmd.AQuery.query(AQuery.java:82)
       at org.basex.core.cmd.XQuery.run(XQuery.java:22)
       at org.basex.core.Command.run(Command.java:360)
       at org.basex.core.Command.execute(Command.java:94)
       at org.basex.gui.GUI.exec(GUI.java:417)
       at org.basex.gui.GUI.access$500(GUI.java:41)
       at org.basex.gui.GUI$8.run(GUI.java:361)
If I keep running the command then eventually it will work (eventually is related to the number of documents being replace/in the database). Note that this is in the GUI with the database open in the GUI.
If I do it in the GUI but with the database closed I get alternating errors between 'Key does not exist' and 'Key should not exist' each time I run. The error never corrects itself.
I'm happy to investigate further and provide more details if required but I'm confused as to what might actually be happening to cause this so I'm not sure where to go next. Let me if you need anything from me.
Regards, James

Re: [basex-talk] UPDINDEX and ever growing index size