If the "restored database" (which you are updating in the second step) is still ok, we might be able to trace this down; but you'd probably need to provide us with your data.
halit tiryaki halit.tiryaki@gmail.com schrieb am Do., 17. Mai 2018, 23:15:
How did you invoke flush? Did you use the client/server architecture?
I'm using the C# client. Simply executing the commands on the session:
- SET AUTOFLUSH true
- FLUSH (after ~200 docs of insert)
The stack trace seems to indicate that you were deleting data. Is it
part of your import step? yes, outdated documents are dropped in the import process
Thanks Halit
On Thu, May 17, 2018 at 10:28 PM, Christian Grün < christian.gruen@gmail.com> wrote:
Hi Halit,
Some more questions:
The Flush() invoke in longer periods during the process might have
been corrupted the state in race conditions.
How did you invoke flush? Did you use the client/server architecture?
Restored the db from a backup and restarted the import with
AutoFlush:true, this time I got:
The stack trace seems to indicate that you were deleting data. Is it part of your import step?
Best, Christian
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.0.2 beta Java: Oracle Corporation, 1.8.0_151 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Key does not exist: '86295d851a084fb2bde4181e80760099' at org.basex.util.Util.notExpected(Util.java:61) at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:99) at org.basex.data.Data.indexDelete(Data.java:1060) at org.basex.data.Data.delete(Data.java:663) at org.basex.query.up.atomic.Delete.apply(Delete.java:39) at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:296) at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:280) at org.basex.query.up.DataUpdates.apply(DataUpdates.java:168) at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) at org.basex.query.up.Updates.apply(Updates.java:157) at org.basex.query.QueryContext.iter(QueryContext.java:341) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at org.basex.core.cmd.AQuery.query(AQuery.java:92) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.server.ClientListener.run(ClientListener.java:140)
On Mon, May 14, 2018 at 4:10 PM, Christian Grün < christian.gruen@gmail.com> wrote:
Hi Halit,
Thanks for reporting the stack trace. The only thing that I can derive from your error message is that something has gone wrong with your database in an earlier state. Unfortunately, it’s difficult to guess what might have been the reason. Feel free to give us an update if you should manage to get the issue reproduced and reproducible for us.
Best, Christian
On Wed, May 9, 2018 at 4:28 PM, halit tiryaki halit.tiryaki@gmail.com wrote:
I'm using the C# driver. but still, it doesn't look like a high level problem. neither a query problem. thats why I didn't post some code snippets. To make sure, I just tried to execute the INSPECT command
in the
CLI. same exception. Also the db:optimize fails with the same
exception,
although with a different stacktrace, which I will post below.
Altough the race condition point of two processes interfering is a
good
point, I'm currently having trouble to do anything useful with a
session
from which a race condition would matter. If, then it must have
happened
while the import was running along with the other, non-importing
process.
As I mentioned in the first post, while the import process was still ongoing, the other process already started to have the exception at a
very
late progress of the import. After the import was done, I was still
able to
operate on the db as normal, but only with the session in the process
which
did the long-run import.
To restore the second process its session state to normal, I
restarted the
webapplication, which closes both sessions. After that, I'm no longer
to
operate on the db. Not even directly from the CLI.
Seems it is a sort of a persisted data state which the low level
basex code
can't handle when trying to reread it into to the memory.
I 've already googled around for the same exception in BaseX, seems
it was
an issue in some older versions too.
here the stacktrace when running db:optimize:
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 9.0.2 beta Java: Oracle Corporation, 1.8.0_151 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: 52 at org.basex.util.hash.TokenSet.key(TokenSet.java:128) at org.basex.data.Data.name(Data.java:388) at org.basex.io.serial.Serializer.node(Serializer.java:414) at org.basex.io.serial.Serializer.node(Serializer.java:158) at org.basex.io.serial.Serializer.node(Serializer.java:345) at org.basex.io.serial.Serializer.node(Serializer.java:158) at org.basex.io.serial.Serializer.serialize(Serializer.java:109) at org.basex.core.cmd.OptimizeAll$DBParser.parse(OptimizeAll.java:200) at org.basex.build.Builder.parse(Builder.java:77) at org.basex.build.DiskBuilder.build(DiskBuilder.java:77) at org.basex.core.cmd.OptimizeAll.optimizeAll(OptimizeAll.java:122) at
org.basex.query.up.primitives.db.DBOptimize.apply(DBOptimize.java:124)
at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175) at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120) at org.basex.query.up.Updates.apply(Updates.java:157) at org.basex.query.QueryContext.iter(QueryContext.java:341) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at org.basex.core.cmd.AQuery.query(AQuery.java:92) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.server.ClientListener.run(ClientListener.java:140)
On Wed, May 9, 2018 at 11:45 AM, Alexander Holupirek <
alex@holupirek.de>
wrote:
Hi,
(please also respond to the list. Others might have suggestions as
well
;-)
it still would be great if you could provide even more details. For instance, what programming language do you use?Ideally include a
Short, Self
Contained, Correct (Compilable), Example (SSCCE) [1] in order to let
people
reproduce the behaviour.
From a high level perspective, are you sure the sessions do not
interfere?
Do you additionally work with BaseXGUI on the database, while your
program
is importing?
Cheers, Alex
> On 9. May 2018, at 09:26, halit tiryaki halit.tiryaki@gmail.com
wrote:
> > hello, thanks for the fast response. > > It happened in the process of a large Import. There were two
processes
> with each a respective Session to BaseX. One process with the Open
session
> which did the Import was doing fine. The Second process however
Starter
> having the mentioned exception. After stopping Both processes and
restarting
> the app, no basex Session is now able to operate normal. > > The ArrayIndexOutOfBoundsException already comes in executing the > command INSPECT. > > The db-Info returns following: > > > info db: > Database Properties > NAME: XXX > SIZE: 5877 kB > NODES: 124051 > DOCUMENTS: 5497 > BINARIES: 0 > TIMESTAMP: 2018-05-09T05:20:44.000Z > UPTODATE: false > > Resource Properties > INPUTPATH: > INPUTSIZE: 0 b > INPUTDATE: 2018-05-09T04:32:10.539Z > > Indexes > TEXTINDEX: true > ATTRINDEX: true > TOKENINDEX: false > FTINDEX: false > TEXTINCLUDE: /item_User/node()/User:Email > ATTRINCLUDE: /item_UserImage/node()/UserImage:Id > TOKENINCLUDE: > FTINCLUDE: /item_XXX/node()/XXX:[german] > LANGUAGE: English > STEMMING: false > CASESENS: false > DIACRITICS: false > STOPWORDS: > UPDINDEX: true > AUTOOPTIMIZE: false > MAXCATS: 100 > MAXLEN: 96 > SPLITSIZE: 0
> Alexander Holupirek alex@holupirek.de schrieb am Mi., 9. Mai
2018
> 08:44: > Hi, > > could you please provide some more details? > The best would be a reproducible example. > > Thanks, > Alex > > > > On 9. May 2018, at 06:50, halit tiryaki <halit.tiryaki@gmail.com
> > wrote: > > > > Hello, > > > > got a serious problem here. any ideas? thanks > > > > Improper use? Potential bug? Your feedback is welcome: > > Contact: basex-talk@mailman.uni-konstanz.de > > Version: BaseX 9.0.2 beta > > Java: Oracle Corporation, 1.8.0_151 > > OS: Linux, amd64 > > Stack Trace: > > java.lang.ArrayIndexOutOfBoundsException: 4288 > > at
org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:151)
> > at org.basex.data.Data.kind(Data.java:294) > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:332) > > at org.basex.query.value.node.DBNode$4.next(DBNode.java:323) > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:38) > > at org.basex.query.expr.path.IterStep$1.next(IterStep.java:32) > > at org.basex.query.QueryContext.next(QueryContext.java:392) > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:50) > > at org.basex.query.expr.path.IterPath$1.next(IterPath.java:34) > > at org.basex.query.expr.ParseExpr.item(ParseExpr.java:58) > > at org.basex.query.expr.ParseExpr.atomItem(ParseExpr.java:84) > > at org.basex.query.func.fn.FnConcat.item(FnConcat.java:20) > > at org.basex.query.expr.ParseExpr.value(ParseExpr.java:71) > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > > at
org.basex.query.expr.path.CachedPath.nodeIter(CachedPath.java:36)
> > at org.basex.query.expr.path.AxisPath.iter(AxisPath.java:69) > > at org.basex.query.expr.gflwor.For$1.next(For.java:107) > > at org.basex.query.expr.gflwor.OrderBy$1.sort(OrderBy.java:73) > > at org.basex.query.expr.gflwor.OrderBy$1.next(OrderBy.java:54) > > at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:72) > > at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:177) > > at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:87) > > at org.basex.query.QueryContext.next(QueryContext.java:392) > > at org.basex.query.scope.MainModule$1.next(MainModule.java:122) > > at org.basex.core.cmd.AQuery.query(AQuery.java:94) > > at org.basex.core.cmd.XQuery.run(XQuery.java:22) > > at org.basex.core.Command.run(Command.java:257) > > at org.basex.core.Command.execute(Command.java:93) > > at org.basex.server.ClientListener.run(ClientListener.java:140) >