Greetings,
We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached).
Does anyone have experience with hw to handle this?
Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64
Thank you, Robert
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474) at org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180) at org.basex.data.Data.textRef(Data.java:467) at org.basex.data.DiskData.text(DiskData.java:272) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1) at org.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108) at org.basex.query.expr.IterMap.value(IterMap.java:106) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.constr.Constr.add(Constr.java:72) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.List$1.iter(List.java:213) at org.basex.query.expr.List$1.next(List.java:181) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147) at org.basex.query.func.FuncCall.value(FuncCall.java:53) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:840)
Hi Robert,
The size in the screenshot indicates that your input may have exceeded some limits for a single database instance [1]. Usually, an insert operation that would cause an overflow is rejected. Based on your database logs, can you trace back what was the last seemingly successful insert, and the first operation that raised the error message?
With regard to the database optimization, does it make a difference if you choose "Full optimization"?
Best, Christian
[1] https://docs.basex.org/main/Statistics
On Wed, Aug 7, 2024 at 11:07 PM Chavez, Robert r.chavez@northeastern.edu wrote:
Greetings,
We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached).
Does anyone have experience with hw to handle this?
Version: BaseX 10.4
Java: Amazon.com Inc., 17.0.12
OS: Linux, amd64
Thank you,
Robert
Unexpected error: Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 10.4
Java: Amazon.com Inc., 17.0.12
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322
atorg.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514)
atorg.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474)
atorg.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180)
at org.basex.data.Data.textRef(Data.java:467) at org.basex.data.DiskData.text(DiskData.java:272) atorg.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96)
atorg.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1)
atorg.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108)
at org.basex.query.expr.IterMap.value(IterMap.java:106) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.constr.Constr.add(Constr.java:72) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.List$1.iter(List.java:213) at org.basex.query.expr.List$1.next(List.java:181) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) atorg.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147)
at org.basex.query.func.FuncCall.value(FuncCall.java:53) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) atorg.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79)
at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) atorg.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
atorg.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
atorg.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
atorg.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
atorg.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
atorg.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
atorg.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
atorg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
atorg.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
atorg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
atorg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
atorg.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
atorg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
atorg.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
atorg.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
atorg.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
atorg.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)
atorg.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
atjava.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
atjava.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
atorg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:840)
Thank you for the suggestion Christian.
In the logs everything seems to be working just fine until this one call – which returns the error “Out of Main Memory”. However, looking at the systems logs themselves, the system reports ample memory at that time. I’m not exactly sure when BaseX reported that error at that time.
After that initial “Out of Main Memory” error, all requests (PUT requests, simple GUI requests) fail with either:
java.lang.ArrayIndexOutOfBoundsException
-or
java.lang.NegativeArraySizeException
The only thing I can think of is that he system was reaching maximum harddisk capacity and maybe ran out of wiggle room at that point.
3:59:40.282 10.112.72.182:36826 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-11-p194.xml 13:59:43.889 10.112.72.182:36826 basex-rest 500 Out of Main Memory. 3607.46 ms 13:59:45.358 10.112.72.182:36828 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-09-p165.xml 13:59:45.746 10.112.72.182:36828 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Free slot exceeds file offset: 49357094248 + 110461 > 49357197312 at org.basex.util.Uti14:08:00.444 10.112.72.182:37070 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException 9.96 ms 14:08:05.549 10.112.72.182:37074 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v51-1803-08-p001.xml 14:08:05.556 10.112.72.182:37074 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException: -1236248751 at org.basex.io.random.DataAccess.readBytes(DataAccess.java:196) at org.basex.io.random.DataAccess.readToken(DataAccess.java:175) at org.basex.io.random.DataAccess.readToken(DataAccess.java:166) at org.basex.data.DiskData.txt(DiskData.java:306) at org.basex.data.DiskData.text(DiskData.java:273) at org.basex.index.value.DiskValues.key(DiskValues.java:430) at org.basex.index.value.DiskValues.indexEntry(DiskValues.java:330) at org.basex.index.value.DiskValues.get(DiskValues.java:206) at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:98) at org.basex.data.Data.indexDelete(Data.java:1083) at org.basex.data.Data.delete(Data.java:683) at org.basex.query.up.atomic.Replace.apply(Replace.java:57) at org.basex.query.up.atomic.AtomicUpdateCache.ap... 7.06 ms 14:08:10.590 10.112.72.182:370
From: Christian Grün christian.gruen@gmail.com Date: Friday, August 9, 2024 at 5:06 AM To: Chavez, Robert r.chavez@northeastern.edu Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Basex java.lang.ArrayIndexOutOfBoundsException Hi Robert,
The size in the screenshot indicates that your input may have exceeded some limits for a single database instance [1]. Usually, an insert operation that would cause an overflow is rejected. Based on your database logs, can you trace back what was the last seemingly successful insert, and the first operation that raised the error message?
With regard to the database optimization, does it make a difference if you choose "Full optimization"?
Best, Christian
[1] https://docs.basex.org/main/Statistics
On Wed, Aug 7, 2024 at 11:07 PM Chavez, Robert <r.chavez@northeastern.edumailto:r.chavez@northeastern.edu> wrote: Greetings,
We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached).
Does anyone have experience with hw to handle this?
Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64
Thank you, Robert
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474) at org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180) at org.basex.data.Data.textRef(Data.java:467) at org.basex.data.DiskData.text(DiskData.java:272) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1) at org.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108) at org.basex.query.expr.IterMap.value(IterMap.java:106) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.constr.Constr.add(Constr.java:72) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.List$1.iter(List.java:213) at org.basex.query.expr.List$1.next(List.java:181) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147) at org.basex.query.func.FuncCall.value(FuncCall.java:53) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:840)
Hi Robert,
In fact out of memory errors are always critical in Java. Maybe you will need (well, would have needed) to adjust the memory that is used by the JVM with the Xmx flag [1]. What value is currently assigned?
Best, Christian
[1] https://docs.basex.org/main/Start_Scripts
On Mon, Aug 19, 2024 at 3:37 PM Chavez, Robert r.chavez@northeastern.edu wrote:
Thank you for the suggestion Christian.
In the logs everything seems to be working just fine until this one call – which returns the error “Out of Main Memory”. However, looking at the systems logs themselves, the system reports ample memory at that time. I’m not exactly sure when BaseX reported that error at that time.
After that initial “Out of Main Memory” error, all requests (PUT requests, simple GUI requests) fail with either:
java.lang.ArrayIndexOutOfBoundsException
-or
java.lang.NegativeArraySizeException
The only thing I can think of is that he system was reaching maximum harddisk capacity and maybe ran out of wiggle room at that point.
3:59:40.282 10.112.72.182:36826 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-11-p194.xml
13:59:43.889 10.112.72.182:36826 basex-rest 500 Out of Main Memory. 3607.46 ms
13:59:45.358 10.112.72.182:36828 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-09-p165.xml
13:59:45.746 10.112.72.182:36828 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Free slot exceeds file offset: 49357094248 + 110461 > 49357197312 at org.basex.util.Uti14:08:00.444 10.112.72.182:37070 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException 9.96 ms
14:08:05.549 10.112.72.182:37074 basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v51-1803-08-p001.xml
14:08:05.556 10.112.72.182:37074 basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException: -1236248751 at org.basex.io.random.DataAccess.readBytes(DataAccess.java:196) at org.basex.io.random.DataAccess.readToken(DataAccess.java:175) at org.basex.io.random.DataAccess.readToken(DataAccess.java:166) at org.basex.data.DiskData.txt(DiskData.java:306) at org.basex.data.DiskData.text(DiskData.java:273) at org.basex.index.value.DiskValues.key(DiskValues.java:430) at org.basex.index.value.DiskValues.indexEntry(DiskValues.java:330) at org.basex.index.value.DiskValues.get(DiskValues.java:206) at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:98) at org.basex.data.Data.indexDelete(Data.java:1083) at org.basex.data.Data.delete(Data.java:683) at org.basex.query.up.atomic.Replace.apply(Replace.java:57) at org.basex.query.up.atomic.AtomicUpdateCache.ap... 7.06 ms
14:08:10.590 10.112.72.182:370
*From: *Christian Grün christian.gruen@gmail.com *Date: *Friday, August 9, 2024 at 5:06 AM *To: *Chavez, Robert r.chavez@northeastern.edu *Cc: *basex-talk@mailman.uni-konstanz.de < basex-talk@mailman.uni-konstanz.de> *Subject: *Re: [basex-talk] Basex java.lang.ArrayIndexOutOfBoundsException
Hi Robert,
The size in the screenshot indicates that your input may have exceeded some limits for a single database instance [1]. Usually, an insert operation that would cause an overflow is rejected. Based on your database logs, can you trace back what was the last seemingly successful insert, and the first operation that raised the error message?
With regard to the database optimization, does it make a difference if you choose "Full optimization"?
Best,
Christian
[1] https://docs.basex.org/main/Statistics
On Wed, Aug 7, 2024 at 11:07 PM Chavez, Robert r.chavez@northeastern.edu wrote:
Greetings,
We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached).
Does anyone have experience with hw to handle this?
Version: BaseX 10.4
Java: Amazon.com Inc., 17.0.12
OS: Linux, amd64
Thank you,
Robert
Unexpected error: Improper use? Potential bug? Your feedback is welcome:
Contact: basex-talk@mailman.uni-konstanz.de
Version: BaseX 10.4
Java: Amazon.com Inc., 17.0.12
OS: Linux, amd64
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322
atorg.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514)
atorg.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474)
atorg.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180)
at org.basex.data.Data.textRef(Data.java:467) at org.basex.data.DiskData.text(DiskData.java:272) atorg.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96)
atorg.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1)
atorg.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108)
at org.basex.query.expr.IterMap.value(IterMap.java:106) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.constr.Constr.add(Constr.java:72) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.List$1.iter(List.java:213) at org.basex.query.expr.List$1.next(List.java:181) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) atorg.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147)
at org.basex.query.func.FuncCall.value(FuncCall.java:53) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) atorg.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79)
at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) atorg.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
atorg.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
atorg.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
atorg.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
atorg.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
atorg.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
atorg.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
atorg.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
atorg.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
atorg.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
atorg.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
atorg.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
atorg.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
atorg.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
atorg.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
atorg.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
atorg.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589)
atorg.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
atjava.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
atjava.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
atorg.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:840)
Ah, good point. I’m running BaseX in Tomcat so I will have to dig into the config to see where Xmx flag is set. And what the memory relationship is between Tomcat and BaseX
Robert
From: Christian Grün christian.gruen@gmail.com Date: Monday, August 19, 2024 at 9:50 AM To: Chavez, Robert r.chavez@northeastern.edu Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Basex java.lang.ArrayIndexOutOfBoundsException Hi Robert,
In fact out of memory errors are always critical in Java. Maybe you will need (well, would have needed) to adjust the memory that is used by the JVM with the Xmx flag [1]. What value is currently assigned?
Best, Christian
[1] https://docs.basex.org/main/Start_Scripts
On Mon, Aug 19, 2024 at 3:37 PM Chavez, Robert <r.chavez@northeastern.edumailto:r.chavez@northeastern.edu> wrote: Thank you for the suggestion Christian.
In the logs everything seems to be working just fine until this one call – which returns the error “Out of Main Memory”. However, looking at the systems logs themselves, the system reports ample memory at that time. I’m not exactly sure when BaseX reported that error at that time.
After that initial “Out of Main Memory” error, all requests (PUT requests, simple GUI requests) fail with either:
java.lang.ArrayIndexOutOfBoundsException
-or
java.lang.NegativeArraySizeException
The only thing I can think of is that he system was reaching maximum harddisk capacity and maybe ran out of wiggle room at that point.
3:59:40.282 10.112.72.182:36826http://10.112.72.182:36826/ basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-11-p194.xml 13:59:43.889 10.112.72.182:36826http://10.112.72.182:36826/ basex-rest 500 Out of Main Memory. 3607.46 ms 13:59:45.358 10.112.72.182:36828http://10.112.72.182:36828/ basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v31-1819-09-p165.xml 13:59:45.746 10.112.72.182:36828http://10.112.72.182:36828/ basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Free slot exceeds file offset: 49357094248 + 110461 > 49357197312 at org.basex.util.Uti14:08:00.444 10.112.72.182:37070http://10.112.72.182:37070/ basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException 9.96 ms 14:08:05.549 10.112.72.182:37074http://10.112.72.182:37074/ basex-rest REQUEST [PUT] /basex/rest/psc/jqa/jqadiaries-v51-1803-08-p001.xml 14:08:05.556 10.112.72.182:37074http://10.112.72.182:37074/ basex-rest 500 Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Red Hat, Inc., 11.0.16 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException: -1236248751 at org.basex.io.random.DataAccess.readBytes(DataAccess.java:196) at org.basex.io.random.DataAccess.readToken(DataAccess.java:175) at org.basex.io.random.DataAccess.readToken(DataAccess.java:166) at org.basex.data.DiskData.txt(DiskData.java:306) at org.basex.data.DiskData.text(DiskData.java:273) at org.basex.index.value.DiskValues.key(DiskValues.java:430) at org.basex.index.value.DiskValues.indexEntry(DiskValues.java:330) at org.basex.index.value.DiskValues.get(DiskValues.java:206) at org.basex.index.value.UpdatableDiskValues.delete(UpdatableDiskValues.java:98) at org.basex.data.Data.indexDelete(Data.java:1083) at org.basex.data.Data.delete(Data.java:683) at org.basex.query.up.atomic.Replace.apply(Replace.java:57) at org.basex.query.up.atomic.AtomicUpdateCache.ap... 7.06 ms 14:08:10.590 10.112.72.182:370http://10.112.72.182:370/
From: Christian Grün <christian.gruen@gmail.commailto:christian.gruen@gmail.com> Date: Friday, August 9, 2024 at 5:06 AM To: Chavez, Robert <r.chavez@northeastern.edumailto:r.chavez@northeastern.edu> Cc: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Basex java.lang.ArrayIndexOutOfBoundsException Hi Robert,
The size in the screenshot indicates that your input may have exceeded some limits for a single database instance [1]. Usually, an insert operation that would cause an overflow is rejected. Based on your database logs, can you trace back what was the last seemingly successful insert, and the first operation that raised the error message?
With regard to the database optimization, does it make a difference if you choose "Full optimization"?
Best, Christian
[1] https://docs.basex.org/main/Statistics
On Wed, Aug 7, 2024 at 11:07 PM Chavez, Robert <r.chavez@northeastern.edumailto:r.chavez@northeastern.edu> wrote: Greetings,
We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached).
Does anyone have experience with hw to handle this?
Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64
Thank you, Robert
Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474) at org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180) at org.basex.data.Data.textRef(Data.java:467) at org.basex.data.DiskData.text(DiskData.java:272) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1) at org.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108) at org.basex.query.expr.IterMap.value(IterMap.java:106) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.constr.Constr.add(Constr.java:72) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.List$1.iter(List.java:213) at org.basex.query.expr.List$1.next(List.java:181) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147) at org.basex.query.func.FuncCall.value(FuncCall.java:53) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:840)
basex-talk@mailman.uni-konstanz.de
-
Chavez, Robert -
Christian Grün