Hi Christian,
Thank you for pointing out ADDCACHE option. It's working now.
Regards, Paul
-----Original Message----- From: BaseX-Talk [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of basex-talk-request@mailman.uni-konstanz.de Sent: Tuesday, November 20, 2018 12:09 AM To: basex-talk@mailman.uni-konstanz.de Subject: BaseX-Talk Digest, Vol 107, Issue 30
Send BaseX-Talk mailing list submissions to basex-talk@mailman.uni-konstanz.de
To subscribe or unsubscribe via the World Wide Web, visit https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk or, via email, send a message with subject or body 'help' to basex-talk-request@mailman.uni-konstanz.de
You can reach the person managing the list at basex-talk-owner@mailman.uni-konstanz.de
When replying, please edit your Subject line so it is more specific than "Re: Contents of BaseX-Talk digest..."
Today's Topics:
1. Re: db:add fails to add large XML file (Christian Gr?n) 2. Re: OverlappingFileLockException when using db:add concurrently for large XML files (Christian Gr?n)
----------------------------------------------------------------------
Message: 1 Date: Tue, 20 Nov 2018 09:06:54 +0100 From: Christian Gr?n christian.gruen@gmail.com To: "Han (US), Paul P" Paul.P.Han@boeing.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] db:add fails to add large XML file Message-ID: CAP94bnOMv4qmuFjGDaXn0evJKYwgeyc2bQs8ONGT+CKe-fxqVA@mail.gmail.com Content-Type: text/plain; charset="utf-8"
Hi Paul,
Too large documents should actually be rejected with a proper error message; but maybe you get around the problem by enabling the ADDCACHE option.
Best, Christian
Am Di., 20. Nov. 2018, 01:50 hat Han (US), Paul P Paul.P.Han@boeing.com geschrieben:
Hello,
We encountered the following error intermittently when using db:add within the XQuery on a large XML.
Regards,
Paul
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]POST failed with response 400: Improper use? Potential bug? Your feedback is welcome:
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT Contact: basex-talk@mailman.uni-konstanz.de
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT Version: BaseX 9.1 beta
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT Java: Oracle Corporation, 1.8.0_162
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT OS: Linux, amd64
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT Stack Trace:
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT java.lang.RuntimeException: Data Access out of bounds:
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT - pre value: 2053429413
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT - table size: -2143755530
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT - first/next pre value: -2143755776/-2143755530
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT - #total/used pages: 8403171/8403171
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT - accessed page: 8403170 (8403171 > 8403170]
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.util.Util.notExpected(Util.java:61)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:456)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.io.random.TableDiskAccess.read4(TableDiskAccess.java:165)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.data.Data.id(Data.java:294)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.data.Data.insert(Data.java:828)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.query.up.primitives.db.DBAdd.apply(DBAdd.java:67)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.query.up.DataUpdates.apply(DataUpdates.java:175)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.query.up.ContextModifier.apply(ContextModifier.java:120)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.query.up.Updates.apply(Updates.java:156)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.query.QueryContext.iter(QueryContext.java:350)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.core.cmd.AQuery.query(AQuery.java:109)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.core.cmd.XQuery.run(XQuery.java:22)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.core.Command.run(Command.java:257)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.http.rest.RESTCmd.run(RESTCmd.java:105)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.http.rest.RESTQuery.query(RESTQuery.java:69)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.http.rest.RESTQuery.run0(RESTQuery.java:37)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.http.rest.RESTCmd.run(RESTCmd.java:70)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.core.Command.run(Command.java:257)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.core.Command.execute(Command.java:93)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.core.Command.execute(Command.java:116)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.http.rest.RESTServlet.run(RESTServlet.java:27)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.basex.http.BaseXServlet.service(BaseXServlet.java:59)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appli cationFilterChain.java:230)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFi lterChain.java:165)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appli cationFilterChain.java:192)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFi lterChain.java:165)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.filters.CorsFilter.handleNonCORS(CorsFilter.java:4 19)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.filters.CorsFilter.doFilter(CorsFilter.java:169)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appli cationFilterChain.java:192)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFi lterChain.java:165)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperVa lve.java:198)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.StandardContextValve.invoke(StandardContextVa lve.java:96)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticat orBase.java:474)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.ja va:140)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.ja va:79)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAcces sLogValve.java:624)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValv e.java:87)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java :349)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java: 783)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLigh t.java:66)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractP rotocol.java:798)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoi nt.java:1434)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase .java:49)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j ava:1149)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. java:624)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThr ead.java:61)
2018-11-19T16:10:45.80-0800 [APP/PROC/WEB/0]OUT at java.lang.Thread.run(Thread.java:748)