Hi Christian,
it works like a charm. Neither of the reported bugs could be reproduced with the latest build (no matter what I've tried ;o)).
So *many* thanks for your prompt response and bugfix. This is why I love open source projects in general und especially this project: there are people who really care...
So keep this cool project going: it does everything I wanted so far!
Best, Bodo
-----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Mittwoch, 13. März 2013 23:36 An: Bodo Hüsemann Cc: BaseX Betreff: Re: [basex-talk] Bugreport: org.basex.server.ClientQuery.cache(ClientQuery.java:81)
Bodo,
thanks for your in-depth bug report, which allowed me to track down and fix the bug. Could you check out the latest stable snapshot [1] and give us some feedback if the problem is resolved?
Best, Christian
[1] http://files.basex.org/releases/latest/ ___________________________
On Wed, Mar 13, 2013 at 9:06 PM, Bodo Hüsemann bodo.huesemann@gmx.de wrote:
Hi Christian,
I've tried to set GLOBALLOCK to TRUE today and this prevents the error.
a) I performed some more tests and found some more other errors (without global locking, see attached log). Anyhow, I suspect an encoding issue for specific input data so included the documents I have saved in the DB.
b) I've added mySession.close() to the finally clause but the error still persists.
c) Following a query which reproduces the problem (as you can see, I
declare
a parameter variable and call xq:getResult). Every call of the query will
be
concurrently executed to corresponding parallel uploads of documents that are read from an internet database meanwhile.
module namespace xq = "xq"; import module namespace html = 'html' at 'html.xq';
declare function xq:insertTable($inGuild as xs:anyAtomicType) as
element()*
{ let $guild_sheet :=
collection('default')/apiresponse[guild/@name=$inGuild]
return if ($guild_sheet) then <tr id='{string($inGuild)}'>
<td>{string($guild_sheet/guild/@name)}</td> <td>{string($guild_sheet/guild/@memberCount)}</td> <td>{if (not(empty($guild_sheet))) then string($guild_sheet/cache_info/@cached_until_gmt) else ()}</td> </tr> else (<error id='{string($inGuild)}'/>)};
declare function xq:getResult($inGuild as xs:anyAtomicType) as element()*{ (xq:insertTable($inGuild)) };
For analysis purposes I've sent the documents to you separately.
How can we proceed (I know a selfcontained example would be nice and of great use, but this will take some time for me). Any other things we could try?
Many thanks for your help, Bodo
WARNING 2013.03.13 20:42:39 Error while loading document: 60284427BAD70B0E537DC3214A56091E3FC9DFF4 org.basex.core.BaseXException: Improper use? Potential bug? Your feedback
is
welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0_15 OS: Windows 7, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException java.lang.System.arraycopy(Native Method) org.basex.io.random.DataAccess.writeToken(DataAccess.java:347) org.basex.io.random.DataAccess.writeToken(DataAccess.java:329) org.basex.data.DiskData.index(DiskData.java:394) org.basex.data.Data.attr(Data.java:1027) org.basex.data.Data.insert(Data.java:811) org.basex.core.cmd.Add.run(Add.java:118) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.core.Command.execute(Command.java:90) org.basex.server.ClientListener.execute(ClientListener.java:386) org.basex.server.ClientListener.add(ClientListener.java:357) org.basex.server.ClientListener.run(ClientListener.java:100)
at org.basex.server.ClientSession.receive(ClientSession.java:259) at org.basex.server.ClientSession.send(ClientSession.java:246) at org.basex.server.ClientSession.send(ClientSession.java:284) at org.basex.server.ClientSession.add(ClientSession.java:134) at
de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:24
at
de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:56
) at de.infabswissarmy.control.UpdateManager.getFile(UpdateManager.java:204) at
de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:144)
at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(UnknownSource) at java.lang.Thread.run(Unknown Source)
org.basex.core.BaseXException: Improper use? Potential bug? Your feedback
is
welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0_15 OS: Windows 7, amd64 Stack Trace: java.lang.RuntimeException: Data Access out of bounds:
pre value: 4517
#used blocks: 18
#total locks: 18
access: 17 (18 > 17] org.basex.util.Util.notexpected(Util.java:53) org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:488) org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:211) org.basex.data.Data.textOff(Data.java:422) org.basex.data.DiskData.text(DiskData.java:234) org.basex.index.resource.Docs.paths(Docs.java:107) org.basex.index.resource.Docs.docs(Docs.java:230) org.basex.index.resource.Resources.docs(Resources.java:110) org.basex.query.QueryResources.collection(QueryResources.java:178) org.basex.query.func.FNGen.collection(FNGen.java:121) org.basex.query.func.FNGen.value(FNGen.java:74) org.basex.query.func.StandardFunc.compile(StandardFunc.java:48) org.basex.query.expr.Arr.compile(Arr.java:38) org.basex.query.func.StandardFunc.compile(StandardFunc.java:46) org.basex.query.QueryContext.analyze(QueryContext.java:225) org.basex.query.QueryContext.compile(QueryContext.java:209) org.basex.query.QueryProcessor.compile(QueryProcessor.java:66) org.basex.core.cmd.AQuery.query(AQuery.java:69) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145)
at org.basex.server.ClientSession.receive(ClientSession.java:259) at org.basex.server.ClientSession.execute(ClientSession.java:162) at org.basex.server.Session.execute(Session.java:50) at
de.infabswissarmy.control.XMLDBManagerBaseX.executeCommand(XMLDBManagerBaseX
.java:172) at
de.infabswissarmy.control.XMLDBManagerBaseX.executeXQuery(XMLDBManagerBaseX.
java:163) at
de.infabswissarmy.control.XMLDBManagerBaseX.executeXQuery(XMLDBManagerBaseX.
java:159) at
de.infabswissarmy.control.XMLDBManagerBaseX.existsDoc(XMLDBManagerBaseX.java
:145) at
de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:23
at
de.infabswissarmy.control.XMLDBManagerBaseX.putDoc(XMLDBManagerBaseX.java:56
) at de.infabswissarmy.control.UpdateManager.getFile(UpdateManager.java:204) at
de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:144)
at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(UnknownSource) at java.lang.Thread.run(Unknown Source) -----Ursprüngliche Nachricht----- Von: Christian Grün [mailto:christian.gruen@gmail.com] Gesendet: Sonntag, 10. März 2013 13:33 An: Bodo Hüsemann Cc: BaseX Betreff: Re: [basex-talk] Bugreport: org.basex.server.ClientQuery.cache(ClientQuery.java:81)
Hi Bodo,
thanks for your report. I would be interested to know
a) if the errors also occurs if you set the GLOBALLOCK flag in your .basex configuration file to "true" b) if the error persists if you close your client sessions via mySession.close() (in both cases, this call is always recommendable) c) which queries you are running? your query strings may be pretty
important
to find out what causes the problems, in particular if the problems only occur for GLOBALLOCK=false
Best, Christian ___________________________
On Fri, Mar 8, 2013 at 3:06 PM, Bodo Hüsemann bodo.huesemann@gmx.de
wrote:
Hi all,
I apologize for the german post on this english mailinglist...So here we go in english:
I have recently migrated my BaseX application from version 7.2 to 7.6 (and 7.6.1 Beta). The BaseX lib throws exceptions while running concurrent XQuery queries and updates (based on a threadpool), see the exceptions attached below (the specific numeric error-code changes slightly but the error occurs every time on the same line of code). The error itself seems to happen unter heavy concurrency (sequential queries have no problem) and not every time query executes.
The application code is as follows (class de.infabswissarmy.control.XMLDBManagerBaseX.query):
public ArrayList<String> query(String inQuery, String[][] inArgs){ ClientSession mySession = null; ClientQuery myQuery = null; ArrayList<String> tmpString = new ArrayList<String>(); try { mySession = getClientSession(); myQuery = mySession.query(inQuery);
// bind variable if (inArgs != null){ for (int i=0; i < inArgs.length; i++){ myQuery.bind("$" + inArgs[i][0], inArgs[i][1]); } } while (myQuery.more()){ // **exception is in this line
**
tmpString.add(myQuery.next()); } } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } finally { if (myQuery != null){ try { myQuery.close(); } catch (BaseXException e) { // ignore } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } }return tmpString; }
To be complete this is the used helper function:
private static ClientSession getClientSession() throws IOException { ClientSession mySession = new ClientSession("localhost", 1984, "admin", "admin"); return mySession; }
Version 7.2 is running without problems. In version 7.6 I have seen some rare Java heap space errors at the same location, and once I even saw a massive growth of the DB > 1GB (normal size would be 1MB), but this may be another error and I cannot reproduce it. I will try to deliver a selfcontained min-example, but this could be quite an effort to do (it will take some time to produce it). So I hope that you might find the problem with the delivered information.
Many thanks once again for this great XML-DB and thanks in advance for your help! Bodo
-- attached error stack with version 7.6.1 (in 7.6 the error occurs in row 79 instead of 81) org.basex.core.BaseXException: -146032 at org.basex.server.ClientQuery.cache(ClientQuery.java:81) at org.basex.server.Query.more(Query.java:74) at
de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86)
at
de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java:
- at
de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180)
at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) at java.util.concurrent.ThreadPoolExecutor.runWorker(UnknownSource)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(UnknownSource) at java.lang.Thread.run(Unknown Source) org.basex.core.BaseXException: 0 at org.basex.server.ClientQuery.cache(ClientQuery.java:81) at org.basex.server.Query.more(Query.java:74) at
de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86)
at
de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java:
- at
de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180)
at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) at java.util.concurrent.ThreadPoolExecutor.runWorker(UnknownSource)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(UnknownSource) at java.lang.Thread.run(Unknown Source) org.basex.core.BaseXException: java.lang.ArrayIndexOutOfBoundsException at org.basex.server.ClientQuery.cache(ClientQuery.java:81) at org.basex.server.Query.more(Query.java:74) at
de.infabswissarmy.control.XMLDBManagerBaseX.query(XMLDBManagerBaseX.java:86)
at
de.infabswissarmy.control.XMLDBManagerBaseX.queryDoc(XMLDBManagerBaseX.java:
- at
de.infabswissarmy.control.QueryTask.httpQuery(QueryTask.java:180)
at de.infabswissarmy.control.QueryTask.run(QueryTask.java:271) at java.util.concurrent.ThreadPoolExecutor.runWorker(UnknownSource)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(UnknownSource) at java.lang.Thread.run(Unknown Source)
BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk