BaseX-Talk

basex-talk@mailman.uni-konstanz.de

4 participants
5268 discussions

Problem with XQJ-Implementation (method XQResultSequence.getSequenceAsStream)
by Isidre Guixà 03 Apr '12

03 Apr '12

Hi I am experiencing the BaseX API XQJ and I compare to other XQJ NXD and I find the following problem. Suppose we have the next XML file in database "test": <?xml version="1.0" encoding="ISO-8859-1"?> <bookstore param="hao"> <book> <title>Book01</title> </book> <book> <title>Book02</title> </book> </bookstore> If I execute: XQDataSource xqs = ... XQConnection conn = xqs.getConnection(); XQExpression xqe = conn.createExpression(); String cad= "collection('test')//title"; XQResultSequence xqrs = xqe.executeQuery(cad); XMLStreamReader result = xqrs.getSequenceAsStream(); System.out.println("Iteration on XMLStreamReader..."); for(; result.hasNext(); result.next()) System.out.println(getEventString(result)); xqrs = xqe.executeQuery(cad); System.out.println("Using a Transformer..."); Source resultXML = new StAXSource(xqrs.getSequenceAsStream()); StreamResult resultSR = new StreamResult(System.out); TransformerFactory factory = TransformerFactory.newInstance(); Transformer tr = factory.newTransformer(); tr.setOutputProperty(OutputKeys.INDENT, "yes"); tr.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no"); tr.setOutputProperty("{http://xml.apache.org/xslt}indent-amount","2"); tr.transform(resultXML, resultSR); where static String getEventString(final XMLStreamReader reader) { switch(reader.getEventType()) { case XMLStreamConstants.START_ELEMENT: return "START_ELEMENT:\t\"" + reader.getLocalName() + "\""; case XMLStreamConstants.END_ELEMENT: return "END_ELEMENT:\t\"" + reader.getLocalName() + "\""; case XMLStreamConstants.START_DOCUMENT: return "START_DOCUMENT"; case XMLStreamConstants.END_DOCUMENT: return "END_DOCUMENT"; case XMLStreamConstants.CHARACTERS: return "CHARACTERS:\t\"" + reader.getText() + "\""; } return ""; } I get: Iteration on XMLStreamReader... START_ELEMENT: "title" CHARACTERS: "Book01" END_ELEMENT: "title" END_DOCUMENT START_ELEMENT: "title" CHARACTERS: "Book02" END_ELEMENT: "title" END_DOCUMENT Using a Transformer... <?xml version="1.0" encoding="UTF-8"?> <title>Book01</title> It's not good! When I use a Transformer I should get something similar to: <?xml version="1.0" encoding="UTF-8"?> <result> <title>Book01</title> <title>Book02</title> <result> That is, if result sequence length is greater than 1, getSequenceAsStream should wrap the sequence in a containing result element, so to not break JAXP Transformers and XMLStreamReader events. Is there any solution in BaseX ? Thanks a lot! Isidre Guixà

3 2

Concurrency problems with REST API
by Tobias Hofer 01 Apr '12

01 Apr '12

Hi, I'm using the REST API of BaseX 7.2 and run into concurrency problems. If I add new documents in parallel then the server will stop working after adding a few documents. My command template looks like: <?xml version="1.0" encoding="UTF-8"?> <command xmlns="http://basex.org/rest"><text><![CDATA[ ADD TO $KEY$ <node id="$KEY$" /> ]]></text></command> I call the REST API using curl: for ((i=0; i<10; i++)); do (cp tx-template tx$i; replace '$KEY$' $i -- tx$i; curl -i -X POST -H "Content-Type: application/xml" -T tx$i "admin:admin@localhost:8984/rest/QDEV") & done After four or less insert the following error occurs: HTTP/1.1 400 Bad Request Content-Type: text/plain; charset=utf-8 Content-Length: 2247 Server: Jetty(6.1.26) java.lang.NullPointerExceptionjava.lang.NullPointerExceptionjava.lang.NullPointerExceptionorg.xml.sax.SAXParseException: cvc-type.3.1.2: Element 'command' is a simple type, so it must have no element information item [children].org.xml.sax.SAXParseException: cvc-type.3.1.2: Element 'command' is a simple type, so it must have no element information item [children].Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.2 Java: Sun Microsystems Inc., 1.6.0_26 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException org.basex.core.cmd.Add.run(Add.java:92) org.basex.core.Command.run(Command.java:345) org.basex.core.Command.exec(Command.java:324) org.basex.core.Command.execute(Command.java:76) org.basex.server.LocalSession.execute(LocalSession.java:134) org.basex.server.LocalSession.execute(LocalSession.java:126) org.basex.server.Session.execute(Session.java:51) org.basex.http.rest.RESTCommand.run(RESTCommand.java:40) org.basex.http.rest.RESTPost.run(RESTPost.java:129) org.basex.http.rest.RESTServlet.run(RESTServlet.java:14) org.basex.http.BaseXServlet.service(BaseXServlet.java:38) javax.servlet.http.HttpServlet.service(HttpServlet.java:820) org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) org.mortbay.jetty.handler.HandlerList.handle(HandlerList.java:49) org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) org.mortbay.jetty.Server.handle(Server.java:326) org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) Now, the server cannot be accessed any more. Any further call will return "Database 'MYDB' is currently opened by another client. A restart helps. The error does not happen if the commands are executed in sequence. Please help. Regards, Tobias Hofer

2 1

possible bug? or opportunity for documentation enhancement? [2d sending]
by C. M. Sperberg-McQueen 01 Apr '12

01 Apr '12

[2d posting - this time with the promised attachment] I got some error messages today which seem to indicate a problem, either in BaseX or in (my understanding of) the documentation. Since the error messages (in the attached log) suggest contacting you about it, I am doing so. The document at http://www.w3.org/TR/xsd-unicode-blocknames/ includes a table of Unicode block names created by querying a database consisting of XML representations of the various versions of the Unicode Database. Today I noticed that a new version of the Unicode Database was issued in January, and so I downloaded the XML data and added the document to my existing BaseX database of Unicode information. The command seemed to run successfully, but when I ran my old query, I got an error message saying "Data access out of bounds". I installed version 7.2 then, which gave the same error message. Rebuilding the database from scratch allowed me to run the query without any problems, so I don't actually have a problem at the moment. But I'm sending this problem report in the hopes that it may help you diagnose either a problem with dynamic adds to a database or perhaps a problem with the server command language documentation (it's entirely possible that I did something wrong when adding the new document to the database). If there is any information I can provide that would be helpful, please let me know. best, Michael Sperberg-McQueen -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

2 1

possible bug? or opportunity for documentation enhancement?
by C. M. Sperberg-McQueen 30 Mar '12

30 Mar '12

I got some error messages today which seem to indicate a problem, either in BaseX or in (my understanding of) the documentation. Since the error messages (in the attached log) suggest contacting you about it, I am doing so. The document at http://www.w3.org/TR/xsd-unicode-blocknames/ includes a table of Unicode block names created by querying a database consisting of XML representations of the various versions of the Unicode Database. Today I noticed that a new version of the Unicode Database was issued in January, and so I downloaded the XML data and added the document to my existing BaseX database of Unicode information. The command seemed to run successfully, but when I ran my old query, I got an error message saying "Data access out of bounds". I installed version 7.2 then, which gave the same error message. Rebuilding the database from scratch allowed me to run the query without any problems, so I don't actually have a problem at the moment. But I'm sending this problem report in the hopes that it may help you diagnose either a problem with dynamic adds to a database or perhaps a problem with the server command language documentation (it's entirely possible that I did something wrong when adding the new document to the database). If there is any information I can provide that would be helpful, please let me know. best, Michael Sperberg-McQueen -- **************************************************************** * C. M. Sperberg-McQueen, Black Mesa Technologies LLC * http://www.blackmesatech.com * http://cmsmcq.com/mib * http://balisage.net ****************************************************************

1 0

Module import times and caching
by Charles Duffy 30 Mar '12

30 Mar '12

import module namespace reporting="http://dyfis.net/reporting"; 1+1 Timing: - Parsing: 4.33 ms - Compiling: 4205.89 ms <<< (!) - Evaluating: 7.55 ms - Printing: 6.39 ms - Total Time: 4224.18 ms ...after some analysis and optimization this is down to ~115ms, but that's still a fairly high floor for query execution time of anything using this module (which almost everything I write does). Said module includes several lines of the form declare variable $reporting:foo_db := db:open("foo", "db") ...and a lot of function declarations, some of which are fairly complex. Have there been thoughts to a module cache, even if enabled only via pragma? It would be nice to be able to reuse the optimizer's results and variable definitions (even if we completely expired the cache on writes to any database referenced during the module's compilation phase).

1 0

Java interop - Unexpected FORG0006 ("Invalid arguments")
by Charles Duffy 30 Mar '12

30 Mar '12

Howdy -- I have a thin wrapper around the 3rd-party svnkit library. At present, it provides two functions: module namespace svnkit = "http://dyfis.net/svnkit-wrapper"; declare namespace j="java:net.dyfis.svnkit_wrapper.SvnWrapper"; declare function svnkit:cat($path as xs:string) as xs:string { j:cat($path) }; declare function svnkit:cat($path as xs:string, $rev as xs:integer) as xs:string { j:cat($path, $rev) }; SvnWrapper (written in Clojure) provides both versions of cat as static methods: (ns net.dyfis.svnkit-wrapper.core (:gen-class :name net.dyfis.svnkit_wrapper.SvnWrapper :methods [#^{:static true} [cat [java.lang.String] java.lang.String] #^{:static true} [cat [java.lang.String int] java.lang.String]])) svnkit:cat($path) works fine. svnkit:cat($path, $revno) results in the following: [FORG0006] Invalid arguments: SvnWrapper.cat(xs:string, xs:integer) found. SvnWrapper.cat(xs:string, xs:integer) is exactly what I *expect* it to find, so the error doesn't make a great deal of sense to me. Any hints?

2 3

BaseX XQuery vs. python / lxml performance
by Ronny Möbius 30 Mar '12

30 Mar '12

Hi, we use now for a couple of years XML files for organizing the schedule of lectures in our institute. The amount data for one semester is about one megabyte. For presenting the schedule on the web we use currently python in combination with the C-based lxml XPath implementation. Only basic selecting nodes happens on the layer of lxml, but the most of intelligence is implemented in filtering python lists and lists of dictionaries. We read the xml data from hard disk and parse them into lxml objects. >From time to time we thought about storing the data in a XML database as this seems intuitive. One main point is, that we need some sort of transaction management, because different applications may manipulate the files simultaneously. Now I gave BaseX a try and implemented one basic output of our online schedule in XQuery. Soon I noticed, that all of the intelligence should be managed inside XQuery as it should return the full-prepared html for presenting online. Unfortunately we now have the impression, that we did not gain speed - to the contrary the query itself needs more execution time than the whole corresponding ready interface (including the user interface etc.) I'm now interested in your general opinion about this: Is it surprising, that the XQuery implementation than the lxml/python one (For me it is, as I thought the indices etc. created when importing the data should decrease computational affords in searching the tree)? Is there some catch in my approach? May the reason be bad designing the query? I would appreciate to hear from you, Ronny

5 7

Updates failing due to "opened by another client" on a system running only basexhttp
by Charles Duffy 30 Mar '12

30 Mar '12

Howdy -- I recently had an update run via the REST interface fail due to the database being "opened by another client". This would be unsurprising were it taking place on a workstation where a concurrent process such as basexgui might have been running, but the host in question was a server, with its basexhttp instance controlled by system init scripts, running as a dedicated user, and otherwise unlikely to be interfered with by a separate process. There are, however, multiple concurrent REST clients using that same HTTP server. As basexhttp should be the only service directly accessing the underlying database on this host, this surprises me -- my expectation is that the transaction monitor would defer any execution of an update until a full lock could be grabbed, preventing it from interfering with other read or update actions happening within the same process. Is this expectation valid? If not, what do I need to do to be able to apply updates to a database with minimal disruption to readers?

4 5

Declarations for external variables not enforced
by Charles Duffy 30 Mar '12

30 Mar '12

Howdy -- Using the REST interface, I'm currently able to do the following: <rest:query xmlns:rest="http://basex.org/rest"> <rest:text>$foo</rest:text> <rest:variable name="foo" value="bar"/> </rest:query> ...and get back "bar" as a result. Note the lack of a "declare variable $foo external" line. I haven't read the spec to determine permissibility of this behavior, but it does come in handy at times (particularly, for permitting maximally terse one-liners). Is this something likely to go away in the future? Thanks!

4 3

Out-of-band logging from XQuery functions
by Charles Duffy 29 Mar '12

29 Mar '12

Howdy -- Is any mechanism available for queries to log errors out-of-band without modifying their return values? Something like a wrapper around log4j would be perfect; right now I'm using file:append(), but as this requires my queries to know where the system administrator wants things to be logged to (which should be an entirely separate concern), it's very far from ideal.

2 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk