BaseX adds bogus duplicate default namespace prefixes when inserting nodes
To follow up on my previous email, we observe the following. We run a query: declare namespace libx='http://libx.org/xml/libx2'; declare namespace atom='http://www.w3.org/2005/Atom'; declare variable $entry as xs:anyAtomicType external; declare variable $id as xs:string external; declare variable $feed as xs:anyAtomicType := doc('libx2_feed')/atom:feed; delete node $feed/atom:entry[atom:id=$id], (: INVESTIGATE: this results in wrong xmlns entries :) *insert node $entry into $feed* whereby we bind 'entry' to a DOM node using the org.w3c. Implementation. When we print the about-to-be-inserted entry from Java code prior to insertion, we see: <?xml version="1.0" encoding="UTF-8"?><entry xmlns=" http://www.w3.org/2005/Atom" xmlns:libx="http://libx.org/xml/libx2"> <id>3</id> (rest omitted) If we run a database dump via the org.basex.BaseX program (query: doc('libx2_feed')) right after the update, we see this: <entry xmlns="" xmlns="http://www.w3.org/2005/Atom" xmlns:libx=" http://libx.org/xml/libx2"> <id>3</id> <title> (rest omitted) Other than the spurious 'xmlns=""', the rest looks ok. It seems to me that BaseX should never allow two default namespace prefixes for any element. - Godmar
True; I've passed this on to our expert on XQuery Update. Thanks, Christian ___________________________ On Sun, Jul 11, 2010 at 11:34 PM, Godmar Back <godmar@gmail.com> wrote:
To follow up on my previous email, we observe the following. We run a query: declare namespace libx='http://libx.org/xml/libx2'; declare namespace atom='http://www.w3.org/2005/Atom'; declare variable $entry as xs:anyAtomicType external; declare variable $id as xs:string external; declare variable $feed as xs:anyAtomicType := doc('libx2_feed')/atom:feed; delete node $feed/atom:entry[atom:id=$id], (: INVESTIGATE: this results in wrong xmlns entries :) insert node $entry into $feed whereby we bind 'entry' to a DOM node using the org.w3c. Implementation. When we print the about-to-be-inserted entry from Java code prior to insertion, we see: <?xml version="1.0" encoding="UTF-8"?><entry xmlns="http://www.w3.org/2005/Atom" xmlns:libx="http://libx.org/xml/libx2"> <id>3</id> (rest omitted) If we run a database dump via the org.basex.BaseX program (query: doc('libx2_feed')) right after the update, we see this: <entry xmlns="" xmlns="http://www.w3.org/2005/Atom" xmlns:libx="http://libx.org/xml/libx2"> <id>3</id> <title> (rest omitted) Other than the spurious 'xmlns=""', the rest looks ok. It seems to me that BaseX should never allow two default namespace prefixes for any element. - Godmar
Has there been any progress on this issue? Independent of whether we are using the XQJ binding to access BaseX, it is nearly useless to us if it can't handle XML namespaces on operations as simple as replacing a node. Would we be running into this issue if we didn't use XQJ? Please give us some indication on how mature you think BaseX's support for XML namespaces is. Thanks. - Godmar On Sun, Jul 11, 2010 at 5:38 PM, Christian Grün <christian.gruen@gmail.com>wrote:
True; I've passed this on to our expert on XQuery Update.
Thanks, Christian ___________________________
On Sun, Jul 11, 2010 at 11:34 PM, Godmar Back <godmar@gmail.com> wrote:
To follow up on my previous email, we observe the following. We run a query: declare namespace libx='http://libx.org/xml/libx2'; declare namespace atom='http://www.w3.org/2005/Atom'; declare variable $entry as xs:anyAtomicType external; declare variable $id as xs:string external; declare variable $feed as xs:anyAtomicType :=
doc('libx2_feed')/atom:feed;
delete node $feed/atom:entry[atom:id=$id], (: INVESTIGATE: this results in wrong xmlns entries :) insert node $entry into $feed whereby we bind 'entry' to a DOM node using the org.w3c. Implementation. When we print the about-to-be-inserted entry from Java code prior to insertion, we see: <?xml version="1.0" encoding="UTF-8"?><entry xmlns="http://www.w3.org/2005/Atom" xmlns:libx=" http://libx.org/xml/libx2"> <id>3</id> (rest omitted) If we run a database dump via the org.basex.BaseX program (query: doc('libx2_feed')) right after the update, we see this: <entry xmlns="" xmlns="http://www.w3.org/2005/Atom" xmlns:libx="http://libx.org/xml/libx2"> <id>3</id> <title> (rest omitted) Other than the spurious 'xmlns=""', the rest looks ok. It seems to me that BaseX should never allow two default namespace prefixes for any element. - Godmar
Godmar,
Has there been any progress on this issue?
It has been put on our todo list.
Independent of whether we are using the XQJ binding to access BaseX, it is nearly useless to us if it can't handle XML namespaces on operations as simple as replacing a node.
Note that XQuery Update is still a new standard, and a comparison of the regular, and the Update XQuery test results will show you that there are hardly any implementations available out there that both offer XQUTS and reach 100% Update conformance: http://dev.w3.org/2007/xquery-update-10-test-suite/results/XQUTSReportSimple... http://dev.w3.org/2006/xquery-test-suite/PublicPagesStagingArea/XQTSReportSi... "100% conformance" means that the test suite has been correctly answered; it doesn't mean of course that an XQuery Update implementation is bug free. – This doesn't mean that your bug report is irrelevant; it just means it has to wait until other, more important todos have been resolved. Christian
On Wed, Jul 14, 2010 at 12:11 PM, Christian Grün <christian.gruen@gmail.com>wrote:
Godmar,
Has there been any progress on this issue?
It has been put on our todo list.
Independent of whether we are using the XQJ binding to access BaseX, it is nearly useless to us if it can't handle XML namespaces on operations as simple as replacing a node.
Note that XQuery Update is still a new standard, and a comparison of the regular, and the Update XQuery test results will show you that there are hardly any implementations available out there that both offer XQUTS and reach 100% Update conformance:
http://dev.w3.org/2007/xquery-update-10-test-suite/results/XQUTSReportSimple...
http://dev.w3.org/2006/xquery-test-suite/PublicPagesStagingArea/XQTSReportSi...
"100% conformance" means that the test suite has been correctly answered; it doesn't mean of course that an XQuery Update implementation is bug free. – This doesn't mean that your bug report is irrelevant; it just means it has to wait until other, more important todos have been resolved.
Let me ask it differently. Can BaseX be used to maintain XML documents that use multiple namespaces (for instance, a atom:feed documenting containing foreign XML)? By maintain, I mean whether it supports operations such as adding, changing, and removing XML fragments? We only need these very basic operations, but we *DO* have multiple namespaces in our documents. I'm not tied to XQuery, nor its Java binding, nor any specific way of interacting with the database. - Godmar
Let me ask it differently. Can BaseX be used to maintain XML documents that use multiple namespaces (for instance, a atom:feed documenting containing foreign XML)?
…sure – many people use it for this purpose. We have introduced a total limit of 256 different namespaces; that's the only limit you should encounter. Christian
On Wed, Jul 14, 2010 at 12:23 PM, Christian Grün <christian.gruen@gmail.com>wrote:
Let me ask it differently. Can BaseX be used to maintain XML documents that use multiple namespaces (for instance, a atom:feed documenting containing foreign XML)?
…sure – many people use it for this purpose. We have introduced a total limit of 256 different namespaces; that's the only limit you should encounter.
So, in order to implement the update problem originally mentioned in this thread, which language/API would I use in BaseX? - Godmar
So, in order to implement the update problem originally mentioned in this thread, which language/API would I use in BaseX? - Godmar
I'm not sure what you mean with "implement the update problem"; if you want to bypass the current bug, you might try to create a simple document without namespace (<root/>), and add new nodes as children. The language you're using, or API, doesn't matter (as mentioned before, you'll have most freedom with our own API). Christian
On Wed, Jul 14, 2010 at 12:56 PM, Christian Grün <christian.gruen@gmail.com>wrote:
So, in order to implement the update problem originally mentioned in this thread, which language/API would I use in BaseX? - Godmar
I'm not sure what you mean with "implement the update problem"; if you want to bypass the current bug, you might try to create a simple document without namespace (<root/>), and add new nodes as children. The language you're using, or API, doesn't matter (as mentioned before, you'll have most freedom with our own API).
I apologize for being slow here. Suppose I have a Atom feed XML document. I need to replace one entry. The entry contains XML from multiple namespaces. I tried using the XQuery below: declare namespace libx='http://libx.org/xml/libx2'; declare namespace atom='http://www.w3.org/2005/Atom'; declare variable $entry as xs:anyAtomicType external; declare variable $id as xs:string external; declare variable $feed as xs:anyAtomicType := doc('libx2_feed')/atom:feed; delete node $feed/atom:entry[atom:id=$id], *insert node $entry into $feed* * * I ran it via the XQJ binding, setting $entry to a DOM node implementation. This fails. What do you suggest I do instead? If I want this to work, should I not be using XQuery? Should I not be using a standard DOM node (using the built-in JDK org.w3c's implementation)? Should I not be using XQJ? I don't understand what creating a new document <root /> would do to address this problem. Also, it seems to me that replacing a part of a XML document (which is what I'm trying to do) should be a straightforward operation for an XML database. - Godmar - Godmar
[...] it seems to me that replacing a part of a XML document (which is what I'm trying to do) should be a straightforward operation for an XML database.
Sure it is. What have your tried so far? XQuery offers (actually too) many ways to do the same thing; you could opt for replacing the $entry variable with the XML fragment in question. Or you could use the replace expression, e.g. declare namespace atom = 'http://www.w3.org/2005/Atom'; let $old := doc('libx2_feed')/atom:feed/atom:entry[atom:id=$id] let $new := <node>to be added</node| replace node $old with $new Christian
Use replace makes no difference, the same problem persists. Also, I think a 'return' is needed for this to be a well-formed query. We're still unable to use BaseX to perform simple node replacement operations when the nodes use namespaces. - Godmar On Wed, Jul 14, 2010 at 5:55 PM, Christian Grün <christian.gruen@gmail.com>wrote:
[...] it seems to me that replacing a part of a XML document (which is what I'm trying to do) should be a straightforward operation for an XML database.
Sure it is. What have your tried so far? XQuery offers (actually too) many ways to do the same thing; you could opt for replacing the $entry variable with the XML fragment in question. Or you could use the replace expression, e.g.
declare namespace atom = 'http://www.w3.org/2005/Atom'; let $old := doc('libx2_feed')/atom:feed/atom:entry[atom:id=$id] let $new := <node>to be added</node| replace node $old with $new
Christian
Thanks for the feedback, Godmar, it looks as if we can't reproduce your problem this easily. Does it occur both with pure XQuery as well as XQJ? Could you provide us with an XML snippet which demonstrates the phenomena? Best, Christian On Thu, Jul 22, 2010 at 9:14 PM, Godmar Back <godmar@gmail.com> wrote:
Use replace makes no difference, the same problem persists.
Also, I think a 'return' is needed for this to be a well-formed query.
We're still unable to use BaseX to perform simple node replacement operations when the nodes use namespaces.
- Godmar
On Wed, Jul 14, 2010 at 5:55 PM, Christian Grün <christian.gruen@gmail.com> wrote:
[...] it seems to me that replacing a part of a XML document (which is what I'm trying to do) should be a straightforward operation for an XML database.
Sure it is. What have your tried so far? XQuery offers (actually too) many ways to do the same thing; you could opt for replacing the $entry variable with the XML fragment in question. Or you could use the replace expression, e.g.
declare namespace atom = 'http://www.w3.org/2005/Atom'; let $old := doc('libx2_feed')/atom:feed/atom:entry[atom:id=$id] let $new := <node>to be added</node| replace node $old with $new
Christian
Hey Godmar, I'm quite positive that I resolved your reported namespace issue and would be pleased if you could provide us with some feedback. Thanks for your help on improving BaseX. Regards, Lukas On Jul 14, 2010, at 9:33 PM, Godmar Back wrote:
On Wed, Jul 14, 2010 at 12:56 PM, Christian Grün <christian.gruen@gmail.com
wrote: So, in order to implement the update problem originally mentioned in this thread, which language/API would I use in BaseX? - Godmar
I'm not sure what you mean with "implement the update problem"; if you want to bypass the current bug, you might try to create a simple document without namespace (<root/>), and add new nodes as children. The language you're using, or API, doesn't matter (as mentioned before, you'll have most freedom with our own API).
I apologize for being slow here. Suppose I have a Atom feed XML document. I need to replace one entry. The entry contains XML from multiple namespaces. I tried using the XQuery below:
declare namespace libx='http://libx.org/xml/libx2'; declare namespace atom='http://www.w3.org/2005/Atom'; declare variable $entry as xs:anyAtomicType external; declare variable $id as xs:string external; declare variable $feed as xs:anyAtomicType := doc('libx2_feed')/ atom:feed; delete node $feed/atom:entry[atom:id=$id], insert node $entry into $feed
I ran it via the XQJ binding, setting $entry to a DOM node implementation.
This fails.
What do you suggest I do instead? If I want this to work, should I not be using XQuery? Should I not be using a standard DOM node (using the built-in JDK org.w3c's implementation)? Should I not be using XQJ?
I don't understand what creating a new document <root /> would do to address this problem. Also, it seems to me that replacing a part of a XML document (which is what I'm trying to do) should be a straightforward operation for an XML database.
- Godmar
- Godmar _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Here's a self-contained test case that reproduces the problem: http://top.cs.vt.edu/~gback/bx/b622/Test.java - Godmar
On Fri, Jul 23, 2010 at 12:59 AM, Godmar Back <godmar@gmail.com> wrote:
Here's a self-contained test case that reproduces the problem:
PS: as this test case shows, BaseX fails when doing the simple operations of creating a database, inserting a XML with two elements, and replacing one node in it. Christian mentioned that this may be due to the use of XQuery/XUpdate/XQJ and that it could be avoided by not using those interfaces. As I pointed out before, it is crucial for us to use DOM nodes that support the EventTarget interface internally. We would prefer to use a standard technology that doesn't tie us to BaseX to interact with the XML database, but if it doesn't exist, or doesn't work, we would be open to guidance on how we can get BaseX to perform the (simple) operations we need it for (e.g., inserting nodes, updating them, etc. etc.) - Godmar
Godmar, thanks for the test case. Indeed, the problem was not intrinsic to the implementation of XQuery Update, but occurred in the peripheral wrapper for DOM instances. Your bug might now be fixed with version 6.2.2, which I've just uploaded: http://www.inf.uni-konstanz.de/dbis/basex/maven/org/basex/basex/6.2.2/ As mentioned, our DOM interfaces haven't been tested very much, but your focus on DOM and XQJ might inspire other users to play around with this interface as well, and give us more feedback. Christian On Fri, Jul 23, 2010 at 7:59 PM, Godmar Back <godmar@gmail.com> wrote:
On Fri, Jul 23, 2010 at 12:59 AM, Godmar Back <godmar@gmail.com> wrote:
Here's a self-contained test case that reproduces the problem:
PS: as this test case shows, BaseX fails when doing the simple operations of creating a database, inserting a XML with two elements, and replacing one node in it.
Christian mentioned that this may be due to the use of XQuery/XUpdate/XQJ and that it could be avoided by not using those interfaces. As I pointed out before, it is crucial for us to use DOM nodes that support the EventTarget interface internally. We would prefer to use a standard technology that doesn't tie us to BaseX to interact with the XML database, but if it doesn't exist, or doesn't work, we would be open to guidance on how we can get BaseX to perform the (simple) operations we need it for (e.g., inserting nodes, updating them, etc. etc.)
- Godmar _______________________________________________ BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
On Fri, Jul 23, 2010 at 2:46 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Godmar,
thanks for the test case. Indeed, the problem was not intrinsic to the implementation of XQuery Update, but occurred in the peripheral wrapper for DOM instances. Your bug might now be fixed with version 6.2.2, which I've just uploaded:
http://www.inf.uni-konstanz.de/dbis/basex/maven/org/basex/basex/6.2.2/
This version fixes the test case, but doesn't fix the problem in our actual application, where it persists. I'll try to create a new test case. I'm still curious what XML interface I'm missing. - Godmar
On Fri, Jul 23, 2010 at 4:51 PM, Godmar Back <godmar@gmail.com> wrote:
On Fri, Jul 23, 2010 at 2:46 PM, Christian Grün <christian.gruen@gmail.com> wrote:
Godmar,
thanks for the test case. Indeed, the problem was not intrinsic to the implementation of XQuery Update, but occurred in the peripheral wrapper for DOM instances. Your bug might now be fixed with version 6.2.2, which I've just uploaded:
http://www.inf.uni-konstanz.de/dbis/basex/maven/org/basex/basex/6.2.2/
This version fixes the test case, but doesn't fix the problem in our actual application, where it persists. I'll try to create a new test case.
Here's a new test case that still breaks: http://top.cs.vt.edu/~gback/bx/b622/Test.java This is the change I made: http://top.cs.vt.edu/~gback/bx/b622/old-vs-new.diff In exchange for helping you debug your XQJ implementation, could you help me understand my options when working with BaseX? Specifically, let's say I'll switch to the client/server interface where I send XQuery and XQuery Update expressions and receive the result as textual XML. In this case, what do you recommend I do with respect to engineering my queries? I'm thinking of turning my existing XQJ queries into templates, removing all 'external' variables, then performing textual replacement, perhaps via String.printf(). This should be easily doable, though by definition it will not be as robust as XQJ. For instance, I need to sanity check strings input by the user to not fall victim to injection attacks, need to take care of proper escaping, etc. etc. Before I embark on this, I'm wondering if such a templating system already exists, or if I would essentially be reimplementing XQJ, or what other options exist. - Godmar
participants (3)
-
Christian Grün -
Godmar Back -
Lukas Kircher