BaseX-Talk May 2013

basex-talk@mailman.uni-konstanz.de

43 participants
61 discussions

Referential Queries
by James Wright 14 May '13

14 May '13

Hello Again, If this is the wrong forum for these type of questions let me know. By the way Liam I picked up your book last night, I like the flavor as it differs from my other reads such as those from Kay. Although I have been using XML for years and understand the core concepts it should be a great refresher. If you have time to read this and respond I appreciate it... If not I understand. :) The Organizational Overall Problem: There aren't many people in my industry that use XQuery and xml in the way it was intended (IMHO). In fact most developers in my organization are rather uneducated in it and as you know there is some un-rational backlash as many correlate XML to the DOM and XPath/XSLT 1.0 and as a competitor to JSON which is ludicrous. The DOM has its issues of scale-ability which our products are currently running into. This isn't really xml's or the DOM's problem but simply poor implementation. As you know though, all that matters is perception. I am having to work with a large number of un-schemad, basically hack job, developed xml documents and workflows. A lot of our product utilizes XSLT for reporting and transformation however only a few in the team understand the concepts and due to MS/Managerial BS we are stuck with a .NET XSLT 1.0 processor. Also almost all of our XSLT scripts utilize a Pull pattern which I find overly verbose and inefficient, but that is a personal opinion. I have been researching alternatives to the DOM and .NET's standard processor (I have used Saxon too) because I personally find XML useful and the query semantics of XQuery 3.0 awesome. Proposed Solutions: We have two primary use cases: 1) as a local db to replace the context DOM for our 'documents' which in our case relates to Utilitiy GIS Designs of circuit, subdivisions, fiber, etc... I am thinking BaseX coupled with RestXQ could replace our DOM for local installs and allow ourselves to decouple from the Geodatabase and provide a browser based UI. (Currently our xml documents are stored in a blob in the GIS database) The GIS we are using is ESRI however we are interested in also supporting Open Street Maps. (Noticed the Geo Module) 2) as a service for hosting uploading and allowing users/delivery and support to view, query and modify complex sets of interrelated XML configuration files. Some of our applications have hundreds. Again all these documents follow a similar semantics however their is no defined schema for any of them. Currently the client has to read a 50 page manual and edit the files one at time. Often there are nodes in several files which must match exactly or the entire application fails.... Its a nightmare and there is no concept of generalization in my organization when it comes to development. Every tool we do have to 'configure' is hand crafted and unique..... Its abysmal! Because of this we have tools that cover probably only 20% of the configuration. I think we can accomplish both of the above tasks using a single codebase and restXQ I have written an XQuery expression which using our 'common' xml semantics can ascertain entities/properties/relationships and distill this in the form of metadata which then using RestXQ is distilled into a metadata driven api for manipulated data centric xml documents. Similar to how you can transform a well formed xml document into an xsd however our metadata format compliments XSD but servers broader purposes for example to annotate triggers/mappings etc. This metadata is then consumed for each RestXQ operation to allow for a generalized API and all markup is applied on the Client to allow for more efficient caching mechanisms since users can share resources between different representations more efficiently. We currently use Knockout.js on the client for this. Here is an example of a generalized API endpoint for retrieving a unique entity by type. (: Removed error handling and some other stuff for brevity. Its not fully functional/vindicated but is more a representation of the concept :)declare %restxq:path("api/{$entityListName}/{$entityId}") %restxq:GET function page:GetEntity($entityListName as xs:string, $entityId as xs:string) { let $entityMetadata := $page:database/metadata/entity[@type = $entityListName] let $entity := if($entityMetadata/property[@key = 'true']) then (: Use the properties marked in the metadata as the key :) $page:database//*[name() = $entityListName and data((*|@*)[name() = $entityMetadata/property[@key = 'true']/@name]) = $entityId] <entity> { (: Construct the representation based on the metadata :) for $propMeta in $entityMetadata/property return attribute {$propMeta/@name} { data($entity/(*|@*)[name() = $propMeta/@name]) } } { (: Include a link to all related entities for further discovery by the client.. :) for $relMeta in $entityMetadata/relationship return if($relMeta/@multiplicity = '*') then (: creates a href like: api/worklocation/23/cost which would return the sequence of cost items associated with the work location 23 :) element link { (attribute href { concat( 'api/, $entityListName, '/', $entityId, '/', $relMeta/@with ), attribute rel { $relMeta/@with })} else ( (: creates a href to a single related entity :) let $relatedEntityMeta := $page:database/metadata/entity[@type = $relMeta/@with'] element link { (attribute href { concat( 'api/', $relMeta/@with, data( $entity/(@*|*)[name() = $relMeta/@type]/((@*|*)[$relatedEntityMeta/property[@key = 'true'] ) ) }, attribute rel { $relMeta/@with } ) } ) } </entity> }; With a metadata mapping of the document we can convert the 'tree' structure into a virtual 'relational table' structure allowing for granular node modification/addition/removal in a generalized way. Once the metadata is distilled using the same api with metadata describing the metadata we can then allow the client to further manipulate the view/aliasing/permissions/triggers/mappings of the entities archetypes metadata allowing for quick UI/Workflow generation. At least that is the idea... This obviously requires a client side library which understand this metadata orchestration. The XML/BaseX Question:We need to be able to query effectively across relationships. Are there any facilities in XML/XQuery/3rd party that do this? I was hoping Id and IDref could accomplish this but as you stated that is not the case.... Basically I won't know before hand whether a child node is inline or a reference but would like to be able to query both as those they were the same.. If I had two nodes (I will call them parents) both with the same child. One has the actual inline child node and the other has just a reference to it. Lets say the child's attribute name is 'Tom'. I would like to be able to query like this and return both parents nodes: //parents[child/@name = 'Tom'] Anything? If not, is this a weird use case? I wouldn't think so. I could imagine however that this use case may be hard to 'generally' support in xml given referential loops etc especially in a non schemed/validated document... - James

2 2

Namespaces on element construction
by Arto Viitanen 14 May '13

14 May '13

Why does this query (on BaseX 7.5, basexgui) let $ns := namespace {''} {'the-namespace'} let $c1 := <c xmlns="the-namespace">text</c> let $c2 := <c>{ $ns,'text'}</c> let $c3 := element {'c'} {$ns,'text'} let $qn := QName($ns, 'c') let $c4 := element {$qn} {'text'} let $nl := "
" return ($c1,$nl, $c2,$nl, $c3,$nl, $c4,$nl, '|' || namespace-uri-from-QName(node-name($c1)) || '|',$nl, '|' || namespace-uri-from-QName(node-name($c2)) || '|',$nl, '|' || namespace-uri-from-QName(node-name($c3)) || '|',$nl, '|' || namespace-uri-from-QName(node-name($c4)) || '|',$nl) outputs <c xmlns="the-namespace">text</c> <c xmlns="the-namespace">text</c> <c xmlns="the-namespace">text</c> <c xmlns="the-namespace">text</c> |the-namespace| || || |the-namespace| I.e, the namespace seems to exist on all four cases when outputted, by node-name finds it only if the namespace is given in element as text, or as QName value in computed element. -- Arto Viitanen Microteam Oy Finland

2 2

BasexGUI (minor) issue
by Rob Stapper 14 May '13

14 May '13

Hi guys, In the BaseXGUI, when expanded the windowframe with F11, clicking on the "opens existing file"-button, at the right-hand top of the editor-window, stops the GUI. Ectually the GUI is waiting for input of a filename but the inputbox is not visible. When this happens you have to abort the GUI with the windows taskmanager (in windows antway). Have fun, Rob Stapper

3 4

full text with search term in a variable gives no matches
by Liam R E Quin 14 May '13

14 May '13

The following query gives me no results: for $city as xs:string in ("Paris", "Cambridge", "London", "Oxford") return ( $city, count(/dictionary/letter/entry[.//p//text() contains text {$city}]), "
" ) However, BaseX rewrites it to for $city as xs:string in ("Paris", "Cambridge", "London", "Oxford", ...) return ($city, fn:count(db:fulltext("with-sources", { $city })/ancestor::*:p/ancestor::*:entry), " ") If I remove the braces around $city in the argument to db:fulltext() I get the right answers (or at least plausible answers). So this looks like a bug in the rewriting engine. Liam -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org freenode/#xml

2 3

Proposal for programmatically setting index options
by Andy Bunce 14 May '13

14 May '13

Hi, The index settings for a database can make a huge difference to query performance. I have been playing with a database where the time for a query using distinct-values went from 4secs to 4ms by increasing the MAXCATS value. Currently it is rather fiddly to change these values, particularly to this programmatically. with different values per database. I would find it useful there was an easier way to do this. A quick suggestion as to how this might be done: db:optimize could be extended to taker a third argument that matched the format of in indexes portion of the db:info command <indexes> <uptodate>true</uptodate> <textindex>ON</textindex> <attributeindex>ON</attributeindex> <fulltextindex>OFF</fulltextindex> <updindex>OFF</updindex> <maxcats>100</maxcats> <maxlen>96</maxlen> </indexes> e.g db:optimize("mydb",fn:true(),$indexopts) This would rebuild "mydb" using the settings specified in $indexopts. The extra argument could also be added to db:create. /Andy

2 1

encountered baseX error/potential bug
by Rob Stapper 14 May '13

14 May '13

Hi guys, Probably no news but just to be sure: - The use of the 'http://www.basex.org/serialize'-package gives problems in Basex 7.7 beta. See message below success, Rob Stapper MESSAGE: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.7 beta Java: Oracle Corporation, 1.7.0_21 OS: Windows Vista, x86 Stack Trace: java.lang.RuntimeException: $2 as item()* org.basex.util.Util.notexpected(Util.java:53) org.basex.query.QueryStack.pos(QueryStack.java:67) org.basex.query.QueryStack.set(QueryStack.java:90) org.basex.query.QueryContext.set(QueryContext.java:651) org.basex.query.value.item.FuncItem.bindVars(FuncItem.java:122) org.basex.query.value.item.FuncItem.invValue(FuncItem.java:136) org.basex.query.func.FNFunc$1.next(FNFunc.java:100) org.basex.query.iter.Iter.value(Iter.java:64) org.basex.query.expr.ParseExpr.value(ParseExpr.java:73) org.basex.query.QueryContext.value(QueryContext.java:322) org.basex.query.gflwor.Let$1.next(Let.java:75) org.basex.query.gflwor.GFLWOR$2.next(GFLWOR.java:74) org.basex.query.expr.ParseExpr.item(ParseExpr.java:53) org.basex.query.func.StaticFunc.invItem(StaticFunc.java:172) org.basex.query.func.BaseFuncCall.item(BaseFuncCall.java:38) org.basex.query.func.FNStr.concat(FNStr.java:303) org.basex.query.func.FNStr.item(FNStr.java:99) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) org.basex.query.func.FNStr.iter(FNStr.java:44) org.basex.query.gflwor.GFLWOR$2.next(GFLWOR.java:78) org.basex.query.func.FNStr.strjoin(FNStr.java:264) org.basex.query.func.FNStr.item(FNStr.java:81) org.basex.query.func.FNStr.concat(FNStr.java:303) org.basex.query.func.FNStr.item(FNStr.java:99) org.basex.query.expr.ParseExpr.value(ParseExpr.java:70) org.basex.query.func.FNStr.value(FNStr.java:57) org.basex.query.QueryContext.value(QueryContext.java:322) org.basex.query.func.StaticFunc.invValue(StaticFunc.java:193) org.basex.query.func.BaseFuncCall.value(BaseFuncCall.java:62) org.basex.query.func.StaticFuncCall.iter(StaticFuncCall.java:85) org.basex.query.MainModule.iter(MainModule.java:69) org.basex.query.QueryContext.iter(QueryContext.java:265) org.basex.query.QueryContext.execute(QueryContext.java:524) org.basex.query.QueryProcessor.execute(QueryProcessor.java:95) org.basex.core.cmd.AQuery.query(AQuery.java:76) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.gui.GUI.exec(GUI.java:397) org.basex.gui.GUI$7.run(GUI.java:349)

2 2

Question regarding BaseX support for id and idref
by James Wright 14 May '13

14 May '13

Hello BaseX-Talk, I am new to the boards and BaseX in general. I have been looking for a open source NoSQL database with expressive query abilities and I think BaseX fits the bill. Its been an exciting journey as I am new to the whole X platform aside from the angle brackets... As I think many are. I am currently exploring the capabilities of RestXQ and the xml stack. I especially like its potential coupled with knockout.js, and a truly flexible RESTful metadata/hypermedia driven api. I have implemented an api of this nature in .Net however the metadata and querying capabilities are hindered and unnecessarily complex where I feel xml and XQuery make this dynamic orchestration quite gracefully. Unfortunately I am stuck now that I am exploring referential relationships and xml. I am trying to understand the support BaseX and XML in general has for referential nodes especially when dealing with queries. For example I know the xml standard defines id and idref however I have not been able to find any documentation on these in BaseX or XML in general. For example does BaseX handle idiosyncrasies of id and idref or must I handle these in my queries? For example: let $context := <item id="10"> <child idref="2" /><item><item id="20"> <child idref="2" /> <child idref="1" /></item><item id="30" /> <child id="1" name="firstChild" /><child id="2" name="secondChild" /> for $itemWithFirstChild in $context/item[child/@name = 'firstChild']return $itemWithFirstChild/@id I would like this to return "10 20"Now this example uses a dynamic context node however in the application these would exist as nodes in the database... If it does fully support this how do I enable it or get it to work? Also what about XLink? Any help would be greatly appreciated. Thanks again for your time.- James

2 4

Chat with lonely women in your area and hook up.
by root＠readymadetopps.com 12 May '13

12 May '13

Chat with lonely women in your area and hook up. http://redirect.surfsportpages.info 72749 alftj

1 0

now I busy reporting bugs, here another (minor) one
by Rob Stapper 11 May '13

11 May '13

Hi, I found another minor bug. In case of an error in an imported package-file ( index.xqm) while running the mainprogram the package-file cannot be opened from the error-message at the bottom of the editor-window. BaseX returns an message: 'file could not be opened'. Opening the package-file ( index.xqm) from within the query-info window works fine. It's a minor issue but still nice if it is solved. I use BaseX version 7.7 beta If you need more info about the issue, let me know. Rob Stapper

2 2

http-client issue
by Marco Lettere 10 May '13

10 May '13

Hello all, I've stumbled upon an issue related maybe to my scarce knowledge of HTTP. I've setup a Jetty based "jax-ws" server application which I call from basex through the "http:send-request" function. Everything works neatly until the response document (plus the SOAP overhead) violates the 8192 bytes threshold in size. At that point the Xquery containing the request stops working. Investigating I've found out that the server starts sending back the data encoded as base64 instead of plain XML thus causing the failure of the following xquery expressions ( err:XPTY0019:Steps within a path expression must yield nodes; xs:base64Binary found. ) Sniffing around on my HTTP connection I've seen that the server automatically alters the HTTP response at the size of 8192 bytes sending the header "Transfer-Encoding: Chunked" instead of the usual two ("Content-Type:test/xml" and "Content-Length"). This problem doesn't occur if I use Firefox Poster or socat (yet to try other neutral tools). Is it because they're able to handle this aspect transparently? What is the best solution to this? Is there a way (in the http request) to ask the server to avoid this change in behaviour? Do I need to handle this aspect at xquery application level? I'd be rather happy about any suggestions. Thank you in advance. M.

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk May 2013