- BaseX-Talk - mailman.uni-konstanz.de

Issue with client:connect() on servlet
by France Baril 10 May '13

10 May '13

Hi, I have created a connect function (restxq) that: - Works perfectly well on my Mac. - Returns a *BXCL0001 *when used on the Linux production server. The function uses: - *client:connect*(*$app:feedback-server-host*, 1984, *$session-user*, * $session-password*) The only reason I can find is that the production server is deployed through a servlet. Could this affect the behaviour? - The admin:users() function does list the user we are trying to use in 'connect', so we are assuming that the server is connected and that the user does exist. - We are using the correct password. - We have tried setting up the value of the SERVERHOST option in .basex to make sure it matches $app:feedback-server-host, although that didn't seem necessary for the mac setup. - 1984 is the configured PORT. - Could the fact that the production server is using TomCat have an impact on the client:connect function? We are out of things to try. Any clue would be helpful. Thanks, France -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com (514) 572-0341

3 7

SET command?
by Pascal Heus 08 May '13

08 May '13

Having a weird behavior with the SET command as I can'r seem to change any option. Just launching from the shell using the "basex" script, I can GET but not SET. I must be doing something obviously wrong.... This is with 7.6 on MacOSX. BaseX 7.6 [Standalone] Try help to get more information. > GET DBPATH DBPATH: /Applications/basex/data > SET DBPATH /Users/Pascal/BaseXData Unknown option 'DBPATH'. > GET LANG LANG: English > SET LANG French Unknown option 'LANG'. > Also tried: basex -c "SET DBPATH /Users/Pascal/BaseXData" but same results.

3 3

inexplicable query parsing issue; "could not execute XQUERY..."
by Graydon Saunders 08 May '13

08 May '13

Running BaseX77-20130503.190321 The horrible query works until I add the second ("local:addArea") function to it; that works fine in its own smaller query, which I've attached. (as far as I can tell, it really is the function causing the problem.) I've attached the horrible query as well, just in case it's not all in the error message. The GUI is showing me "OK" in the bottom left corner. Contents of error popup: Could not execute XQUERY declare function local:addArea($area as xs:string,$uri as xs:string,$path as xs:string,$element as element()) { element {node-name($element)} { $element/@*, attribute {'area'} {$area}, attribute {'uri'} {$uri}, attribute {'path'} {$path}, $element/node() } }; declare function local:reportError($type as xs:string,$file as xs:anyURI,$match as item(),$addInfo as item()*) as element(error) { let $errorMsg := <messages> <msg><type>BASE0000</type><words>Document loaded into collection is not a WKNA-Shared-CMS.dtd document; document element is not wkna-shared-cms</words></msg> <msg><type>CITE0000</type><words>Cite attribute values are required to contain non-whitespace characters</words></msg> <msg><type>CITE0005</type><words>Cite attribute value contains characters not permitted in cite attribute values</words></msg> <msg><type>CITE0010</type><words>num element must have at least one of an associated cite attribute with a non-whitespace string value or non-whitespace string-contents</words></msg> <msg><type>CITE0020</type><words>num element with associated type attribute with value "heading" has been deprecated and is not permitted</words></msg> <msg><type>CITE0030</type><words>duplicate locative citations</words></msg> <msg><type>FILE0010</type><words>document-defining element area-cite pair does not match basename component of pathname</words></msg> <msg><type>DATE0010</type><words>date element has no associated status attribute or status attribute value</words></msg> <msg><type>DATE0020</type><words>date element has an associated date attribute with a null value</words></msg> <msg><type>DATE0030</type><words>date element has an associated date attribute with a value outside the permissible date range</words></msg> <msg><type>DATE0040</type><words>date element has an associated status attribute value not on the permitted list of status values</words></msg> <msg><type>DATE0050</type><words>date element has no associated date attribute while having a status value which does not permit the date attribute to be absent</words></msg> <msg><type>AREA0000</type><words>area attribute with no value</words></msg> <msg><type>AREA0001</type><words>area attribute value does not conform to NCName type restrictions</words></msg> <msg><type>AREA0010</type><words>area attribute associated with an assembly element has a value which is not a defined area</words></msg> <msg><type>AREA0020</type><words>area attribute associated with a horizon-level element has a value which is not a defined area</words></msg> <msg><type>AREA0030</type><words>area attribute NOT associated with a horizon-level element</words></msg> <msg><type>AREA0040</type><words>area attribute NOT associated with a feature horizon-level element</words></msg> <msg><type>AREA0045</type><words>area attribute associated with a feature horizon-level element has a value which is not a defined area</words></msg> <msg><type>AREA0050</type><words>area attribute associated with an include element has a value which is not a defined area</words></msg> <msg><type>AREA0051</type><words>include element with special-case associated area attribute value "fragments" has an associated cite attribute with a value OTHER THAN "empty" or "omitted" which is not permitted</words></msg> <msg><type>AREA0060</type><words>area attribute associated with a source element has a value which is not a defined area</words></msg> <msg><type>AREA0070</type><words>area attribute associated with a related-content-ref element has a value which is not a defined area</words></msg> <msg><type>AREA0080</type><words>area attribute associated with a link-type element has a value which is not a defined area</words></msg> <msg><type>REFR0010</type><words>include elements must reference an existing locative citation</words></msg> <msg><type>REFR0020</type><words>source elements must reference a locative citation (area-cite pair) which exists in the same document</words></msg> <msg><type>REFR0030</type><words>related-content-ref elements must reference a locative citation (area-cite pair) which exists in the same document</words></msg> <msg><type>LINK0010</type><words>link elements must have a non-whitespace string value</words></msg> </messages> return <error type="{xs:NCName($type)}"> <uri>{$file}</uri> <description>{string($errorMsg/msg[type eq $type]/words)} {if (normalize-space($addInfo) or $addInfo instance of node()) then $addInfo else ()}</description> <path>{replace(path($match),'/Q\{[^\}]*\}','/')}</path> {if ($match[self::element() and (self::assembly or self::va-object or self::provision or self::agency-doc or self::commentary or self::decision or self::fragment or self::index)]) then element {name($match)} {$match/@area} else $match} </error> }; (: definitions for cite and cite-related checks:) let $citeChars := 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-.(),'/[]%:|⦅ÀÂÇÉÈÊËÎÏÔÛÙÜŸÑàâçéèêëîïôûùüÿñœ ' (: definitions for date checks :) let $dateStart := xs:date('1900-01-01') let $dateEnd := current-date() + xs:yearMonthDuration('P5Y0M') let $statusList := ('active', 'amended', 'archived', 'cancelled', 'consolidated', 'effective', 'first-reading', 'former', 'heard', 'oral', 'published', 'released', 'repealed', 'replaced', 'resolved', 'reviewed', 'revised', 'revoked', 'royal-assent', 'written') let $absentDateStatus := ('amended','archived','former','repealed','revoked') (: definitions for area checks :) let $areasList := doc(concat('file:',string-join(tokenize(substring-after(static-base-uri(),':'),'/')[not(position() = (last(),last() -1))],'/'),'/XSLT/DTDs/areas-metadata.xml'))/wkna-shared-cms/area/@area (: definitions for reference checks :) let $locativeNums := for $x in collection()//num[ancestor::*[@area][@area castable as xs:NCName]][@cite][matches(@cite,'\P{Zs}')][translate(@cite,$citeChars,'') eq ''] return local:addArea($x/ancestor::*[@area][@area castable as xs:NCName][1]/@area/string(),base-uri($x),replace(path($x),'/Q\{[^\}]*\}','/'),$x) let $sortedNums := for $x in $locativeNums order by $x/@area,$x/@cite return $x (: perform actual checks:) return file:write('/home/graydon/checks.xml', <bucket> { for tumbling window $w in $sortedNums start $s previous $pitem when concat($s/@area,'|',$s/@cite) ne concat($pitem/@area,'|',$pitem/@cite) return if ($w[2]) then local:reportError('CITE0030','',$w,'') else () } { for $file in collection() let $path := document-uri($file) return if (not($file/wkna-shared-cms)) then local:reportError('BASE0000',$path,'','') else let $bad := ( (:check area + primary cite matches file name :) for $x in $file/wkna-shared-cms/*[self::assembly or self::provision or self::decision or self::agency-doc or self::commentary or self::fragment or self::index][1] return if ($x!concat(@area,'+',replace(replace(translate(translate(num[@cite][1]/@cite,'ÀÂÇÉÈÊËÎÏÔÛÙÜŸÑàâçéèêëîïôûùüÿñ','AACEEEEIIOUUUYNaaceeeeiiouuuyn'),' /[]%:','______'),'_+','_'),'^_+|_+$',''),'.xml') eq tokenize($path,'/')[last()]) then () else local:reportError('FILE0010',$path,$x,$x/num[@cite][1]), (: checks on cite values :) for $x in $file//@cite[not(matches(.,'\P{Zs}'))] return local:reportError('CITE0000',$path,$x/..,''), for $x in $file//@cite[matches(.,'\P{Zs}')][translate(.,$citeChars,'') ne ''] return local:reportError('CITE0005',$path,$x/..,concat(' The characters ',translate($x,$citeChars,''),' are not permitted in cite attribute values.')), (: checks on num elements:) for $x in $file//num[not(normalize-space(@cite) or matches(.,'\P{Zs}'))] return local:reportError('CITE0010',$path,$x,''), for $x in $file//num[@type eq 'heading'] return local:reportError('CITE0020',$path,$x,''), (: checks on date elements :) for $x in $file//date[not(@status) or not(normalize-space(@status))] return local:reportError('DATE0010',$path,$x,''), for $x in $file//date[@date][not(normalize-space(@date))] return local:reportError('DATE0020',$path,$x,''), for $x in $file//date[@date castable as xs:date or @date castable as xs:gYear or @date castable as xs:gYearMonth] let $compare := if ($x/@date castable as xs:date) then xs:date($x/@date) else if ($x/@date castable as xs:gYearMonth) then xs:date(concat(xs:string(xs:gYearMonth($x/@date)),'-01')) else xs:date(concat(xs:string(xs:gYear($x/@date)),'-01-01')) return if ($compare lt $dateStart or $compare gt $dateEnd) then local:reportError('DATE0030',$path,$x,concat('Permissible dates lie betwee ',$dateStart,' and ',$dateEnd,' inclusive.')) else (), for $x in $file//date[@status][normalize-space(@status)][not(@status = $statusList)] return local:reportError('DATE0040',$path,$x,''), for $x in $file//date[@status][normalize-space(@status)][@status = $statusList][not(@status = $absentDateStatus)][not(@date)] return local:reportError('DATE0050',$path,$x,''), (: checks on link elements :) for $x in $file//link[not(matches(.,'\P{Zs}'))] return local:reportError('LINK0010',$path,$x,''), (: checks on area attributes:) for $x in $file//@area[not(normalize-space())] return local:reportError('AREA0000',$path,$x/..,''), for $x in $file//@area[not(. castable as xs:NCName)] return local:reportError('AREA0001',$path,$x/..,''), for $x in $file//assembly[@area][normalize-space(@area)][@area castable as xs:NCName]/@area[not(. = $areasList)] return local:reportError('AREA0010',$path,$x/..,''), for $x in $file//*[self::provision or self::commentary or self::decision or self::agency-doc or self::fragment][@area][normalize-space(@area)][@area castable as xs:NCName][@area[not(. = $areasList)]] return local:reportError('AREA0020',$path,$x,''), for $x in $file//*[self::provision or self::commentary or self::decision or self::agency-doc][not(@area) or not(normalize-space(@area))] return local:reportError('AREA0030',$path,$x,''), for $x in $file//*[self::va-object][not(@area) or not(normalize-space(@area))] return local:reportError('AREA0040',$path,$x,''), for $x in $file//*[self::va-object][@area][normalize-space(@area)][@area castable as xs:NCName][not(@area = $areasList)] return local:reportError('AREA0045',$path,$x,''), for $x in $file//include[@area][normalize-space(@area)][@area castable as xs:NCName][not(@area = $areasList)] return local:reportError('AREA0050',$path,$x,''), for $x in $file//include[@area][normalize-space(@area)][@area castable as xs:NCName][@area = 'fragments'][not(@cite = ('empty','omitted'))] return local:reportError('AREA0051',$path,$x,''), for $x in $file//source[@area][normalize-space(@area)][@area castable as xs:NCName][not(@area = $areasList)] return local:reportError('AREA0060',$path,$x,''), for $x in $file//related-content-ref[@area][normalize-space(@area)][@area castable as xs:NCName][not(@area = $areasList)] return local:reportError('AREA0070',$path,$x,''), for $x in $file//*[@area][normalize-space(@area)][@area castable as xs:NCName][self::link or self::target][not(@area = $areasList)] return local:reportError('AREA0080',$path,$x,''), (:check references:) for $x in $file//include[not(map:contains($allLocations,concat(@area,'|',@cite)))] return local:reportError('REFR0010',$path,$x,''), for $x in $file//source[not(some $num in $file//num satisfies $num/@cite eq @cite and $num/ancestor::*[@area][1]/@area eq @area)] return local:reportError('REFR0020',$path,$x,''), for $x in $file//related-content-ref[not(some $num in $file//num satisfies $num/@cite eq @cite and $num/ancestor::*[@area][1]/@area eq @area)] return local:reportError('REFR0030',$path,$x,'') ) return if ($bad/*) then $bad else () }</bucket>): Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.7 beta Java: Oracle Corporation, 1.7.0_09 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException org.basex.query.var.StaticVarRef.uses(StaticVarRef.java:61) org.basex.query.expr.Arr.uses(Arr.java:45) org.basex.query.func.FNMap.uses(FNMap.java:137) org.basex.query.expr.Arr.uses(Arr.java:45) org.basex.query.path.Step.get(Step.java:55) org.basex.query.QueryParser.axisStep(QueryParser.java:1950) org.basex.query.QueryParser.step(QueryParser.java:1896) org.basex.query.QueryParser.relativePath(QueryParser.java:1845) org.basex.query.QueryParser.path(QueryParser.java:1819) org.basex.query.QueryParser.value(QueryParser.java:1699) org.basex.query.QueryParser.unary(QueryParser.java:1686) org.basex.query.QueryParser.cast(QueryParser.java:1664) org.basex.query.QueryParser.castable(QueryParser.java:1652) org.basex.query.QueryParser.treat(QueryParser.java:1640) org.basex.query.QueryParser.instanceoff(QueryParser.java:1628) org.basex.query.QueryParser.intersect(QueryParser.java:1607) org.basex.query.QueryParser.union(QueryParser.java:1581) org.basex.query.QueryParser.multiplicative(QueryParser.java:1565) org.basex.query.QueryParser.additive(QueryParser.java:1550) org.basex.query.QueryParser.range(QueryParser.java:1539) org.basex.query.QueryParser.stringConcat(QueryParser.java:1525) org.basex.query.QueryParser.ftContains(QueryParser.java:1499) org.basex.query.QueryParser.comparison(QueryParser.java:1481) org.basex.query.QueryParser.and(QueryParser.java:1467) org.basex.query.QueryParser.or(QueryParser.java:1453) org.basex.query.QueryParser.single(QueryParser.java:1032) org.basex.query.QueryParser.forClause(QueryParser.java:1179) org.basex.query.QueryParser.initialClause(QueryParser.java:1149) org.basex.query.QueryParser.flwor(QueryParser.java:1048) org.basex.query.QueryParser.single(QueryParser.java:1021) org.basex.query.QueryParser.expr(QueryParser.java:1010) org.basex.query.QueryParser.parenthesized(QueryParser.java:2291) org.basex.query.QueryParser.primary(QueryParser.java:2054) org.basex.query.QueryParser.postfix(QueryParser.java:2014) org.basex.query.QueryParser.step(QueryParser.java:1895) org.basex.query.QueryParser.path(QueryParser.java:1811) org.basex.query.QueryParser.value(QueryParser.java:1699) org.basex.query.QueryParser.unary(QueryParser.java:1686) org.basex.query.QueryParser.cast(QueryParser.java:1664) org.basex.query.QueryParser.castable(QueryParser.java:1652) org.basex.query.QueryParser.treat(QueryParser.java:1640) org.basex.query.QueryParser.instanceoff(QueryParser.java:1628) org.basex.query.QueryParser.intersect(QueryParser.java:1607) org.basex.query.QueryParser.union(QueryParser.java:1581) org.basex.query.QueryParser.multiplicative(QueryParser.java:1565) org.basex.query.QueryParser.additive(QueryParser.java:1550) org.basex.query.QueryParser.range(QueryParser.java:1539) org.basex.query.QueryParser.stringConcat(QueryParser.java:1525) org.basex.query.QueryParser.ftContains(QueryParser.java:1499) org.basex.query.QueryParser.comparison(QueryParser.java:1481) org.basex.query.QueryParser.and(QueryParser.java:1467) org.basex.query.QueryParser.or(QueryParser.java:1453) org.basex.query.QueryParser.single(QueryParser.java:1032) org.basex.query.QueryParser.letClause(QueryParser.java:1205) org.basex.query.QueryParser.initialClause(QueryParser.java:1148) org.basex.query.QueryParser.flwor(QueryParser.java:1048) org.basex.query.QueryParser.single(QueryParser.java:1021) org.basex.query.QueryParser.iff(QueryParser.java:1443) org.basex.query.QueryParser.single(QueryParser.java:1025) org.basex.query.QueryParser.flwor(QueryParser.java:1130) org.basex.query.QueryParser.single(QueryParser.java:1021) org.basex.query.QueryParser.expr(QueryParser.java:1001) org.basex.query.QueryParser.enclosed(QueryParser.java:990) org.basex.query.QueryParser.dirElemContent(QueryParser.java:2541) org.basex.query.QueryParser.dirElement(QueryParser.java:2487) org.basex.query.QueryParser.constructor(QueryParser.java:2371) org.basex.query.QueryParser.primary(QueryParser.java:2056) org.basex.query.QueryParser.postfix(QueryParser.java:2014) org.basex.query.QueryParser.step(QueryParser.java:1895) org.basex.query.QueryParser.path(QueryParser.java:1811) org.basex.query.QueryParser.value(QueryParser.java:1699) org.basex.query.QueryParser.unary(QueryParser.java:1686) org.basex.query.QueryParser.cast(QueryParser.java:1664) org.basex.query.QueryParser.castable(QueryParser.java:1652) org.basex.query.QueryParser.treat(QueryParser.java:1640) org.basex.query.QueryParser.instanceoff(QueryParser.java:1628) org.basex.query.QueryParser.intersect(QueryParser.java:1607) org.basex.query.QueryParser.union(QueryParser.java:1581) org.basex.query.QueryParser.multiplicative(QueryParser.java:1565) org.basex.query.QueryParser.additive(QueryParser.java:1550) org.basex.query.QueryParser.range(QueryParser.java:1539) org.basex.query.QueryParser.stringConcat(QueryParser.java:1525) org.basex.query.QueryParser.ftContains(QueryParser.java:1499) org.basex.query.QueryParser.comparison(QueryParser.java:1481) org.basex.query.QueryParser.and(QueryParser.java:1467) org.basex.query.QueryParser.or(QueryParser.java:1453) org.basex.query.QueryParser.single(QueryParser.java:1032) org.basex.query.QueryParser.argumentList(QueryParser.java:2352) org.basex.query.QueryParser.functionCall(QueryParser.java:2308) org.basex.query.QueryParser.primary(QueryParser.java:2063) org.basex.query.QueryParser.postfix(QueryParser.java:2014) org.basex.query.QueryParser.step(QueryParser.java:1895) org.basex.query.QueryParser.path(QueryParser.java:1811) org.basex.query.QueryParser.value(QueryParser.java:1699) org.basex.query.QueryParser.unary(QueryParser.java:1686) org.basex.query.QueryParser.cast(QueryParser.java:1664) org.basex.query.QueryParser.castable(QueryParser.java:1652) org.basex.query.QueryParser.treat(QueryParser.java:1640) org.basex.query.QueryParser.instanceoff(QueryParser.java:1628) org.basex.query.QueryParser.intersect(QueryParser.java:1607) org.basex.query.QueryParser.union(QueryParser.java:1581) org.basex.query.QueryParser.multiplicative(QueryParser.java:1565) org.basex.query.QueryParser.additive(QueryParser.java:1550) org.basex.query.QueryParser.range(QueryParser.java:1539) org.basex.query.QueryParser.stringConcat(QueryParser.java:1525) org.basex.query.QueryParser.ftContains(QueryParser.java:1499) org.basex.query.QueryParser.comparison(QueryParser.java:1481) org.basex.query.QueryParser.and(QueryParser.java:1467) org.basex.query.QueryParser.or(QueryParser.java:1453) org.basex.query.QueryParser.single(QueryParser.java:1032) org.basex.query.QueryParser.flwor(QueryParser.java:1130) org.basex.query.QueryParser.single(QueryParser.java:1021) org.basex.query.QueryParser.expr(QueryParser.java:1001) org.basex.query.QueryParser.parseMain(QueryParser.java:176) org.basex.query.QueryContext.parse(QueryContext.java:176) org.basex.query.QueryProcessor.parse(QueryProcessor.java:53) org.basex.core.cmd.AQuery.updating(AQuery.java:137) org.basex.core.cmd.AQuery.updating(AQuery.java:223) org.basex.gui.GUI.exec(GUI.java:384) org.basex.gui.GUI$7.run(GUI.java:349)

2 2

question about the Java API
by m4058＠aol.com 08 May '13

08 May '13

I would like to know if I can just upload a document to an existing collection or database. Attached is the addExample.java Thanks Mark Shortt

2 1

Improvements. On-fly in-memory temporary index
by Alexander Shpack 07 May '13

07 May '13

Hi Folks! I had one task with csv parsing and transforming. Main file contains approximately 100k of nodes. Transformation process should be in memory only, and I thought, that in-memory temporary index would be useful for this type of tasks. Useful scenario copy $clone := $csvContent modify ( tmp:optimize($clone), ... ) return $clone Main goal - increase performance with huge document fragments, modified and changed on-fly. Is it possible?

2 1

BaseX Capacity
by Rajabrata Chaudhuri 06 May '13

06 May '13

Hello, First I'd like to thank you guys for all your great work on BaseX. I am fairly familiar with XML DBs and have done a significant amount of development on top of Mark Logic. I would like to ask some questions about capacity and scalability. I have reviewed the documentation and see that the biggest store is for SDMX @ approximately 8000 GB. So I am just trying to understand what this means better and would appreciate any of your expert advice for my questions below: 1. Is the expectation that you can query against 8 TB of XML data efficiently? 2. My requirements will be to query across probably 24 TB of XML data. Do you guys feel this is possible? 3. What is the method to scale horizontally and vertically? I.E. Would I be adding more servers, or starting more instances, etc.? 4. How does high availability work? I.E. Can I have multiple active-active nodes, or should it be active-passive, etc.? Any help anyone can render is greatly appreciated. Thanks Raj

3 5

Enhanced Locking in BaseX: Status, "How Do You Use It?" Survey and "Wish List"
by Jens Erat 06 May '13

06 May '13

Dear BaseX-Community, # Status On the Improvements Over the last months we implemented huge enhancements on locking in BaseX. Writing transactions do not block the whole system any more; parallel reading and writing transactions are possible. For the 7.7 release (currently still in beta) we did further improvements like downgrading of locks before applying the pending update list to the database (which is IO-intensive) and more fine granular locks for commands, for example `COPY` only needs a read lock on the source database (which is great for backing up the system without need for stopping it). Nonetheless, estimating which locks are needed is a hard job in such powerful languages like XQuery. Databases can be opened at arbitrary positions in the code, and they can even be dependent on the content of other databases or external resources - it is not always possible to determine them at compile time, which leads to unnecessarily locked databases. # How Do You Use BaseX, What Are Your Locking Requirements? There are definitely more ways to enhance locking in BaseX. We developed the basic infrastructure for further enhancement (like MVCC) and already achieved a noticeable improvement by applying two phase locking, but for knowing how to go on, we'd like to hear your thoughts and needs. Possible ways to go would be "the classic database route" by using MVCC or heading straight to the opposite direction by evaluating optimistic locking or similar "non-locking" protocols. All of these have advantages and drawbacks, I'd be glad to receive your input on a few questions to determine some common use cases: - What is your application doing? - Which BaseX APIs do you use? - How do typical queries look like? If you cannot share them, a description following this guide would suffice: - How often will that query be run? Especially I'm asking you to differentiate between writing and updating queries, and how many databases you will access. - Complexity (using functions, modules, nested loops, lines of code) - Where do function calls opening databases and/or collections occur? - Do you use administrative functions (creating/... databases) inside XQuery? - Do you use / require `xquery:eval(…)`? - Do you access external resources like files or using HTTP requests? Do you need concurrency control for these? - Some enhancements could require transactions to be restarted. This could lead to HTTP-Requests / … to be executed again (we cannot roll back resources outside BaseX). Is this a problem for your application? Would it help if you were notified over the API that the query was restarted? (There will always be an alternative without restarted transactions!) - Any other comments you have with respect to locking and transactions * * * You can post information on that on the mailing list or send me a private [mail] (address below) if you do not share these information publicly. I will develop some use cases for my thesis out of your replies, and we will make decisions based on them how to go on with further improvements. If you cannot / do not want to answer all of these, we're happy on every input you give us. If your reply contains information not to be shared, please give me a hint and it will not be shared beyond the core BaseX team - but we will nonetheless include it in our future plans. Kind regards from Lake Constance, Germany, Jens Erat -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de PGP: 350E D9B6 9ADC 2DED F5F2 8549 CBC2 613C D745 722B

1 0

Salam Sahajetra
by Widiyana Samudra 05 May '13

05 May '13

Assalamualaikum Wr.Wb... Sebelum dan sesudahnya saya ingin memperkenalkan diri saya,nama saya WIDIYANA SAMUDRA berasal dari Indonesia dan bekerja di London (United Kingdom), ingin berkenalan dengan anda di sana. Saya ingin menawarkan satu peluang bisnis yang begitu bagus,dan bagi Anda yang berminat dengan bisnis ... Inilah kesempatan Anda,kapan lagi kalau bukan sekarang bukan.??? Produk ini di namakan (MULITE CLEANSER) kegunaannya untuk mencuci barangan yang sangat berharga sekali seperti batu intan permata yg masih mentah. Dan pada waktu yang sama perusahaan di tempat saya bekerja membutuhkan MULITE CLEANSER dimana mineral tersbut tersedia dari operator yg di Indonesia. Jadi bagi anda yang berminat dengan bisnis ini, saya ingin Anda menjadi Agent untuk menjual produk tersebut ke pada perusahaan tempat saya bekerja, (Anda membeli produk tersebut dari operator yang di Indonesia terlebih dahulu dengan harga 500USD(Lima Juta Rupiah) per karton dan dijualnya kembali ke pada perusahaan di tempat saya bekerja dengan harga 1,500USD(Lima belas Juta per karton) Yang paling penting Niat dan ikhlas untuk mencari rezeki semata-mata karena Allah. Insya Allah, Allah akan membantu kita untuk mencari rezeki yang halal dan berkat,saya ingin Anda buat keputusan yang bijak, Anda bisa merubahnya .. Insya Allah ' Apa bila ada berminat atau ada pertanyaan, silahkan balas email terus ke saya berdasarkan alamat di bawah atau silahkan berikan number supaya bisa di hubungi dan saya akan menerangkannya lebih lanjut di telepon: Insya Allah saya mencoba membantu Anda. Semoga ikatan silaturahmi sesama kita diberkati Allah. Selamat berkenalan. widiyana26(a)yahoo.com Yang benar... WIDIYANA SAMUDRA

1 0

Problem with XQuery Update replace
by Ludovic Kuty 02 May '13

02 May '13

Dear list, I have an XML document in the database which contains datas that has the form below. The attribute "site" represents the location from where the captured data originates, the element "num" contains the number of the measure (the identifier. it is the same as a primary key), "value" contains the value (xs:token even if no WS are allowed inside of it) and "time" the time of the acquisition (xs:dateTime). <datas site="xxx"> <data> <num>5</num> <value>700</value> <time>2013-05-01T21:43:13</time> </data> ... </datas> I want to update specific values at a given time. I usually have 300 or 400 values to update. That means, changing the content of child elements "value" and "time" but not "num" of many "data" elements. I wrote an XQuery script named update.xq that I placed in the Web repository. It will receive the site, the time, the numbers of the values to be updated (PK) and of course the values themselves as variables with a POST HTTP request. I do not use much typing as pretty much everything is xs:string. I guess it could be better but my problem is not there. declare variable $nums as xs:string external; declare variable $values as xs:string external; declare variable $time as xs:dateTime external; declare variable $site as xs:token external; let $n := tokenize($nums,','), $v := tokenize($values,',') return for $x at $i in $n return (replace value of node /datas[@site=$site]/data[num=$x]/value with $v[$i], replace value of node /datas[@site=$site]/data[num=$x]/time with $time) The problem is that sometimes it works and sometimes it gives me an HTTP 400 error with body : Stopped at update.xq, 9/73: [XUTY0008] Single element, text, attribute, comment or pi expected as replace target. I know the replace target has to have the expected format and I think it has. I don't understand why I get this. I though it was due to too long stuff so I divided the list of datas to update into sublists of size 50. Now, the problem appears sometimes but not everytime. When I use another XQuery script test.xq to display what was received, everything looks ok. The test script is : declare variable $nums as xs:string external; declare variable $values as xs:string external; declare variable $time as xs:dateTime external; declare variable $site as xs:token external; let $n := tokenize($nums,','), $v := tokenize($values,',') return for $x at $i in $n return ($x,$v[$i]) For example, this update is ok: <?xml version="1.0"?> <run xmlns="http://basex.org/rest"> <text>update.xq</text> <variable name="site" value="xxx"/> <variable name="time" value="2013-05-02T09:58:38"/> <variable name="nums" value="439,375,128,426,362,243,115,413,349,230,102,400,336,272,217,153,387,259,204,76,438,374,255,127,425,361,114,412,229,101,399,335,271,216,152,386,258,203,75,11,437,373,254,126,424,360,113,411,228,100"/> <variable name="values" value="1515.200,408,true,973.806,3.892,false,false,0.000,0.0,6,true,430.400,false,2,327.670,true,410,true,45.410,true,0.992,408,false,false,575.060,3.545,false,0.995,87.270,false,436.800,false,94.000,45.410,false,410,false,14.135,true,true,2152.800,0,true,true,4.924,3.639,false,0.992,87.070,true" /> </run> But this one is not : <?xml version="1.0"?> <run xmlns="http://basex.org/rest"> <text>update.xq</text> <variable name="site" value="xxx"/> <variable name="time" value="2013-05-02T09:58:38"/> <variable name="nums" value="36,398,334,215,385,202,74,436,372,253,125,423,359,240,112,442,410,378,227,131,99,429,397,365,333,214,118,54,416,384,352,288,233,201,105,73,9,435,403,371,339,275,220,156,124,422,390,358,294,239"/> <variable name="values" value="true,428.400,false,45.059,410,12.710,true,2152.800,0,false,false,2.437,-0.101,false,true,4907.327,0.994,409,0.000,false,true,1012.437,448.800,0.000,false,62.260,false,true,304.600,410,0.0,58,87.249,13.180,true,true,true,2152.800,454.200,0,false,2,757,true,true,0.326,0.000,0.0,41,false" /> </run> I think those documents are similar and I shouldn't see a difference. Either it works for both of them or it doesn't. Any idea to point me in the right direction is welcome. Thank in advance, Ludovic Kuty

2 4

parse against different DTDs?
by Graydon Saunders 30 Apr '13

30 Apr '13

Hi! I'm having a tough time finding where I can use commands to replicate the ticky-box behavior of the GUI to say "use the Java parser" and "parse against the DTD". What I want to be able to do is parse a given document against three different DTDs, validating it each time. It needs to parse against the specific DTD because these DTDs have fixed attributes, and if I don't parse the documents on load, the fixed attribute won't be there and validation will fail. Thanks! Graydon

2 1

xquery:eval yields NullPointerException
by Marco Lettere 29 Apr '13

29 Apr '13

Hi all, on the way to writing a small DSL I stumbled upon the following BaseX 7.6 issue. I'm not able to explain to myself why the first two expressions of the following xquery module return correctly whereas the last yields NullPointerException (as shown in the appended stacktrace). What I'm mtrying to do is to wrap up some modification code into userfriendly function calls that can be used in mini xquery scripts to formalize xml workflow. Thank you for any help, Marco. declare variable $globals := map { 'change' := local:change#1, 'generate' := local:gen#0 }; declare function local:gen(){ }; declare function local:change($doc){ copy $newdoc := $doc modify( insert node <a/> as last into $newdoc ) return $newdoc }; ( xquery:eval("copy $newdoc := modify( insert node <a/> as last into $newdoc ) return $newdoc",$globals), xquery:eval("$generate()", $globals), xquery:eval("$change($doc)", map:new( ($globals, map { "doc" := }))) ) The stacktrace and query plan: Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0_15 OS: Linux, i386 Stack Trace: java.lang.NullPointerException org.basex.query.up.expr.Transform.value(Transform.java:68) org.basex.query.QueryContext.value(QueryContext.java:296) org.basex.query.func.UserFunc.value(UserFunc.java:175) org.basex.query.QueryContext.value(QueryContext.java:296) org.basex.query.func.BaseFuncCall.value(BaseFuncCall.java:68) org.basex.query.QueryContext.value(QueryContext.java:296) org.basex.query.value.item.FuncItem.invValue(FuncItem.java:104) org.basex.query.func.DynamicFunc.value(DynamicFunc.java:56) org.basex.query.QueryContext.value(QueryContext.java:296) org.basex.query.QueryContext.value(QueryContext.java:254) org.basex.query.func.FNXQuery.eval(FNXQuery.java:95) org.basex.query.func.FNXQuery.eval(FNXQuery.java:71) org.basex.query.func.FNXQuery.iter(FNXQuery.java:37) org.basex.query.QueryContext.iter(QueryContext.java:285) org.basex.query.expr.List$1.next(List.java:100) org.basex.query.QueryContext.execute(QueryContext.java:524) org.basex.query.QueryProcessor.execute(QueryProcessor.java:96) org.basex.core.cmd.AQuery.query(AQuery.java:77) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.gui.GUI.exec(GUI.java:397) org.basex.gui.GUI$7.run(GUI.java:349) Compiling: - pre-evaluating map { "change":=function($1) as item()* { local:change(($1)) }, "generate":=function() as item()* { local:gen(()) } } Optimized Query: declare function local:change($doc) { copy $newdoc as node() := $doc modify insert node element a { () } into $newdoc return $newdoc }; declare function local:gen() { element b { () } }; (xquery:eval("copy $newdoc := $generate() modify( insert node <a/> as last into $newdoc ) return $newdoc", map{ "change":=function($1) as item()* { local:change(($1)) }, "generate":=function() as item()* { local:gen(()) } }), xquery:eval("$generate()", map{ "change":=function($1) as item()* { local:change(($1)) }, "generate":=function() as item()* { local:gen(()) } }), xquery:eval("$change($doc)", map:new((map{ "change":=function($1) as item()* { local:change(($1)) }, "generate":=function() as item()* { local:gen(()) } }, map { "doc":=element b { () } })))) Query plan: <QueryPlan> <UserFuncs> <UserFunc name="local:change" arg0="doc"> <Transform> <Let var="$newdoc as node()"> <VarRef> <Var name="$doc" id="2"/> </VarRef> </Let> <Insert> <VarRef> <Var name="$newdoc as node()" id="3"/> </VarRef> <CElem> <QNm value="a" type="xs:QName"/> </CElem> </Insert> <VarRef> <Var name="$newdoc as node()" id="3"/> </VarRef> </Transform> </UserFunc> <UserFunc name="local:gen"> <CElem> <QNm value="b" type="xs:QName"/> </CElem> </UserFunc> </UserFuncs> <VarStack> <Var name="$globals" id="0"/> </VarStack> <List> <FNXQuery name="eval(string[,bindings])"> <Str value="copy $newdoc := $generate()
 modify(
 insert node <a/> as last into $newdoc 
 )
 return $newdoc" type="xs:string"/> <Map size="2"> <Str value="change" type="xs:string"/> <FuncItem type="function(item()*) as item()*"> <Var name="$1" id="1"/> <BaseFuncCall name="local:change(($1))"> <VarRef> <Var name="$1" id="1"/> </VarRef> </BaseFuncCall> </FuncItem> <Str value="generate" type="xs:string"/> <FuncItem type="function() as item()*"> <BaseFuncCall name="local:gen(())"/> </FuncItem> </Map> </FNXQuery> <FNXQuery name="eval(string[,bindings])"> <Str value="$generate()" type="xs:string"/> <Map size="2"> <Str value="change" type="xs:string"/> <FuncItem type="function(item()*) as item()*"> <Var name="$1" id="1"/> <BaseFuncCall name="local:change(($1))"> <VarRef> <Var name="$1" id="1"/> </VarRef> </BaseFuncCall> </FuncItem> <Str value="generate" type="xs:string"/> <FuncItem type="function() as item()*"> <BaseFuncCall name="local:gen(())"/> </FuncItem> </Map> </FNXQuery> <FNXQuery name="eval(string[,bindings])"> <Str value="$change($doc)" type="xs:string"/> <FNMap name="new([maps[,coll]])"> <List> <Map size="2"> <Str value="change" type="xs:string"/> <FuncItem type="function(item()*) as item()*"> <Var name="$1" id="1"/> <BaseFuncCall name="local:change(($1))"> <VarRef> <Var name="$1" id="1"/> </VarRef> </BaseFuncCall> </FuncItem> <Str value="generate" type="xs:string"/> <FuncItem type="function() as item()*"> <BaseFuncCall name="local:gen(())"/> </FuncItem> </Map> <LitMap> <Str value="doc" type="xs:string"/> <CElem> <QNm value="b" type="xs:QName"/> </CElem> </LitMap> </List> </FNMap> </FNXQuery> </List> </QueryPlan>

2 2

validation that returns node identifiers?
by Graydon Saunders 26 Apr '13

26 Apr '13

Hi! Using BaseX77-20130417.002917 though I don't think that's relevant to the question. This may well be an inherent property of the library being used to do validation, but right now, when I use validate:xsd-info, I get error messages that look like: Error: 25:59: cvc-complex-type.2.4.a: Invalid content was found starting with element 'link'. One of '{bold, footnote, italic, superscript}' is expected. These identify the location of the error by line and column number, which is great for code and less than helpful for XML content, since any format-and-indent getting applied to the XML file destroys its usefulness. Is is possible to have the error return a node identifier of some kind, instead? /root/bucket[2]/block[4]/para[3]/emphasis[4] sort of thing? Thanks! Graydon

2 1

Bypassing file locking in BaseX
by Scott Martens 25 Apr '13

25 Apr '13

Hi all, I have an application that's embedding BaseX in Java and generates all xqueries internally. These queries are *always* read-only. The problem I'm running into is that file locking prevents more than one user from reading the same file at the same time - something that BaseX 7.2 didn't do, but BaseX 7.6 does. I've tried using the client-server approach to solve the problem, and it works, sort-of. But it has a number of other problems for the conditions that it's being run under, and the embedded solution was faster and more robust. It looks to me like the BaseX standalone command line app and basexgui just ignore file-locking, and as long as I'm just using read-only queries, I have no trouble issuing two queries at once on the same database. Is this true? And if so, how is it done? Because this app is doing only read-queries, it's a lot better a solution for me than client-server, but I can't figure out where in the source code the database is being opened and how it's being done in a way that bypasses locks. Can anyone give me pointers? -- Scott Martens

3 3

Indexes creation fails on big database
by Erdal Karaca 23 Apr '13

23 Apr '13

Hi all, I have downloaded/started the latest dev snapshot of 7.7. My database consists of an extract of the OpenStreetMap data (only Germany) which is 29GB in size (xml/OSM format). I have managed to load the file into the database but the indexes creation procedure fails. I have tried -Xmx4g, but if the progress bar of the GUI is right, I would need the 10x of heap space to create the attribute indexes. Any ideas/workarounds to create the attribute indexes for such a big database? Thanks!

2 3

Geospatial indexing
by Erdal Karaca 23 Apr '13

23 Apr '13

Hi all, Just noticed that basex supports geospatial queries since 7.6. But, how do I index geospatial data in my database? Or is that a missing feature? Thanks! Best regards, Erdal

2 1

inserting ampersand
by marco fregonese 23 Apr '13

23 Apr '13

Hi there, I have the following xquery function: *declare updating function local:insert ($a) { insert node <title>{$a}</title> into /web }; local:insert ("&")* but if I try to insert an apersand, basex says "invalid entity '&' ". So, my question: is there any way to insert an ampersand? Thank you.

2 1

Full-Text search, how to mark whole phrases
by Bartosz Marciniak 19 Apr '13

19 Apr '13

Hi, I have placed small document in database with enabled full-text index: <text> Text with hello world phrase. </text> The following expression: ft:mark(db:open('simple')//*[text() contains text 'hello world']) produces: <text>Text with hello world phrase.</text> Is it possible to enclose result in one element hello world ? Regards Bartosz Marciniak

3 2

Fwd: whitespace around comments
by Wendell Piez 19 Apr '13

19 Apr '13

Oops -- just sent this to Christian only, when it was meant for the list -- ---------- Forwarded message ---------- From: Wendell Piez <wapiez(a)wendellpiez.com> Date: Tue, Apr 16, 2013 at 11:15 AM Subject: Re: [basex-talk] whitespace around comments To: Christian Grün <christian.gruen(a)gmail.com> Hi, Thanks to Cerstin for once again urging that CHOP not be set by default, and to Liam for reminding readers that this is *not* a new issue in markup processing -- indeed it was in view of problems with whitespace in earlier technologies (can anyone say "pernicious mixed content"?) that the XML Rec says what it does -- including section http://www.w3.org/TR/REC-xml/#sec-white-space. And thanks also to Christian for reminding us why much of the time, whitespace chopping is innocuous and helpful. Personally, I'm in an uneasy place with respect to these issues. I wholeheartedly agree with Cerstin and Liam that chopping should not be on by default. Leaving it on by default is wrong and non-conformant. However, I'm also working with an application in which it's very convenient to have it -- not only is the data highly structured and controlled -- no mixed content here -- but also part of its job is to offer clean representations of external data that is much dirtier and cruftier. It is supposed to clear bad data away and clean it up, so managing the whitespace on ingest is just part of what we are doing. For both scaling and usability issues (as Christian has suggested), whitespace chopping is nice to have. Yet at the same time, we are already calling XSLT for uses in which the chopping is destructive, and the day is not very far off when we will have mixed content in our primary data set too. Liam points out something very important: it is possible in principle to distinguish between whitespace that can be safely discarded (by design) and whitespace that can't -- if you have a schema or other specification that represents this design. As he notes, the XML Rec distinguishes between "significant" and "insignificant" whitespace by reference to content models that do and don't include #PCDATA (that is, whitespace that appears in "element content" or "mixed content"; cf http://www.w3.org/TR/REC-xml/#dt-elemcontent). If your content model for div says (p+), then whitespace between the 'p' element children of a 'div' (but not inside them) may often be judged safe to discard. (At least in a system in which a schema is used as a warrant of fitness for processing.) When technologies such as XQuery or XSLT are designed to work with and without schemas, however -- or where schemas cannot be considered as reliable indicators of markup semantics -- even relying on this mechanism can't solve the problem (to say nothing of deciding which schema languages you support). However, it can help to mitigate it. Then too, even XSLT 1.0 has strip-space and preserve-space configuration to indicate to a processor where it can "chop" whitespace. While it's a bit crude (it treats all elements with the same name the same), it can be useful. Over the longer term, therefore, I think that (1) CHOP needs to be "false" by default, (2) it should be possible to turn it on (just as I am learning how to turn it off), and also (3) that we also need more flexible and configurable means for discriminating how it should work, with and without schemas to reference. Cheers, Wendell On Sat, Apr 13, 2013 at 7:05 AM, Christian Grün <christian.gruen(a)gmail.com> wrote: > I’d like to add some more info on why we initially decided to chop > whitespaces, and why a sudden change of the default value may break > existing applications (if you know the details, simply skip this > section..): > > Many XML documents contain whitespace-only text nodes for properly > indenting elements. In highly structured data (i.e., when not working > with mixed content), these nodes are in fact completely irrelevant. > For example, if the following document… > > <xml> > <a>X</a> > </xml> > > …is parsed with CHOP set to true, we will get a document with a single > text node. The following query… > > for $t in //text() > return replace node $t with 'x' > > …will generate the following result: > > <xml> > <a>x</a> > </xml> > > If we set CHOP to false, the document will have three text nodes, two > of them whitespace-only, and the same query will create the following > result document: > > <xml>x<a>x</a>x</xml> > > This is just one example to demonstrate that a sudden change of the > default for chop would most probably lead to unwanted side effects in > existing applications. Another side effect: databases are expected to > increase in size, as all whitespace nodes will get their own node ids, > will be fully stored and indexed, etc. > > However, I completely agree that the removal of whitespaces may lead > to serious changes in mixed contents, and I easily admit that we > haven’t been aware of all the implications some years ago when we > started off designing the database. While I still believe that our > storage copes pretty well with nowaday’s requirements, I would love to > have some weeks off to completely rebuild it, and include > optimizations for all kinds of features that are relevant today > (including larger ranges for node ids and namespaces, or support for > other tree formats such as json). > > Thanks for reading, > Christian > ___________________________ > > On Sat, Apr 13, 2013 at 8:28 AM, Liam R E Quin <liam(a)w3.org> wrote: >> On Fri, 2013-04-05 at 11:31 +0200, Dirk Kirsten wrote: >> >>> So if you could point out some details as why this is not conforming >>> behaviour, this would be interesting. >> >> It's a requirement in the XML Spec that the XML parser pass all >> whitespace back to the application. Some whitespace may be marked as not >> significant - that is only possible if there's a DTD and the space is in >> a context where only elements would be valid, not #PCDATA. There's no >> formal specification, although constructing an XDM instance from an >> infoset, and constructing an infoset from XML, does not entail >> discarding these spaces: >> Chopping internal whitespace nodes in mixed content contexts is not >> sanctioned by any version of any XML specification, with any setting of >> xml:space. I think the onus would be on you to justify the non-standard >> behaviour. >> >> On the other hand I can see its uses too. But I don't want it, and >> always turn it off with BaseX :-) >> >> Best, >> >> Liam >> >> -- >> Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ >> Pictures from old books: http://fromoldbooks.org/ >> Ankh: irc.sorcery.net irc.gnome.org freenode/#xml >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^ -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

2 3

java.lang.NoClassDefFoundError while deploying on Tomcat server
by Carlos MARCOS LAGUNAR 19 Apr '13

19 Apr '13

Hi, I've downloaded last BaseX version from the download page and what I've done it is just put the BaseX76.war into webapps deployment directory of Tomcat. As far as I know, it has deployed correctly, since I can make REST requests successfully like http://MiServer/BaseX76/rest?command=show+users. The problem comes when I try to access it from my Java Application, where I've tried with two methods: JavaClient and XQJ. With JavaClient it works, but with XQJ the response is always the same (log from [TOMCAT_DEPLOYMENT_DIRECTORY]/BaseX76/data/.logs): 13:32:40.053 172.24.41.93:59521 admin ERROR Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0 OS: Linux, i386 Stack Trace: java.lang.NoClassDefFoundError: org/basex/index/value/DiskValues org.basex.data.DiskData.<init>(DiskData.java:88) org.basex.core.cmd.Open.open(Open.java:70) org.basex.core.cmd.Open.run(Open.java:36) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145) 453.12 ms The block of Java code where the Exception is thrown is the following (for the case of XQJ): log.debug("[BaseX] executing query:"+query); XQDataSource xqs = new BaseXXQDataSource(); xqs.setProperty("databaseName", dbName); xqs.setProperty("serverName", serverNameBaseX); xqs.setProperty("port", portBaseX); xqs.setProperty("user", usuario); xqs.setProperty("password", clave); xqs.setLogWriter(new PrintWriter(System.out, true)); XQConnection xqc = xqs.getConnection(); ß Here is where the Exception is thrown XQExpression xqe = xqc.createExpression(); XQSequence xq = xqe.executeQuery(query); StringBuilder res = new StringBuilder(); while (xq.next()) { res.append(xq.getItemAsString(null)); } log.debug("[BaseX] result:"+res); xqc.close(); It seems that it is not finding the jars inside the WAR where DiskData class is located, but I've entered into the Tomcat directory where the WAR has been deployed (/BaseX76/WEB-INF/lib) and all of them are there: -rw-rw---- 1 2558399 2013-02-05 18:22 basex-7.6.jar -rw-rw---- 1 295821 2013-01-30 11:42 basex-xqj-1.2.3.jar -rw-rw---- 1 58160 2013-01-30 11:42 commons-codec-1.4.jar -rw-rw---- 1 59590 2013-01-30 11:42 commons-fileupload-1.2.2.jar -rw-rw---- 1 109043 2013-01-30 11:42 commons-io-1.4.jar -rw-rw---- 1 153115 2013-01-30 11:42 jdom-1.1.jar -rw-rw---- 1 794991 2013-01-31 15:34 jts-1.13.jar -rw-rw---- 1 410933 2013-01-30 11:42 milton-api-1.7.2.jar -rw-rw---- 1 119180 2013-01-30 11:42 mime-util-2.1.3.jar -rw-rw---- 1 23445 2013-01-30 11:42 slf4j-api-1.5.8.jar -rw-rw---- 1 5284 2013-01-30 11:42 slf4j-nop-1.5.8.jar -rw-rw---- 1 9260 2013-01-30 11:42 xmldb-api-1.0.jar -rw-rw---- 1 7188 2013-01-30 11:45 xmlpull-1.1.3.1.jar -rw-rw---- 1 24956 2013-01-30 11:45 xpp3_min-1.1.4c.jar -rw-rw---- 1 4578 2013-01-30 11:42 xqj2-0.1.0.jar -rw-rw---- 1 16221 2013-01-30 11:42 xqj-api-1.0.jar -rw-rw---- 1 481672 2013-01-30 11:42 xstream-1.4.2.jar Any ideas?. Other projects deployed into that server worked without any problem. I know that I could use JavaClient method for accessing but then the problem is that I cannot recover the result of the query execution, while with XQJ I can: JavaClient: log.debug("[BaseX] executing query:"+query); ClientSession cSes = new ClientSession(serverNameBaseX, Integer.valueOf(portBaseX), user, pass); cSes.setOutputStream(System.out); ClientQuery cQuery = cSes.query(query); cQuery.execute(); String res = cQuery.info(); ß This res only stores the execution time and not the result of the query, so it is useless (for me :-) ) cQuery.close(); XQJ: XQSequence xq = xqe.executeQuery(query); ß Instead, this do stores the result of the query (exactly what I want) Thank you and best regards, C. Carlos Marcos Lagunar Health Technical Consultant Research & Innovation carlos.marcos(a)atosresearch.eu <mailto:carlos.marcos@atosresearch.eu> Valladolid - Spain www.atosresearch.eu <http://www.atosresearch.eu/> es.atos.net ------------------------------------------------------------------ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. Este mensaje y los ficheros adjuntos pueden contener informacion confidencial destinada solamente a la(s) persona(s) mencionadas anteriormente pueden estar protegidos por secreto profesional. Si usted recibe este correo electronico por error, gracias por informar inmediatamente al remitente y destruir el mensaje. Al no estar asegurada la integridad de este mensaje sobre la red, Atos no se hace responsable por su contenido. Su contenido no constituye ningun compromiso para el grupo Atos, salvo ratificacion escrita por ambas partes. Aunque se esfuerza al maximo por mantener su red libre de virus, el emisor no puede garantizar nada al respecto y no sera responsable de cualesquiera danos que puedan resultar de una transmision de virus. ------------------------------------------------------------------

2 2

Running and configuring update operations outside the GUI
by Wendell Piez 18 Apr '13

18 Apr '13

Hi, I too have a question about database updates, in particular Fabrice's remark that "The GUI may not be the best place to run updates, did you try the basex command line tools ?" I have an update that is working okay from the GUI, but I wouldn't mind it if it were faster. :-) Is simply invoking the same XQuery using RUN from the command line all I have to do to get the benefits of using the CL? In addition: my update query is configured with a bit of XML, something like declare variable $build-db := <build db="new"> <source db="old1"/> <source db="old2"/> <source db="old3"/> </build>; (Of course it's more complicated than this but you get the essence.) But I can't pass this XML into a function to perform the updates, since I can't put update operations into a function at all. (So the XML is piled into a global variable, which the update operations reference.) Do you have any suggestions for how I can encapsulate the updating logic and also make it configurable externally like this? And run it from the command line or a script for speed and efficiency? Thanks, Wendell -- Wendell Piez | http://www.wendellpiez.com XML | XSLT | electronic publishing Eat Your Vegetables _____oo_________o_o___ooooo____ooooooo_^

3 3

Adding millions of XML files
by freesoft 18 Apr '13

18 Apr '13

Hi, I'm new to BaseX and to XQuery. I already knew XPath. I'm evaluating BaseX to store our XML files and make queries on them. We have to store about 1 million of XML files per month. The XML files are little (~1 KB to 5 KB). So our case is: High number of files, little size. I've read that BaseX is scalable and has high performance, so it is probably a good tool for us. But, in the tests I'm doing, I'm getting an "Out of Main Memory" error when loading high number of XML files. For exaple, if I create a new database ("testdb"), and add 3 XML files, no problem occurs. Files are stored correctly, and I can make queries on them. Then, if I try to add 18000 XML files to the same database ("testdb") (by using GUI > Database > Properties > Add Resources), then I see how the coloured memory bar grows and grows... until an error appears: Out of Main Memory. You can try to: - increase Java's heap size with the flag -Xmx<size> - deactivate the text and attribute indexes. The text and attribute indexes are disabled, so it is not the cause. And I increased the Java size with the flag -Xmx<size> (by editing the basexgui.bat script), and same error happens. Probaly BaseX loads all files to main memory first, and then dumps them to the database files. That shouldn't be done in that way. For each XML file, it should be loaded into main memory, then procesed and then dumped to the db files. For each file, independently from the rest. So I have two questions: 1. Do I have to use an special way to add high number of XML files? 2. Is BaseX sufficiently stable to store and manage our data (about 1 million of files will be added per month)? Thank you for our help and for your great software, and excuse me if I am asking for recurrent questions.

3 7

timezone formatting
by Marco Lettere 17 Apr '13

17 Apr '13

Hi again, when, in Basex 7.6, formatting a datetime with format-dateTime(current-dateTime(), "[Y0001][M01][D01][h01][m01][s01][Z0000]") I was expecting something like 20130314035118+0100 for my area (Central Europe) and instead I get 201303140351180060. Looks like minutes are not translated to hours and the sign is missing. For documentation I'm referring to http://www.w3.org/TR/xpath-functions-30/#rules-for-datetime-formatting. Thanks, M.

2 3

Re: [basex-talk] Adding millions of XML files
by pw＠themail.co.uk 17 Apr '13

17 Apr '13

Responding to the last question about querying over collections: I had the same issue and Lukas Kircher provided the answer - http://www.mail-archive.com/basex-talk@mailman.uni-konstanz.de/msg0210 0.html > > > >---- Original Message ---- >From: kqfhjjgrn(a)yahoo.es >To: christian.gruen(a)gmail.com, fetanchaud(a)questel.com >Subject: Re: [basex-talk] Adding millions of XML files >Date: Mon, 15 Apr 2013 13:12:55 +0100 (BST) > >>Worked! :-) >> >>I have uninstalled 7.6 and installed 7.7 beta. Then, created the >empty db, added the 3 files, run the "set addcache true" command, >added the 17828 files... and no "out of memory" error, just the >processing info: >> >> Path "everything" added in 462943.7 ms. >> >>that is ~8 minutes (in my development machine, not in our server). >> >>Now I'm going to do some more tests (both for adding and for >quering), and I'm going to try the "basex" command, in order to add >XML files automatically to the db. >> >>Anyway I would ask some more questions: >> >>1. Is 7.7 beta sufficiently stable to be used in our production >server? Shoud I wait for the final 7.7 release? >> >>2. Is the "addcache" property value permanently saved to the db? >Should I run the "set addcache true" command everytime I add files? >> >>3. Should I keep disabled the Text & Attribute indexes? Is the >"addcache=on" option sufficient to allow the adition of XML files, so >I can enable those indexes? Will my queries be slow with those >indexes disabled? >> >>4. Should I run Optimize after every massive insertion (even with >"addcache=on")? >> >>Thank you for the information on limits, it is very useful. In >particular, the following limits: >> >>FileSize: 512 GiB >>#Files: 536,870,912 >> >>mean a medium value of exactly 1 KB/file. Since my files are bigger >than 1 KB (in medium), then the size limit will be reached first (512 >GiB). So my Perl scripts will have to detect the size of the db, and >if it is bigger than ~500 GB, then they will create a new db and add >new XML files to it. >> >>Please show me an easy example of how to use several databases in >the same query. Perhaps something like: >> >> for $doc in (collection("db1"), collection("db2")) >> for $node in $doc/$a_node_path >> etc... >> >>Well, thank you very much for your help. And excuse me for the huge >amount of questions from a newbie like me :-) >> >> >>freesoft >> >> >> >>________________________________ >> De: Christian Grün <christian.gruen(a)gmail.com> >>Para: Fabrice Etanchaud <fetanchaud(a)questel.com> >>CC: freesoft <kqfhjjgrn(a)yahoo.es>; >"basex-talk(a)mailman.uni-konstanz.de" ><basex-talk(a)mailman.uni-konstanz.de> >>Enviado: Lunes 15 de abril de 2013 12:12 >>Asunto: Re: [basex-talk] Adding millions of XML files >> >> >>Hi kgfhjjgrn, >> >>I believe that Fabrice already mentioned all details that should >help >>you to build larger databases. The ADDCACHE option [1] (included in >>the latest stable snapshot [2]) may already be sufficient to add >your >>documents via the GUI: simply run the "set addcache true" command >via >>the input bar of the main window before opening the Properties >dialog. >> >>Note that you can access multiple databases with a single XQuery >call, >>so if you know that youll exceed the limits of a single database at >>some time (see [3]), simply create new databases in certain >intervals. >> >>Hope this helps, >>Christian >> >>[1] http://docs.basex.org/wiki/Options#ADDCACHE >>[2] http://files.basex.org/releases/latest/ >>[3] http://docs.basex.org/wiki/Statistics >>_________________________________________ >> >>> The size of your test should not cause any problem to BaseX (18 >000 files >>> from 1 up to 5 KB) >>> >>> >>> >>> 1. Did you try to set the ADDCACHE option ? >>> >>> 2. You should OPTIMIZE your collection after each batch of >ADD >>> commands, even if no index is set. >>> >>> 3. Did you try to unset the AUTOFLUSH option, and explicitly >FLUSH the >>> updates at batchs end ? >>> >>> 4. The GUI may not be the best place to run updates, did you >try the >>> basex command line tools ? >>> >>> >>> >>> Opening a collection containing a huge number of documents may >take a long >>> time from my experience. >>> >>> It seems to be related to the kind of memory data structure used >to store >>> the document names. >>> >>> A workaround could be to insert your documents under a common root >xml >>> element with XQuery Update. >>> >>> >>> >>> >>> >>> >>> Best, >>> >>> Fabrice Etanchaud >>> >>> Questel-Orbit >>> >>> >>> >>> >>> >>> De : basex-talk-bounces(a)mailman.uni-konstanz.de >>> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de >freesoft >>> Envoyé : lundi 15 avril 2013 10:19 >>> À : basex-talk(a)mailman.uni-konstanz.de >>> Objet : [basex-talk] Adding millions of XML files >>> >>> >>> >>> Hi, I'm new to BaseX and to XQuery. I already knew XPath. I'm >evaluating >>> BaseX to store our XML files and make queries on them. We have to >store >>> about 1 million of XML files per month. The XML files are little >(~1 KB to 5 >>> KB). So our case is: High number of files, little size. >>> >>> I've read that BaseX is scalable and has high performance, so it >is probably >>> a good tool for us. But, in the tests I'm doing, I'm getting an >"Out of Main >>> Memory" error when loading high number of XML files. >>> >>> For exaple, if I create a new database ("testdb"), and add 3 XML >files, no >>> problem occurs. Files are stored correctly, and I can make queries >on them. >>> Then, if I try to add 18000 XML files to the same database >("testdb") (by >>> using GUI > Database > Properties > Add Resources), then I see how >the >>> coloured memory bar grows and grows... until an error appears: >>> >>> Out of Main Memory. >>> You can try to: >>> - increase Java's heap size with the flag -Xmx<size> >>> - deactivate the text and attribute indexes. >>> >>> The text and attribute indexes are disabled, so it is not the >cause. And I >>> increased the Java size with the flag -Xmx<size> (by editing the >>> basexgui.bat script), and same error happens. >>> >>> Probaly BaseX loads all files to main memory first, and then dumps >them to >>> the database files. That shouldn't be done in that way. For each >XML file, >>> it should be loaded into main memory, then procesed and then >dumped to the >>> db files. For each file, independently from the rest. >>> >>> So I have two questions: >>> 1. Do I have to use an special way to add high number of XML >files? >>> 2. Is BaseX sufficiently stable to store and manage our data >(about 1 >>> million of files will be added per month)? >>> >>> Thank you for our help and for your great software, and excuse me >if I am >>> asking for recurrent questions. >>> >>> >>> _______________________________________________ >>> BaseX-Talk mailing list >>> BaseX-Talk(a)mailman.uni-konstanz.de >>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>>

2 1

Proposition. Execution feedback
by Alexander Shpack 15 Apr '13

15 Apr '13

Hi! It would be perfect to take execution feedback from GUI or server. Execution plan: N nodes Processed: K nodes It may help in script optimization and understanding complexity of xQuery task -- s0rr0w

2 1

Use xquery:eval in insert statement
by Lauener Jonas 15 Apr '13

15 Apr '13

I want to use xquery:eval in an insert statement: declare updating function local:insert($document as xs:string, $input as element()) { insert node $input after xquery:eval(fn:concat($document, '//', node-name($input), '[last()]')) }; let $input := <Customer CustomerID="IZZTT"><CompanyName>App</CompanyName><ContactName>Steve Jobs</ContactName><ContactTitle>CEO in Heaven</ContactTitle><Phone>(777) 7777777777</Phone><FullAddress><Address>Apple Road 3</Address><City>Cuppertino</City><Region>OR</Region><PostalCode>00000</PostalCode><Country>Heaven</Country></FullAddress></Customer> return local:insert("doc('CustomersOrders.xml')", $input) On the BaseX GUI it works fine, but if I want to use it with the Java Client, it fails. I use this code in Java: public final class InsertExample { /** Hidden default constructor. */ private InsertExample() {} /** Session reference. */ static ClientSession session; /** * Main method. * @param args command-line arguments */ public static void main(final String[] args) { try { // Create a client session with host name, port, user name and password session = new ClientSession("localhost", 1984, "admin", "admin"); String command = "declare updating function local:insert($document as xs:string, $input as element()) {\n insert node $input after xquery:eval(fn:concat($document, '//', node-name($input), '[last()]'))\n};\n\nlet $input := <Customer CustomerID=\"IZZTT\"><CompanyName>App</CompanyName><ContactName>Steve Jobs</ContactName><ContactTitle>CEO in Heaven</ContactTitle><Phone>(777) 7777777777</Phone><FullAddress><Address>Apple Road 3</Address><City>Cuppertino</City><Region>OR</Region><PostalCode>00000</PostalCode><Country>Heaven</Country></FullAddress></Customer>\n\nreturn local:insert(\"doc('CustomersOrders')\", $input)"; System.out.println(command); XQuery xquery = new XQuery(command); System.out.println(session.execute(xquery)); // close session session.close(); } catch(IOException e) { e.printStackTrace(); } } } When I run it, I get this exception: Version: BaseX 7.6 Java: Oracle Corporation, 1.7.0_11 OS: Windows 8, amd64 Stack Trace: java.lang.RuntimeException: Exclusive lock could not be acquired. org.basex.util.Util.notexpected(Util.java:53) org.basex.io.random.TableDiskAccess.lock(TableDiskAccess.java:151) org.basex.data.DiskData.startUpdate(DiskData.java:202) org.basex.query.up.DatabaseUpdates.startUpdate(DatabaseUpdates.java:158) org.basex.query.up.ContextModifier.apply(ContextModifier.java:86) org.basex.query.up.Updates.apply(Updates.java:120) org.basex.query.QueryContext.update(QueryContext.java:270) org.basex.query.QueryContext.value(QueryContext.java:255) org.basex.query.QueryContext.iter(QueryContext.java:240) org.basex.query.QueryProcessor.iter(QueryProcessor.java:76) org.basex.core.cmd.AQuery.query(AQuery.java:84) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:342) org.basex.core.Command.exec(Command.java:321) org.basex.core.Command.execute(Command.java:78) org.basex.server.ClientListener.run(ClientListener.java:145) at org.basex.server.ClientSession.receive(ClientSession.java:259) at org.basex.server.ClientSession.execute(ClientSession.java:162) at org.basex.server.ClientSession.execute(ClientSession.java:167) at org.basex.server.Session.execute(Session.java:37) at org.basex.examples.api.InsertExample.main(InsertExample.java:42) Is this a bug in BaseX? Or how can I solve this issue? Thanks, Jonas

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk