- BaseX-Talk - mailman.uni-konstanz.de

Can't update or delete JSON objects
by E. Wray Johnson 13 Nov '17

13 Nov '17

I have tried XQUERY updates that should work but they don't work. My technique of storing JSON documents is perhaps not traditional. I have reduced my example to this simple example. FRIST CREATE DATABASE WITH ONE DOCUMENT: db:create('geography', json:parse('{ "code": "USA", "name": "United States of America", "birth": 1776 }'), '/country') THEN ADD ANOTHER DOCUMENT: db:add('geography', json:parse('{ "code": "CAN", "name": "Canada", "birth": 1867 }'), '/country') 1. How can I update either of these documents using 'code' as the unique identifier? 1. How can I change the 'name' or 'birth'? 2. How can I add a 'population' number? 2. How can I delete either of these documents using 'code' as the unique identifier?

2 1

GEO module not found with BaseX GUI
by E. Wray Johnson 12 Nov '17

12 Nov '17

Error: Stopped at D:/projects/xquery/geo test.xq, 1/57: [XQST0059] Module not found: http://expath.org/ns/geo. Query: import module namespace geo = "http://expath.org/ns/geo"; declare namespace gml='http://www.opengis.net/gml'; for $x in db:open('geonames2','country') where index-of(['US'], $x/json/countryCode)>0 return geo:envelope( <gml:LinearRing> <gml:coordinates>($x/west, $x/south), ($x/east, $x/south), ($x/east, $x/north), ($x/west, $x/north), ($x/west, $x/south)</gml:coordinates> </gml:LinearRing> ) OS: Windows 7 Installation Method: Windows installer Launch: basexgui.bat I had this problem long ago. https://www.mail-archive.com/search?l=basex-talk@mailman.uni-konstanz.de&q=…

3 12

repo install
by Kendall Shaw 10 Nov '17

10 Nov '17

I have been updating packages by modifying the files in REPOPATH instead of creating a xar file and using repo install. Is it safe to expect that to continue to work for the foreseeable future? Kendall

2 1

HTTPS:PATCH - Request using http:send-request()
by Wissemann, Bodo 07 Nov '17

07 Nov '17

1 0

Issue with Full Text Retrieval
by Ron Katriel 06 Nov '17

06 Nov '17

Hi, I am seeing strange behavior with Full Text retrieval. The following query fails for a number of words that are in the XML document (see attached): for $trial in db:open('CTGovDebug') (: [clinical_study/id_info/nct_id='NCT00473512'] :) return $trial contains text { 'neoplasms' } It fails on a good number of words including neoplasms, cougar, industry, yes, completed, november, 2005, interventional, single, male, female, assignment, none, research, principal, primary, secondary, age, years, gender, etc. But it matches most of the words in the file. Observation: The words that fail are located at the beginning and/or end of the text *and* do not occur anywhere else in the middle of any text. The document is the only one in the database. It does not make a difference whether full text indexing is on or off. My BaseX version is 8.6.4. Thanks, Ron Ron Katriel, Ph.D. | Principal Data Scientist | Medidata Solutions <http://www.mdsol.com/> 350 Hudson Street, 7th Floor, New York, NY 10014 rkatriel(a)mdsol.com <tbrophy(a)mdsol.com> | direct: +1 201 337 3622 <//201 337 3622> | mobile: +1 201 675 5598 <//+1 201 675 5598> | main: +1 212 918 1800 <//+1 212 918 1800>

4 9

Re: [basex-talk] Memory requirements, release memory, heap shrink
by Dinu Marina 04 Nov '17

04 Nov '17

Indeed, I use the create function from the GUI, I just assumed it's the same 2 separate operations. Indeed, with CREATE DB it doesn't get out of memory at 1G. And it also gets GC'ed and returned to system afterwards with no additional intervention, after CREATE DB memory shrinks immediately back to ~30M. So confirmed, huge memory usage and memory "leak" (or whatever it is) is linked to ADD only. Thanks, Dinu On 04.11.2017 18:46, Christian Grün wrote: > Hi Dinu, > > yes, I have downloaded the file. > > Just one more question: > >> 2) using basexclient: >> >> CHECK somedb >> ADD /path/to/1_feed.zip > If you use the GUI, do you really add your zip file to an existing > database, or do you specify it as initial input when creating a new > database? The latter option is definitely more efficient, and the > command-line equivalent would be > > CREATE DB somedb /path/to/1_feed.zip > > For adding resources to existing databases, enabling ADDCACHE can help [1]. > > Cheers, > Christian > > [1] http://docs.basex.org/wiki/Options#ADDCACHE > > > >> Result: >> Out of Main Memory. >> >> To reproduce 2), start server with -Xmx2048m, repeat operations, then drop >> db, close client, check server memory usage. >> >> Thanks, >> Dinu >> >> >> >> On 04.11.2017 18:18, Christian Grün wrote: >>>> The fact is, the GUI runs with no problem with -Xmx512M to do the same >>>> thing, while basexclient fails without -Xmx2048M. >>> That’s surprising indeed – mostly because I would have expected the >>> BaseX client to always consume a small and constant amount of memory >>> (the BaseX server instance should be the process to consume all the >>> memory). I did some quick tests with large zipped input, but I failed >>> to reproduce the behavior you described. Feel free to provide me with >>> a step-by-step guide. >>> >>>> I will try that, thanks, but shouldn't this be the case automatically? >>>> Since >>>> I assume BaseX does free references to data structures, at least to a >>>> dropped DB? >>> Absolutely. Anything that’s reproducible is welcome. >>> >>> >>> >>>> On 04.11.2017 18:00, Christian Grün wrote: >>>>> Hi Dinu, >>>>> >>>>>> Question 1: >>>>> Memory consumption of the BaseX GUI is similar as on command-line, but >>>>> it may be due to garbage collection that some memory will be freed. >>>>> How do you add documents outside the GUI? >>>>> >>>>>> Question 2: >>>>> If a certain amount of memory is reserved by Java’s virtual machine, >>>>> it may still be used by other applications on your system (provided >>>>> that the memory can be freed by garbage collection). You can enforce >>>>> some GC calls by running the following XQuery expression (this should >>>>> only be done for testing purposes): >>>>> >>>>> (1 to 5) ! Q{java:java.lang.System}gc() >>>>> >>>>> Best, >>>>> Christian >>>>> >>>>> >>>>>> After the data is extracted, it's no longer needed and I DROP the DB; >>>>>> also >>>>>> connection is closed. But memory (the huge 2G mentioned above) is never >>>>>> returned to the system. >>>>>> >>>>>> The script I use to run BaseX is: >>>>>> >>>>>> export BASEX_JVM="-Xmx2048m -XX:MinHeapFreeRatio=10 >>>>>> -XX:MaxHeapFreeRatio=20 >>>>>> -XX:+UseSerialGC -Dorg.basex.LOG=false >>>>>> -Dorg.basex.DBPATH=/var/basex/data >>>>>> -Dorg.basex.REPOPATH=/var/basex/repo" >>>>>> BaseX/bin/basexserver -S >>>>>> >>>>>> So basically I tried specifying MaxHeapFreeRatio and SerialGC for java, >>>>>> but >>>>>> it's no improvement and it doesn't help so I assume the memory isn't >>>>>> hogged >>>>>> in java... is there a way to free up the memory once operations >>>>>> complete >>>>>> (like mentioned above, "complete" means created DB is dropped, >>>>>> connection >>>>>> closed, waiting for another batch to start over). >>>>>> >>>>>> Thanks, >>>>>> Dinu >>>>>>

2 11

Memory requirements, release memory, heap shrink
by Dinu Marina 04 Nov '17

04 Nov '17

Hi guys, I'm running BaseX as server; it has some processing to do every now and then, huge pieces of data. But most of the times it's idle. The "pieces of data" are huge XML files that are ADD'ed to a new DB, then read. Basically I'm using BaseX as an intermediate random-access parser/indexer for some huge XML files. Question 1: XML files are such that I need to specify -Xmx2048M, otherwise I get an out of memory error when ADDing the files. However, I notice that in the BaseX GUI adding the same XML files reports a memory usage of <300M. Is there some special option that the GUI is using that I could use too on the server so that memory usage is not so severe? Question 2: After the data is extracted, it's no longer needed and I DROP the DB; also connection is closed. But memory (the huge 2G mentioned above) is never returned to the system. The script I use to run BaseX is: export BASEX_JVM="-Xmx2048m -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:+UseSerialGC -Dorg.basex.LOG=false -Dorg.basex.DBPATH=/var/basex/data -Dorg.basex.REPOPATH=/var/basex/repo" BaseX/bin/basexserver -S So basically I tried specifying MaxHeapFreeRatio and SerialGC for java, but it's no improvement and it doesn't help so I assume the memory isn't hogged in java... is there a way to free up the memory once operations complete (like mentioned above, "complete" means created DB is dropped, connection closed, waiting for another batch to start over). Thanks, Dinu

2 3

Thanks for Enforce Index Rewriting!
by Omar Siam 03 Nov '17

03 Nov '17

This sounds like a great improvement for our code: http://docs.basex.org/wiki/Indexes#Enforce_Rewritings. Many thanks!

1 0

losses von tags?
by michael4you＠arcor.de 02 Nov '17

02 Nov '17

Hello, There is a problem in transforming data vom XML in HTML. In XML there are HTML-tags included. XML original: <dat>text is <b>important</b>.</dat> HTML goal: <div>text is <b>important</b>.</div> (1) result: <div>text is important.</div> let $data := <dat>text is <b>important</b>.</dat> return <div>{$data/data()}</div> (2) result: <div>text is important.</div> let $data := <dat>text is <b>important</b>.</dat> return <div>{$data/text()}</div> (3) result: <div><dat>text is <b>important</b>.</dat></div> let $data := <dat>text is <b>important</b>.</dat> return <div>{$data}</div> Is there another possibility to get the result desired? Thanking you, Michael

3 2

Re: [basex-talk] Rounding/parsing decimal vs float
by Marco Lettere 01 Nov '17

01 Nov '17

Thanks Kristian, unfortunately this introduces rounding errors which I'm trying to avoid because managing units, measures and monetary amounts. M. Il 01 nov 2017 12:57 PM, "Kristian Kankainen" <kristian(a)keeleleek.ee> ha scritto: You need to use xs:float or xs:double instead of xs:decimal to be able to use the 'e' or 'E' as the exponent separator. Br, Kristian K1. nov 2017 13:18 kirjutas kuupäeval Marco Lettere < m.lettere(a)gmail.com>: > > Hi all, > > I thought of asking this in parallel of hacking my own parsing procedure > ... > > Is there a native way to parse scientific notation string into > xs:decimal since a direct casting (xs:decimal("1e1")) is not allowed? > > Thanks, > > Marco. >

2 2

Re: [basex-talk] Rounding/parsing decimal vs float
by Kristian Kankainen 01 Nov '17

01 Nov '17

You need to use xs:float or xs:double instead of xs:decimal to be able to use the 'e' or 'E' as the exponent separator. Br, Kristian K1. nov 2017 13:18 kirjutas kuupäeval Marco Lettere <m.lettere(a)gmail.com>: > > Hi all, > > I thought of asking this in parallel of hacking my own parsing procedure > ... > > Is there a native way to parse scientific notation string into > xs:decimal since a direct casting (xs:decimal("1e1")) is not allowed? > > Thanks, > > Marco. >

1 0

Rounding/parsing decimal vs float
by Marco Lettere 01 Nov '17

01 Nov '17

Hi all, I thought of asking this in parallel of hacking my own parsing procedure ... Is there a native way to parse scientific notation string into xs:decimal since a direct casting (xs:decimal("1e1")) is not allowed? Thanks, Marco.

1 0

Flexible update via function
by michael4you＠arcor.de 01 Nov '17

01 Nov '17

Hello, How to add/replace a complete node in a node tree, with done by a function and by according function parameters. That means with flexible instructions. Such as: declare function local:NodeAdd($XML, $XMLPath, $XMLValue) { ?? }; let $XML := <root><xml>text</xml></root> let $XMLAdd := <B>BBB</B> let $XMLPath := $XML/xml return local:NodeAdd($XML, $XMLPath , $XMLAdd) Problem: Following examples didn't work, because an internal reference to copy-XML-structure is needed/forced. example 1: copy / modify declare function local:NodeAdd($XML, $XMLPath, $XMLValue) { copy $XML2 := $XML modify ( insert node $XMLValue into $XML2/xml ) return $XML2 }; But: Instead of into $XML2/xml, into $XML/xml is needed, respectively into $XMLPath Thanks a lot Michael

3 3

UnaryLookup expression failure
by Gunther Rademacher 01 Nov '17

01 Nov '17

While testing some XQuery lookup expressions on BaseX 8.6.7, I got a wrong result for this one: (map{'a':(1,3)},map{'a':(2,4)})!?a[2] It should have returned the second item of the 'a' entry for each of the maps, i.e. 3 4 but it returns just 3 Apparently the predicate is applied as if everything before it was in parentheses, i.e. as if it were ((map{'a':(1,3)},map{'a':(2,4)})!?a)[2] The (simplified) parse tree for the original expression and the XQuery 3.1 grammar however indicates that the predicate is part of the mapping target expression: <SimpleMapExpr> <ParenthesizedExpr> ... </ParenthesizedExpr> <TOKEN>!</TOKEN> <PostfixExpr> <UnaryLookup> <TOKEN>?</TOKEN> <NCName>a</NCName> </UnaryLookup> <Predicate> <TOKEN>[</TOKEN> <IntegerLiteral>2</IntegerLiteral> <TOKEN>]</TOKEN> </Predicate> </PostfixExpr> </SimpleMapExpr> Best regards Gunther

2 1

Re: [basex-talk] Peculiar External Binding and Casting Behaviour
by Shaun Flynn 31 Oct '17

31 Oct '17

Hello there, I found the problem! My code added a trailing space to the values of the variables when binding. After tokenizing in the XQuery, this creates a zero length string which when cast to xs:integer then becomes an xs:double according to the error message. Is this intended behaviour? Shaun On 31 Oct 2017 06:20, "Shaun Flynn" <scfy12(a)googlemail.com> wrote: > Hello there, > > I found the problem! My code added a trailing space to the values of the > variables when binding. > > After tokenizing in the XQuery, this creates a zero length string which > when cast to xs:integer then becomes an xs:double according to the error > message. > > Is this intended behaviour? > > Shaun > > On 30 Oct 2017 17:39, "Shaun Flynn" <scfy12(a)googlemail.com> wrote: > >> Hello there, >> >> I created a simple example using the BaseX Java API, and that worked >> fine, also using my code to create the query and submit it to a >> ClientSession worked ok too :( >> >> This was the query I ran: >> >> declare variable $items external := "123 NIL"; >> declare variable $tokenizedItems:= tokenize($items," "); >> >> for $item in $tokenizedItems >> return >> if($item = "NIL") then ("Nil") else (xs:integer($item)) >> >> Unable to reproduce the earlier result from today using my code as well :( >> >> So I can understand the error better: "[FORG0001] Cannot cast to >> xs:double: ." >> >> The full stop after xs:double , does that relate to a decimal point or >> the current item (xquery .) >> >> Shaun >> >> On 30 October 2017 at 10:31, Christian Grün <christian.gruen(a)gmail.com> >> wrote: >> >>> Thanks. Some few lines of Java Code might already be sufficient >>> (provided that the problem can be reproduced with the XQuery snippet >>> that you sent in your last mail). >>> >>> On Mon, Oct 30, 2017 at 11:24 AM, Shaun Flynn <scfy12(a)googlemail.com> >>> wrote: >>> > Hello there, >>> > >>> > I will put something together for you later on today in a .zip file. >>> > >>> > Shaun >>> > >>> > On 30 October 2017 at 10:22, Christian Grün <christian.gruen(a)gmail.com >>> > >>> > wrote: >>> >> >>> >> > However, when I bind variables externally in Java, still getting the >>> >> > same >>> >> > problem, and I definitely not putting quotes around them there. >>> >> >>> >> Do you have an example that might help me to reproduce it? >>> >> >>> >> >>> >> > $depotConstraint: PM >>> >> > $resourceConstraint: 143/6 >>> >> > $lowerDiagConstraint: 251 >>> >> > $upperDiagConstraint: 258 >>> >> > Oct 30, 2017 9:05:33 AM units.forms.UnitDiagramsController >>> >> > fetchSelectedDiagrams >>> >> > SEVERE: null >>> >> > org.basex.core.BaseXException: Stopped at ., 29/61: >>> >> > [FORG0001] Cannot cast to xs:double: . >>> >> > >>> >> > >>> >> > >>> >> > On 30 October 2017 at 09:57, Christian Grün < >>> christian.gruen(a)gmail.com> >>> >> > wrote: >>> >> >> >>> >> >> HiShaun, >>> >> >> >>> >> >> > I do not understand why that should be as there is clearly no >>> mention >>> >> >> > of >>> >> >> > xs:double in this query! >>> >> >> >>> >> >> I agree that the error message is misleading. You already get it >>> when >>> >> >> invoking the following expression: >>> >> >> >>> >> >> 'a' cast as xs:integer >>> >> >> >>> >> >> I need to check the code, but I assume that an additional double >>> >> >> conversion check takes place in the background in order to ensure >>> that >>> >> >> the value can actually be converted to a non-fractional number. >>> >> >> >>> >> >> For more information on your query, it would be great if you could >>> >> >> reduce it to a self-contained example that runs out of the box (I >>> >> >> tried to invoke it, but your code seems to rely on an opened >>> >> >> database). >>> >> >> >>> >> >> Best, >>> >> >> Christian >>> >> > >>> >> > >>> > >>> > >>> >> >>

1 0

Peculiar External Binding and Casting Behaviour
by Shaun Flynn 30 Oct '17

30 Oct '17

Hello there, I am having issues with the following query: declare default element namespace "http://www.tracsis.com/TTVP_Interface"; declare variable $date := "2017-07-10"; declare variable $daysRun := 3; declare variable $diagrams := diagramExchange/unitDiagramList/unitDiagram[@startdate<=$date][@enddate>=$date][substring(@daysrun,$daysRun,1)="Y"]; *declare variable $depotConstraint external;* *declare variable $resConstraint external;* *declare variable $lowerDiagNoConstraint external;* *declare variable $upperDiagNoConstraint external;* *declare variable $depots := tokenize($depotConstraint," ");* *declare variable $resources := tokenize($resConstraint," ");* *declare variable $lowerDiagNo := tokenize($lowerDiagNoConstraint," ");* *declare variable $upperDiagNo := tokenize($upperDiagNoConstraint," ");* declare function local:begin($diagrams, $index) { local:applyDiagStartFilter($diagrams,$index) }; (: Chain of responsibility pattern used here : once one function exits, passes to the next:) declare function local:applyDiagStartFilter($diagrams, $index) { let $diagNo := $lowerDiagNo[$index] return if($diagNo = "NIL") then (local:applyDiagEndFilter($diagrams, $index)) else *(local:applyDiagEndFilter($diagrams[@id >= $diagNo cast as xs:integer]*, $index)) }; declare function local:applyDiagEndFilter($diagrams, $index) { let $diagNo := $upperDiagNo[$index] return if($diagNo= "NIL") then (local:applyResFilter($diagrams, $index)) else (local:applyResFilter($diagrams[@id <= $diagNo cast as xs:integer], $index)) }; declare function local:applyResFilter($diagrams, $index) { let $res := $resources[$index] return if($res = "NIL") then (local:applyDepotFilter($diagrams, $index)) else (local:applyDepotFilter($diagrams[res/@id = $res], $index)) }; declare function local:applyDepotFilter($diagrams, $index) { let $depot := $depots[$index] return if($depot = "NIL") then ($diagrams) else ($diagrams[depot/@id = $depot]) }; for $depot in $depots count $count return local:begin($diagrams, $count) The underlined sections are the ones giving an issue. When I set the following like so: *declare variable $depotConstraint := "PM OC";* *declare variable $resConstraint := "NIL 180/1";* *declare variable $lowerDiagNoConstraint := "NIL NIL";* *declare variable $upperDiagNoConstraint := "NIL NIL";* The query runs fine. When I try and set these externally, I get error "FORG0001 : Cannot cast to xs:double" at the following underlined location: declare function local:applyDiagStartFilter($diagrams, $index) { let $diagNo := $lowerDiagNo[$index] return if($diagNo = "NIL") then (local:applyDiagEndFilter($diagrams, $index)) else *(local:applyDiagEndFilter($diagrams[@id >= $diagNo cast as xs:integer]*, $index)) }; I do not understand why that should be as there is clearly no mention of xs:double in this query! Also, this happens when $diagNo is "NIL" which is even more perplexing as that would suggest it fails the "if" test somehow. The $xxxxConstraint variables are all supplied as strings. What am I doing wrong? I am using BaseX 8.6.7 in Java 8. Shaun

2 1

Using Basex-GUI in Java
by Tushar Deshmukh 29 Oct '17

29 Oct '17

Namaste All, I wants to use Tableview of basex for reporting purpose in my java application. But unable to instantiate. Please help. Bellow is the tried code. public static void main(String[] args) { TableView t = new TableView( ) ; } Can I use this for RDBMS applications also. Regards, Tushar

2 1

Architecture question
by Dave Day 27 Oct '17

27 Oct '17

Greetings list, I have an idea of how I want to use Basex, and would like to know if what I am wanting to do will work. My background is IBM mainframe, so the terminology I use is influenced by that. I have a server running in an address space on a mainframe. Within that address space are multiple tasks active. I believe the term task is equivalent to a thread in the distributed world. I would like to have a task initialize when my server comes up and establish a connection to an instance of a Basex server. This task would be responsible for processing requests that come in from the Basex server. On a browser, I want to have a user connect to an instance of the Basex HTTP server. From the manual, it states that if the HTTP server is started, the client server is started as well. I don't know if this is all one instance of a Basex, or if there are actually two processes that get established. The HTTP server will need to know if the connection has been established between the mainframe and the client server. Is this possible, and if so, how would that be accomplished? Assuming that the user on the browser needs some information from my mainframe server, is there a mechanism in place to facilitate the HTTP server sending the request to the client server, and the client server sending the request to the mainframe. Would the mainframe have to do a poll of some kind to see if there was data, or can the client server push the request out across the connection? If the above works, that is, Basex acting as a message broker between a browser and the mainframe, I want to be able to create another connection from another task on the mainframe to the Basex client server as a result of a user request on a browser, have that 2nd task create a new database, and start pushing xml across that 2nd connection. While that new database is being updated from the mainframe, can the user on the browser query the new database to get the most current data? Regards, -- Dave Day

4 21

Re: [basex-talk] BaseXdb/basex] Packaging: Java and XQuery code (#1413)
by Tim Thompson 26 Oct '17

26 Oct '17

Christian, all, The new "combined" packaging option ( https://github.com/BaseXdb/basex/issues/1413) seems like an interesting enhancement. I have been working on my first BaseX repository module/package (though unfortunately I am not yet fluent in Java). It is working perfectly so far, but I wanted to be sure I was going about it correctly because it still seems a little like magic to me. I am also wondering whether this new feature is something I might want to take advantage of. The module (https://github.com/timathom/basex-rdf) uses Günther Rademacher's REx Parser Generator[1] to create a Java parser for the RDF TriG/Turtle syntax[2], using the BaseX extension function option. Following the BaseX documentation[3], I have created a XAR package that includes the JAR file and an XQuery wrapper. The wrapper calls the function exposed by the Java module (import module namespace trig = "org.basex.modules.rdf.TriG";) and then processes the raw parse tree, overlaying some additional abstractions (I confess that I cheated and used XSLT for most of the processing functionality). Because of the built-in BaseX extension function, this approach seems different from a straightforward "combined" module approach, but, again, I wanted to ask whether the way I'm doing this seems correct. Thanks in advance. Tim [1] http://www.bottlecaps.de/rex/ [2] https://www.w3.org/TR/trig/ [3] docs.basex.org/wiki/Repository -- Tim A. Thompson Discovery Metadata Librarian Yale University Library

3 9

xquery:eval Issue with bindings
by Shaun Flynn 21 Oct '17

21 Oct '17

Hello there, I am trying to use xquery:eval method supplying bindings as map, and following the example on the website where the bindings are supplied as a map. However, when I run the following query, I get the error "Items of type map(*) cannot be atomized". What am I doing wrong? (:Filter out the required diagrams from $diagrams:) declare variable $depotConstraint external := "OC"; declare variable $resConstraint external := "NIL"; declare variable $lowerDiagNoConstraint external := "NIL"; declare variable $upperDiagNoConstraint external := "NIL"; (:Bindings for diagramSelectionCoR.xq:) declare variable $bindings := map { xs:QName("depotConstraint") : $depotConstraint, xs:QName("resConstraint") : $resConstraint, xs:QName("loweDiagNoConstraint") : $lowerDiagNoConstraint, xs:QName("upperDiagNoConstraint") : $upperDiagNoConstraint}; declare variable $diagramSelection := xquery:eval(file:read-text("D:\\XQuery\\VPExchange Queires\\diagramSelectionCoR.xq", $bindings)); $diagramSelection For completeness, the query being loaded in the $diagramExchange "xquery:eval" is: declare default element namespace "http://www.scfy.com/TTVP_Interface"; declare variable $date := "2017-07-10"; declare variable $daysRun := 3; declare variable $diagrams := diagramExchange/unitDiagramList/unitDiagram[@startdate<=$date][@enddate>=$date][substring(@daysrun,$daysRun,1)="Y"]; declare variable $depotConstraint external := "NIL"; declare variable $resConstraint external := "NIL"; declare variable $lowerDiagNoConstraint external := "NIL"; declare variable $upperDiagNoConstraint external := "NIL"; declare variable $depots := tokenize($depotConstraint," "); declare variable $resources := tokenize($resConstraint," "); declare variable $lowerDiagNo := tokenize($lowerDiagNoConstraint," "); declare variable $upperDiagNo := tokenize($upperDiagNoConstraint," "); declare function local:begin($diagrams, $index) { local:applyDiagStartFilter($diagrams,$index) }; (: Chain of responsibility pattern used here : once one function exits, passes to the next:) declare function local:applyDiagStartFilter($diagrams, $index) { let $diagNo := $lowerDiagNo[$index] return if($diagNo = "NIL") then (local:applyDiagEndFilter($diagrams, $index)) else (local:applyDiagEndFilter($diagrams[@id >= $diagNo cast as xs:integer], $index)) }; declare function local:applyDiagEndFilter($diagrams, $index) { let $diagNo := $upperDiagNo[$index] return if($diagNo= "NIL") then (local:applyResFilter($diagrams, $index)) else (local:applyResFilter($diagrams[@id <= $diagNo cast as xs:integer], $index)) }; declare function local:applyResFilter($diagrams, $index) { let $res := $resources[$index] return if($res = "NIL") then (local:applyDepotFilter($diagrams, $index)) else (local:applyDepotFilter($diagrams[res/@id = $res], $index)) }; declare function local:applyDepotFilter($diagrams, $index) { let $depot := $depots[$index] return if($depot = "NIL") then ($diagrams) else ($diagrams[depot/@id = $depot]) }; for $depot in $depots count $count return local:begin($diagrams, $count)

2 1

Re: [basex-talk] Re : FLWOR expression order is lost
by Christian Grün 20 Oct '17

20 Oct '17

> So I should not rely on the order of a sequence (of XML trees) if I do > a path expression on it? The simple map operator (!) is always the best choice if you want to preserve the node order. It would have been more consistent to first define a basic map operator in the language, and extend it with distinct document order semantics later on. I assume that hardly anyone sensed 18 years ago (when XPath 1.0 was finalized) that XPath and XQuery would be used for much more complex things than navigating through single XML documents... > Le vendredi 20 octobre 2017 à 19:02 +0200, Christian Grün a écrit : >> Hi Iwan, >> >> The nodes of a path expression will always be returned in document >> order. See the following query as example: >> >> let $xml := <xml><a>2</a><a>1</a></xml> >> return ( >> '[A]', sort($xml/a), >> '[B]', sort($xml/a) / self::node() >> ) >> >> The result in BaseX is: >> >> [A] >> <a>1</a> >> <a>2</a> >> [B] >> <a>2</a> >> <a>1</a> >> >> While the results of the first sort function will be in the order you >> would expect, the nodes of the second result were reordered. >> >> The order of nodes that occur in distinct XML tree (such as in your >> example) is implementation-defined, so it could happen that another >> query processor returns "1" before "2". However, the reason would not >> be that your sorting was preserved. Instead, it would reflect another >> internal ordering of the elements that you assigned to $data. More >> information on the document order can be found in the spec [1]. >> >> You can use the map operator to get rid of the implicit reordering >> step. In addition, it will be more efficient: >> >> local:nodes() ! @a ! data() >> >> Hope this helps, >> Christian >> >> [1] https://www.w3.org/TR/xquery-31/#id-document-order >> >> >> >> > I noticed that the order of my FLWOR expression is not kept when I >> > do an >> > xpath on the node list: >> > >> > > basex 'declare function local:nodes() {let $data := (<test >> > > a="2"/>, <test >> > > a="1"/>) let $nodes := trace(for $n in $data order by >> > > xs:integer($n/@a/data()) return $n) return $nodes }; >> > > local:nodes()/@a/data()' >> > >> > <test a="1"/> >> > >> > <test a="2"/> >> > >> > 2 >> > >> > 1 >> > >> > >> > I would expect it to return it in order. >> > Is it a bug or or some optimization allowed by XQuery? >> > >> > I tried it on the latest snapshot (BaseX90-20171020.102055.zip). >> > >> > Best regards, >> > Iwan

1 0

FLWOR expression order is lost
by Iwan Briquemont 20 Oct '17

20 Oct '17

Hello, I noticed that the order of my FLWOR expression is not kept when I do an xpath on the node list: > basex 'declare function local:nodes() {let $data := (<test a="2"/>, <test a="1"/>) let $nodes := trace(for $n in $data order by xs:integer($n/@a/data()) return $n) return $nodes }; local:nodes()/@a/data()' <test a="1"/> <test a="2"/> 2 1 I would expect it to return it in order. Is it a bug or or some optimization allowed by XQuery? I tried it on the latest snapshot (BaseX90-20171020.102055.zip). Best regards, Iwan

2 2

Performance issues and query optimizer
by France Baril 20 Oct '17

20 Oct '17

Hi, We are working hard right now on performance issues. I just read this about an upcoming release: Enforce Rewritings In various cases, existing index structures will not be utilized by the query optimizer. This is usually the case if the name of the database is not a static string (e.g., because it is bound to a variable or passed on as argument of a function call). Furthermore, several candidates for index rewritings may exist, and the query optimizer may decide for a rewriting that turns out to be suboptimal. I'm not sure I read it properly so I would like to confirm my understanding: In our code, all our db accesses are in a format like this: db:open($lang)/*[@id='...']. Does the fact that the db name is $lang, a variable, mean that we have been working without the optimizer all this time and therefore that this new feature is there for people like us? Thanks! -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

3 5

ZIP module and ISO-8859-1 filenames
by Andre Bovy 20 Oct '17

20 Oct '17

<DEAR>BaseX</DEAR> I am using the ZIP module for different purpose and until now I didn’t have any problem. Yesterday I received a ZIP file generated by an application and it was impossible to open the file in BaseX. I tried to write a little class opening it in Java. Same behaviour. After different tries, I noticed that the filenames were containing special characters encoded in ISO-8859-1 In my class, I used the java.util.zip.ZipFile method with a charset parameter. It works perfectly. Is there a way to do that with the present ZIP module ? If not, could you please modify the ZIP module to allow the process of ZIP file or ZIP entry with encoded filenames different from UTF-8 Thank you in advance André

2 1

TR: Options for creating database...
by Fabrice ETANCHAUD 20 Oct '17

20 Oct '17

De : Fabrice ETANCHAUD Envoyé : vendredi 20 octobre 2017 11:46 À : 'France Baril' Objet : RE: [basex-talk] Options for creating database... The XQuery will succeed if you remove the INDENT option (serialization), and lower case all your option names. You are right, you may not have a memory overflow issue on creation. But I had the problem with db:add with several huge files, before I switched to BXS. Best regards, Fabrice De : France Baril [mailto:france.baril@architextus.com] Envoyé : vendredi 20 octobre 2017 11:36 À : Fabrice ETANCHAUD Cc : BaseX Objet : Re: [basex-talk] Options for creating database... In db:create I don't think the issue is memory. I get: [bxerr:BASX0002] Unknown database option 'UPDINDEX'. My function is: let $options := map:merge(( map:entry("CHOP", false()), map:entry('INDENT', false()), map:entry('STRIPNS', false()), map:entry('INTPARSE', true()), map:entry('DTD', false()), map:entry('XINCLUDE', false()), map:entry('UPDINDEX', true()), map:entry('AUTOOPTIMIZE', true()) )) return ( db:create($db-name, $src-folder, (), $options), db:output(<div>done</div>) ) On Fri, Oct 20, 2017 at 11:29 AM, Fabrice ETANCHAUD <fetanchaud(a)pch.cerfrance.fr<mailto:fetanchaud@pch.cerfrance.fr>> wrote: Bonjour France, Didn’t you find Updindex and autooptimize options in the ‘Options’ panel of the db creation window ? Db:create last parameter is the place to put all your options : http://docs.basex.org/wiki/Database_Module#db:create But the Pending update list may overflow memory during db creation/update. You should definitively have a look at the BaseX Scripts (BXS). You can declare a batch of commands in XML, and ask BaseX to run it : http://docs.basex.org/wiki/Commands#Command_Scripts This is the way to set options before invoking the CREATE-DB command : http://docs.basex.org/wiki/Commands#SET I usually write XQuery to generate a BXS that will do the job. Cordialement, Fabrice CERFrance Poitou-Charentes De : basex-talk-bounces(a)mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de> [mailto:basex-talk-bounces@mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de>] De la part de France Baril Envoyé : vendredi 20 octobre 2017 10:44 À : BaseX Objet : [basex-talk] Options for creating database... Hi, I usually create dbs using the gui. I'm now trying to create asb without it to be able to set the updindex and autooptimize options, which I can't find in the GUI. I couldn't figure how to set them out using db:create either, so I switch to command line. Now I'm feeling dumb, I can't find how to set up parsing/indexing options for the command line. I'm looking at: http://docs.basex.org/wiki/Command-Line_Options. I found -sindent and -wchop only. What am I missing? Here are all the options that I'd want to set: let $options := map:merge(( map:entry("CHOP", false()), map:entry('INDENT', false()), map:entry('STRIPNS', false()), map:entry('INTPARSE', true()), map:entry('DTD', false()), map:entry('XINCLUDE', false()), map:entry('UPDINDEX', true()), map:entry('AUTOOPTIMIZE', true()) )) I don't want to set UPDINDEX and AUTOOPTIMIZE in .basex because I only want them to be true() on one of my DBs. Other DBs should remain as is. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com<mailto:france.baril@architextus.com> -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com<mailto:france.baril@architextus.com>

2 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk