- BaseX-Talk - mailman.uni-konstanz.de

fn:trace and http-server, why is it not logged?
by Marc van Grootel 10 Aug '14

10 Aug '14

Hi, I used fn:trace in code and I made sure it is not optimized away. Saw the trace msg in the gui. But when doing it via the http server only the standard request info is shown in the log 18:05:47.810 0:0:0:0:0:0:0:1:50364 admin REQUEST [GET] http://localhost:8984/ 18:05:49.234 0:0:0:0:0:0:0:1:50364 admin 200 REQUEST 1463.52 ms 18:05:49.235 0:0:0:0:0:0:0:1:50364 admin 200 0.32 ms The docs say that trace is sent to stderr, isn't that logged? Is there a way to turn the logging volume for http server logging up a notch or is there some other way to get the trace messages logged? -- --Marc

2 2

BaseX & XQuery 3.1: Arrays
by Christian Grün 09 Aug '14

09 Aug '14

Dear all, As you may have heard, XQuery 3.1 will not only provide maps (which have been around in BaseX for quite a while now) but also arrays... And we are pleased to announce that, since today, arrays are available in BaseX! Some of the new features are documented in our Wiki: http://docs.basex.org/wiki/XQuery_3.1 http://docs.basex.org/wiki/Array_Module Please note that the XQuery 3.1 spec. is at an early stage and subject to change, but the implementation is already fully functional, and we are looking forward to receiving your first feedback. The latest 8.0 snapshot is now available at: http://files.basex.org/releases/latest/ Have fun, Christian

1 0

unexpected behaviour xquery:update()
by Rob Stapper 09 Aug '14

09 Aug '14

Hi, See attachment: "XML update example 2.sq" What happens to the output of this program? I don't see any results in the GUI. Rob --- Dit e-mailbericht bevat geen virussen en malware omdat avast! Antivirus-bescherming actief is. http://www.avast.com

2 1

running in eclipse
by Andy Bunce 09 Aug '14

09 Aug '14

Hi, I am having problems running the source. I have download the zip from github, and added as a java project in eclipse. When I run I get... /lang/English.lang not found. English.lang: 'srv_started_port_%' is missing English.lang: 'srv_stopped_port_%' is missing English.lang: 'srv_running' is missing What am I doing wrong? /Andy

3 5

CSV : Content is not allowed in prolog.
by Roman Mishin 09 Aug '14

09 Aug '14

Hi, Could you, please, tell what can be wrong here. This query works: db:open("IntersectPhoneNumbers","Data/rent_flats.csv")/csv/record/UserPhone With result: <UserPhone>2218520</UserPhone> <UserPhone>2219391</UserPhone> <UserPhone>2348039</UserPhone> ... I can access the file directly from the query, without adding it into a database. This also does work, giving the same result: let $csv := file:read-text(fn:resolve-uri('Data/rent_flats.csv')) let $rent_flats := csv:parse($csv, { 'header': true(), 'separator': 'tab' }) return $rent_flats/csv/record/UserPhone But this does not work: doc("Data/rent_flats.csv")/csv/record/UserPhone Error: [FODC0002] "C:/Projects/IntersectPhoneNumbers/Data/rent_flats.csv" (Line 1): Content is not allowed in prolog. Though I can browse the file from BaseX GUI. It is in DB. Selecting nodes in Folder Pane for that file, shows the actual data in Result Pane. I can also access all records from all files in Data folder, including rent_flats.csv: /csv/record/UserPhone But I wish to refer to rent_flats.csv only. Why is this error? I am using BaseX 7.9 on Windows XP SP3. -- Roman

2 3

map:for-each-entry missing from latest 8.0
by Marc van Grootel 08 Aug '14

08 Aug '14

Hi, I intended to use the 8.0 documented function map:for-each-entry but it doesn't exist (tried latest snapshot). So I thought I open modules.zip to look at the code. And, indeed it wasn't there. Second thing I noticed was a typo in the namespace. http://www.w3.org/2005/xpath-functsions/map this should be: http://www.w3.org/2005/xpath-functions/map Let me know if you need me to log it somewhere. --Marc

2 4

How to update a XML-file from itself?
by Rob Stapper 08 Aug '14

08 Aug '14

Hi, Can anyone help me with this casus? I want to update XML-source: $a, by inserting the content of XML-element: $b, a sub-element of $a, into XML-element: $c, also a sub-element of $a. The relative path from $b to $c is known. See attached example: the problem is that the update works on a copy of $a and that the destination-element: $b/parent::*/default/text1, is actually an element of the original $a. How can I make it point to the corresponding element in the copy? Any suggestions? Thanx in advance, Rob Stapper --- Dit e-mailbericht bevat geen virussen en malware omdat avast! Antivirus-bescherming actief is. http://www.avast.com

2 4

serialisation before xslt
by Marc 07 Aug '14

07 Aug '14

Hi, I want to transform with xslt before to output the file. If I do a serialize on the disk, runs the xslt with saxon et write the result the space are conserved (I use chop=false to load the file in the base). But if I transform the file in BaseX, and serialize the file after, the element are passed to the xslt indented and I lost the space at the begining and the end of element. And other thing, if I put the "declare option db:chop 'false';" at the beginning of the XQuery, the file is indented, if I put it out (it's unuseful) the serialisation without indentation is done. I use the 7.9 version.

2 3

Lower level synching
by Marco Lettere 07 Aug '14

07 Aug '14

Hello to everyone who is still there even in deep Summer! I was just wondering whether anyone had experiences with synchronizing two basex databases (same server or not) at a lower level for example using (r)sync or nfs or even Hdfs. Would it be feasible? Thanks, Marco.

2 1

Different behaviour on exception
by Marco Lettere 07 Aug '14

07 Aug '14

Hi all, I have the following code [1] that I expect to raise a xml parser exception but it raises a typecast exception [2]. Instead I get the expected exception [3] when substituting the call to local:h with direct parse -xml() (as for the snippet in the comment). Any clue? Thanks, Marco. [1] declare function local:f($rdf as node()){ $rdf }; declare function local:h(){ parse-xml("xxx") }; declare function local:g(){ let $rdf := local:h() (: parse-xml("xxx") :) let $validate := local:f($rdf) (:$rdf:) return ($validate) }; local:g() [2] Stopped at /home/lettere/tmp/basex80/basex/file, 2/18: [XPTY0004] Cannot cast from empty-sequence() to node(). [3] Stopped at /home/lettere/tmp/basex80/basex/file, 11/26: [FODC0006] SAX: "file:/home/lettere/tmp/basex80/basex/file" (Line 1): Content is not allowed in prolog.

2 1

Unit module sometimes drives me nuts
by Marc van Grootel 07 Aug '14

07 Aug '14

Really, I want to be good and write my tests together with the code. I still don't do it often enough but I know it's important and for the code I'm currently working on I try to cover most of it with tests. I regret that I'm not able to get down into the guts of the Java code and help fix it, so, instead I moan ;-) No, seriously, here are a few more general things that bug me about the Unit module. - assertions causing an NPE when an XQuery error is raised but no traceback that tells me what/where the error actually is. [ex1] - assertions causing a failure/error reported, produce a test result but stop running further tests after the one that raises the error [ex2] I tried to reduce these issues into a short example but I think that they highlight a more general issue with the current Unit module. It's mainly the fact that there's no indication of the problem that forces me to add unit:ignore to all tests and one by one removing it, or executing the assertion xquery in a client to find out where the issue lies. I also have an improvement idea: - Personally, I run the tests often (minutes rather than hours), and from the shell. The XML format feedback isn't very helpful. It would be for automated test reports but I think the usability (readability) of the feedback during development is just as, if not more, important. Maybe adding an extra switch (-t for textual feedback and -tx for XML feedback) could help. It should just focus on summarizing clearly the expected, return values, failures and errors, and provide helpful context. Note that I always run the code using `basex -t` from bash and I'm running 8.0 beta 7ba9510 Note too that most of these things happen to me when doing map related stuff. ... and now I stop moaning and get back to coding. --Marc [ex1] example of test that causes an NPE without any helpful output *declare* *%unit:test* *function* *test:npe*() { *unit:assert-equals*(*map* {}('x'), *map* { 'x': 1 }('x')) }; [ex2] example of test that causes further tests to be executed *declare* *%unit:test* *function* *test:fail*() { *unit:assert-equals*(*map* {}, *map* { 'x': 1 }) }; doing map {} = map {'x': 1} causes this error [XPTY0004] map(*) and map(*) cannot be compared.

2 4

Same query, huge difference in performance
by Paul Swennenhuis 05 Aug '14

05 Aug '14

Hi, I have two XQueries that do exactly the same thing, i.e. create a list of waters in the Facts database and display where these waters flow into, if any. But Query 1 executes 10 times faster than Query 2 (in the GUI). The only difference is in the selection of the query context. Query 1 assumes the Facts database is open and does not use an explicit doc() or collection() context. Query 2 uses an explicit collection("Facts") context Query 1 executes in appr. 600 msecs, Query 2 in appr. 60 msecs (!) How can there be such a huge difference? Is it because Query 2 operates twice on the $facts variable and does not need to evaluate that again? Listings: Query1 (: list waters and where they stream to (if any):) for $source in //(sea|river|lake) let $toId := $source/to/@water let $to := (//sea|//river|//lake)[@id=$toId][1] let $name := if (empty($to/local-name())) then "none" else $to/local-name() return element water { element {$source/local-name()} {data($source/@name)}, if (not($name="none"))then element streamsTo { attribute {$name} {data($to/@name)} } else () } Query2 (: list waters and where they stream to (if any):) let $facts := collection("Facts")//(sea|river|lake ) for $source in $facts let $toId := $source/to/@water let $to := $facts[@id=$toId][1] let $name := if (empty($to/local-name())) then "none" else $to/local-name() return element water { element {$source/local-name()} {data($source/@name)}, if (not($name="none"))then element streamsTo { attribute {$name} {data($to/@name)} } else () } As a side-question: I want to extend the query to make it recursive: river "Bahr el-Djebel" streams into river "White Nile" streams into river "Nile" streams into sea "Mediterranean Sea" I think I can find out how to do that, but how can I optimize the recursion process? Would a recursive function be efficient? Thanks, Paul

4 12

UPDINDEX and ever growing index size
by James Ball 04 Aug '14

04 Aug '14

Hello, I’ve finally had some time to look at an issue I’ve been having with databases that have UPDINDEX set to true. I’m now running a BaseX 8.0 beta and was on 7.X when I first encountered this. The issue I’m seeing is that the size of the index grows by approximately 1MB with every updating ‘transaction’ (snapshot?) even if there is no new data for the index. For example if I have a database with 100,000 files and I replace one of those files (with itself so there’s no new data) then the size of the index will go up by around 1MB. If I replace 1000 files in the same transaction (again with themselves) the size of the index will go up again by around 1MB. Dropping and recreating the index returns it to its original size. I have a current project where I’m expecting thousands of files a few at time that need to be added/replaced - I completely ran out of disk space before I spotted what was happening when testing. Is this expected behaviour? I don’t know the format for the index files but I’ve looked at atvl.basex just in a text editor. It looks like for each update to the index around 40k blank lines are being added. I don’t know that they are truly blank lines - but that’s how they’re rendering in the editor. I’ve created a small test case to replicate what I’m seeing. [Mac OS 10.9.4, BaseX 8.0 beta 496c381] Thank you for your help. Regards, James 1) SET UPDINDEX TRUE 2) CREATE DB Index-Test-Updindex-XQ 3) Run an XQuery to populate a reasonable database (I do 10,000 items) ------------------------ let $files_from := 1 let $files := 10000 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 4) Check the size of the index - should be about 325kB 5) Run the XQuery again (it will replace files with identical copies) but for just one file: ------------------------ let $files_from := 1 let $files := 1 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 6) Check the size of the index - it will be about 1MB 7) Run the XQuery again for around 100 files ------------------------ let $files_from := 1 let $files := 100 let $xml := <XmlBody DocumentType="Test" DocumentCode="" TimeStamp="2014-07-14T10:57:34."> <DocumentInfo> <Name Code="54321" Value="Name"/> </DocumentInfo> <DataItems> <DataItem Code="12345" Value="Data"/> </DataItems> </XmlBody> for $i in ($files_from to ($files_from + $files -1)) let $d := copy $c := $xml modify ( replace value of node $c/@DocumentCode with $i, replace value of node $c/@TimeStamp with $c/@TimeStamp||$i, replace value of node $c/DocumentInfo/Name/@Code with $c/DocumentInfo/Name/@Code||$i ) return $c return db:replace('Index-Test-Updindex-XQ','Test/'||$i,$d) ------------------------ 8) Check the size of the index - it will be about 2MB. 9) Drop the index and recreate it. It will be about 325kB again. --------------------------------- James Ball me(a)jamesball.co.uk

2 11

Question about the mailing-list
by Paul Swennenhuis 04 Aug '14

04 Aug '14

A bit off-topic but this weekend I ran into a strange issue with this mailinglist. I tried to send a message from the same e-mail address I am using now (paul(a)swennenhuis.nl) but it bounced back every time I tried. I get the feeling that the subscription is not merely e-mail based but also IP address based? Even weirder: the rejected mails were sent to another e-mail address of mine, p.a.m.swennenhuis(a)gmail.com I can't remember even having subscribed with that address, not does it appear in the subscribers list of the mailing list. Paul

2 1

Copy between databases
by Marco Lettere 04 Aug '14

04 Aug '14

3 3

Réponse automatique
by mab 01 Aug '14

01 Aug '14

1 0

Re: [basex-talk] XML-Validation return value
by Martin Hartmann 30 Jul '14

30 Jul '14

2 1

Cannot instantiate java.net.URLEncoder
by Marc van Grootel 30 Jul '14

30 Jul '14

Hi, What could be the reason that importing java.net.URLDecoder works as expected but doing the same with java.net.URLEncoder doesn't. The following works: import module namespace decoder = "java.net.URLDecoder"; decoder:decode('foo/bar%20baz') => 'foo/bar baz' This gives a "Could not instantiate module 'java.net.URLEncoder' import module namespace encoder = "java.net.URLEncoder"; encoder:encode('foo/bar baz') --Marc

2 3

Re: [basex-talk] How to create xml file dynamically and store into database directly without storing into my disk?
by Dirk Kirsten 30 Jul '14

30 Jul '14

Hello Manish, please always reply to the list, so others can also benefit from questions and answers. String query = "db:add("db","xmlFile")"; will certainly not work, this will throw a java error due to the ". This should work: String query = "db:add('db','xmlFile')"; As I already suggested (multiple times) on StackOverflow, you will have to build and modify the document within BaseX using XQuery Update. This way, the XML file can be kept in main memory and later be added to the database. Please read up in our documentation about XQuery Updates. Cheers, Dirk On 30/07/14 11:37, manish payasi wrote: > Thanks Dirk, > > I am trying to write like > String query = "db:add("db","xmlFile")"; where xmlFile is string > variable which has xml file name, > but it is throwing some syntax error (like Syntax error on tokens, delete > these tokens for both the arguments) . > > Another thing is that in second argument we are passing the xml file which > is still coming from disk which we don't want it rather than i want create > the xml file dynamically in java and store this file directly into database. > > Thanks and Regards > manish payasi. > > > > On Tue, Jul 29, 2014 at 6:29 PM, Dirk Kirsten <dk(a)basex.org> wrote: > >> Hi Manish, >> >> You might want to take a look at our java examples here : >> https://docs.basex.org/wiki/Java_Examples >> >> So to simply execute an XQuery and using db:add() (see >> https://docs.basex.org/wiki/Database_Module#db:add) to add to the >> database would look something like that: >> >> Context context = new Context(); >> String query = "db:add("your-database", <xml>here goes your >> xml</xml>, "myfile.xml") >> QueryProcessor proc = new QueryProcessor(query, context); >> proc.execute(); >> proc.close(); >> context.close(); >> >> the second argument of db:add() is an item(), so you can also >> dynamically generate an XML element here. >> >> Cheers, >> Dirk >> >> On 29/07/14 14:46, manish payasi wrote: >>> How to create xml file dynamically in java and store into BaseX database >>> 'db'. That means it shouldn't not be created into my disk(my system) but >> it >>> should be directly created(store) into database 'db' ????? >>> >>> Please give me some better solution. >>> >> >> -- >> Dirk Kirsten, BaseX GmbH, http://basex.org >> |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz >> |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: >> | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle >> `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22 >> > -- Dirk Kirsten, BaseX GmbH, http://basex.org |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 28 28 676, Fax: 0049 7531 20 05 22

1 0

How to create xml file dynamically and store into database directly without storing into my disk?
by manish payasi 29 Jul '14

29 Jul '14

How to create xml file dynamically in java and store into BaseX database 'db'. That means it shouldn't not be created into my disk(my system) but it should be directly created(store) into database 'db' ????? Please give me some better solution.

2 1

How to store dynamically created xml file in basex through java
by manish payasi 25 Jul '14

25 Jul '14

Hi guys, I am new in base x and i got some problem so i would like to ask some questions because i didn't get the solution from anywhere and someone recommended me to raise the questions here. I know the way to create the xml file and store into disk after that we can store these xml files into basex database 'db' after went through official BaseX tutorial . I have attached my two class file through which i am creating and storing xml file into BaseX database,but it's not a feasible solution. So my question is: *How to create xml file dynamically and store into database directly rather than storing into my disk?????* Please revert back to me as soon as possible. Second question i'll ask in another mail. Thanks and Regards manish payasi

2 1

How to optimize "insertion intensive" usage patterns
by Marco Lettere 25 Jul '14

25 Jul '14

Hello all, for the first time I encountered a scenario which runs according to an "insertion intensive" pattern. A lot of documents are stored into a database at a rather fast pace. With the computing resources I currently have, the process slows down rather quickly since indexes (created with the "create index" command) got invalidated immediately. I moved to the UPDINDEX option and this alleviated the slowing down but made my "small" system run quickly into disk space issues since the data files grow at a much higher rate (5.8 GB after one day of continuous work). I can fix this by running the optimize all command. I was not very comfortable with this command so I stopped everything and made a cold run. The result was surprisingly good. The amount of disk space got reduced to 1% of the size it had before the optimize and it also took a small amount of time compared to the dimension of the input data. My first question is whether running optimize command regularly (maybe every hour or so) without stopping the flow of incoming documents is a safe way to go or whether it will impede somehow the workload or introduce any risks on data integrity. Secondly, to improve the workload on a single database (thus reducing the frequency of optimize calls), I was planning to split my data moving less important data to a different database (but same server). Will this be an effective solution in this scenario? Sorry for the possibly naive questions but I'm really not a very expert in this quantitative aspects. Thanks and regards, Marco.

2 4

RESTXQ / xml-stylesheet
by Fabrice Etanchaud 23 Jul '14

23 Jul '14

Dear all, Could you please telle me how I could add a xml-stylesheet processing instruction to a restxq function ? Best regards, Fabrice Questel/Orbit

2 1

flush the pending list
by Marc 23 Jul '14

23 Jul '14

Hi, I have a XQuery who does a lot of insertion in documents in the same collection, but on by one. In an other point of vue, I have a collection that I modify and others collections that I read. We the pending update list, all the modifications stays in memory before to be write and in my case that's a lot of memory. Is there a way to flush the pending update list because I know that there is no reverse action on it. Without this I do one query for each updated collection documents. Marc

2 2

client module
by Fabrice Etanchaud 22 Jul '14

22 Jul '14

Dear all, The client:query function seems to systematically trim text nodes For example, xml ingested with chop = false will be chopped when retrieved with client:query(). The client:execute function is ok. Best regards, Fabrice Questel/Orbit

2 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk