- BaseX-Talk - mailman.uni-konstanz.de

What's wrong here?
by Marco Lettere 01 Feb '17

01 Feb '17

Hello all, when running the following query: hof:until( function($i){ $i = 10}, function($i){ (admin:write-log(xs:string(current-dateTime()) || " -- " || $i), prof:sleep(1000), $i + 1)}, 0 ) I get the following output: 19:49:07.400 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 0 19:49:08.400 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 1 19:49:09.401 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 2 19:49:10.402 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 3 19:49:11.403 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 4 19:49:12.404 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 5 19:49:13.405 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 6 19:49:14.406 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 7 19:49:15.407 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 8 19:49:16.408 SERVER admin INFO 2017-02-01T19:49:07.399+01:00 -- 9 I would expect current-dateTime() to return a dateTime that increases by one second on every output. What is wrong with my understanding? Thanks, Marco.

2 2

XQuery performance degradation as number of json files grows
by Baysdon, Joseph 31 Jan '17

31 Jan '17

The performance of XQuery updates degrades as the number of JSON resources in my BaseX instance grows. In what ways might the number of JSON resources affect the performance of an XQuery update for a particular XML resource? I was under the impression that JSON resource were not indexed. (Note: this particular performance degradation is most dramatic when the BaseX server points to an NFS mount for the BaseX database.) Thanks, Joe

2 3

Is it possible to use indices in sub-queries
by Clemens Müthing 31 Jan '17

31 Jan '17

Hello everyone I’ve recently started to use and stumbled upon a problem with the indices in basex that I do not fully understand and was hoping for some clarification. So basically I’ve got a bunch of xml files in my database with medical ICD-10 codes that all look like this: <classification name=„some-name“ year=„2017“ > <code> <value>03R80JZ</value> <final>true</final> </code> <code> <value>2W3JX1Z</value> <final>false</final> </code> … ... </classification> The db can contain multiple of these files, one for each name and year, and each classification can contain up to 100k of those codes (all values are unique). What I’d like to do now is, given a list of codes, check if those codes actually exist in the classification for a given name and source. So I came up with this straight-forward piece of code: let $proc := ('F0723EZ','0JWS37Z','0W054JZ','0NQ00ZZ','03R80JZ','0PW537Z','0RW940Z','0PPD0KZ','2W3JX1Z','0D9B8ZZ','0SWC30Z','0PRJ4KZ','025R0ZZ','0T1347C','0S950ZX','008J4ZZ','0D110K4','0XUV4JZ','0GTN4ZZ','4A133J1’) for $p in $proc return /classification[@name=‚some-name‘ and @year=2017]/code[value = $p] This does work fine, however it takes roughly one second to evaluate the query, and in the Query Info Window I can see that the text index of the db is not being used. However I do see that the attribute index is being queried. Now I noticed that, if I drop the attribute index, the query is evaluated basically instantaneous and can see that the query is rewritten to use the text index as well. So my question is: Is this normal, i.e. is it not possible to use multiple indices in one query at once? Or is this a problem where the „sub-query“ …./code[value = $p] is only evaluated on the result of the first query, thus the text index is not being considered? If so, is there a way (without rewriting the whole query) to make basex use both indices when compiling the query? Thanks a lot already and all the best Clemens Müthing

2 4

Send multipart binary with http client
by Maximilian Gärber 30 Jan '17

30 Jan '17

Hi, I can't figure out how to send a file in binary (not base64) when using multipart [1] On the body, I've tried method="basex" because the docs says this would write raw bytes to the request Regards, Max [1] declare function _:send-multipart($url, $file-name, $path) as item()* { let $uri := file:path-to-uri($path || $file-name) let $type := fetch:content-type($uri) let $binary := file:read-binary($path || $file-name) let $boundary_id := concat("batch_", random:uuid()) let $req := <http:request method="post" send-authorization="false" override-media-type="text/plain"> <http:header name="Accept" value="application/json"/> <http:header name="Accept-Encoding" value="gzip, deflate"/> <http:multipart media-type="multipart/form-data" boundary="{$boundary_id}"> <http:header name="Content-Transfer-Encoding" value="binary"/> <http:header name="Content-Disposition" value="form-data; name='FILE'; filename='{$file-name}'"/> <http:header name="Content-Type" value="{$type}"/>  <http:body media-type="application/octet-stream" method="binary"> {$binary} </http:body> </http:multipart> </http:request> let $resp := http:send-request($req, $url) return json:parse($resp[2]) };

1 1

Whitespace
by meumapple 30 Jan '17

30 Jan '17

Hi, I have a problem with whitespace handling of the default BaseX XML parser. If I have: <s>and <a>this</a> is</s> the parser deletes the spaces after "and" and before "is". Why not preserving the space as default option here? This kind of space is very important. I know that with adding xml:space I can solve the problem, but this is not easy to do automatically (moreover I cannot do this with copy/modify because all spaces are deleted after adding xm:space ). Is there a parameter which forces the parser to keep the spaces, without modifying the file? And in general, can the behavior of the parser be changed?

3 3

BaseX 8.6: The Winter Edition
by Christian Grün 30 Jan '17

30 Jan '17

Dear all, BaseX 8.6 is out! The new version of our XML database system and XQuery 3.1 processor includes countless improvements and optimizations. Many of them have been triggered by your valuable feedback, many others are the result of BaseX used in productive and commercial environments. The most prominent new features are: LOCKING - jobs without database access will never be locked - read transactions are now favored (adjustable via FAIRLOCK) RESTXQ - file monitoring was improved (adjustable via PARSERESTXQ) - authentication was reintroduced (no passwords anymore in web.xml) - id session attributes will show up in log data DBA - always accessible, even if job queue is full - pagination of table results INDEXING - path index improved: distinct values storage for numeric types XQUERY - aligned with latest version of XQuery 3.1 - updated functions: map:find, map:merge, fn:sort, array:sort, ... - enhancements in User, Process, Jobs, REST and Database Module CSV DATA - improved import/export compatibility with Excel data Visit http://basex.org to find the latest release, and check out http://docs.basex.org/ to get more information. As always, we are looking forward to your feedback. Enjoy! Christian BaseX Team

5 7

Counting
by meumapple 29 Jan '17

29 Jan '17

Hi, I have a structure like this: <c> <a>red</a> <a>red</a> <a>blue</a> <a>green</a> <a>green</a> </c> And I would like to get: <c> <a n="1">red</a> <a n="2">red</a> <a n="1">blue</a> <a n="1">green</a> <a n="2">green</a> </c> I have tried with two for loops after having identified the distinct-values(). This is in principle correct but it takes too long (I have many distinct values and a elements), because I cannot exit the for loop as soon as I get a match. I have been able to get the result with the window clause and then counting the elements of the window with a nested for-loop: is there an alternative solution? Thanks. Joseph

5 10

Importing XML using Java Classes vs Direct Commands
by Shaun Flynn 27 Jan '17

27 Jan '17

Hello there, I am attempting to create a database and add a resource to it programatically in Java. If I use this code: public static void main(String[] args) { try (ClientSession session = new ClientSession("localhost", 1984, "admin", "admin")) { session.execute("SET INTPARSE true"); session.execute("CREATE DB TESTING D:\\VP DB\\export.xml"); session.execute("CLOSE"); session.close(); } catch (IOException ex) { Logger.getLogger(BaseXTesting.class.getName()).log(Level.SEVERE, null, ex); } The code run's without a problem. When I attempt to do this: public static void main(String[] args) { // TODO code application logic here CreateDB createDBCmd = new CreateDB("TESTING"); Open openCmd = new Open("TESTING"); Add addCmd = new Add("\\BaseXTarget", "D:\\VP DB\\export.xml"); Set setCmd = new Set("INTPARSE", true); try (ClientSession session = new ClientSession("localhost", 1984, "admin", "admin")) { session.execute(setCmd); session.execute(openCmd); session.execute(addCmd); session.close(); } catch (IOException ex) { Logger.getLogger(BaseXTesting.class.getName()).log(Level.SEVERE, null, ex); } The following exception is thrown: run: Jan 26, 2017 5:03:17 PM basextesting.BaseXTesting main SEVERE: null org.basex.core.BaseXException: "BaseXTarget.xml" (Line 1): No text allowed before root element. at org.basex.api.client.ClientSession.receive(ClientSession.java:191) at org.basex.api.client.ClientSession.execute(ClientSession.java:160) at org.basex.api.client.ClientSession.execute(ClientSession.java:165) at org.basex.api.client.Session.execute(Session.java:36) at basextesting.BaseXTesting.main(BaseXTesting.java:36) BUILD SUCCESSFUL (total time: 0 seconds) It could be me being daft, but both pieces of code should achieve the same thing as they point to the same file, but when classes are used instead of command strings, the BaseXException is thrown. I have checked my data for the byte order mark, and have the ensure it is enoced using UTF-8. Help! Shaun

2 1

Any consideration given to something like XLP?
by Mike Engledew 26 Jan '17

26 Jan '17

Hello, I have been giving some thought to concurrency and rollback/recovery as we continue to evolve our Java EE application that uses BaseX as its XQuery, (XQuery update facility) engine. >From reading the BaseX Transactions section I concluded that BaseX locking is done at the database level. I can imagine this could ultimately become the major bottleneck in a "reactive cloud" application, has anyone considered the value of implementing a more granular locking mechanism like XLP (Xpath Locking Protocol) on which several academic papers have been written? Are there any similar ideas on the horizon for transactional commit/rollback for XML data? Thanks to the group (and Christian in particular) for such great help, advice and support. Mike.

2 1

Memory management
by meumapple 24 Jan '17

24 Jan '17

I have 223 XML files and I need to perform some actions on each of them, so that the output should be the modified texts (223 new files). What I have noticed is that if I apply my script recursively to each file and then save the results in new files (file:write()), everything works fast. On the other hand, If I do not write out the results in new files (but I expect the results from all the files in the GUI), the GUI freezes and then runs out of memory. My question is: is there a way to get the script working well without using file:write()? Should I always apply this script strategy when I work with many files? Thanks, Joseph

2 5