- BaseX-Talk - mailman.uni-konstanz.de

Xquery update question
by Erol Akarsu 06 Jan '15

06 Jan '15

Hi, I have several xml document index keeping products features and have one master index file of products.. Currently, we have about 25 million products that have size about 25G xquery index size. My xquery application is reading features data from auxiliary xml document and enhancing master document. In order to do it, I am inserting data into master document with a generic insert command: insert node <FEATURES_NODE> into <PRODUCT_NODE> But xquery update is executing update process after all insert requests collected. But with huge size of my index, the memory of machine (35G) is exhausted easily and can NOT finish process. Is there way to execute individual xquery update requests one by one so that process can finish? Erol Akarsu

2 2

Fetching a binary document from BaseX with C# using the server API
by Boxstart, Bjorn 06 Jan '15

06 Jan '15

Hello, I'm currently working on the integration of BaseX in part of our .Net based software. One part of the functionality will be to store and retrieve binary files. I haven't been able to retrieve the raw files from the server yet using C#. I based my solution on the example of the C# client in the source code repository of BaseX. With my current solution I receive only the first few (3 to 7) bytes of the file (at least I think they are...). The code I have at the moment is: /// <summary> /// Retrieves a RAW document from the database. The database must be opened before executing this command. /// </summary> /// <param name="path">The path to the binary data to be retrieved</param> /// <returns>A byte array with the binary data</returns> public byte[] Retrieve(string path) { //stream.WriteByte((byte)BaseXCommand.QUERY); Send("RETRIEVE " + path); //Send(path); using (MemoryStream ms = new MemoryStream()) { Receive(ms); info = Receive(); if (!Ok()) { throw new IOException(info); } return ms.ToArray(); } } Can anybody please tell me what the problem with this solution can be or give me a hint for a correct way to retrieve the raw bytes? I know I can use the files on disk directly, but would rather not go that way. Thanks in advance for your reaction! Kind regards, Björn Boxstart

2 2

Question / Suggestion : Template mechanisms and operator overloading for XQUERY 3.++ ?
by jean-marc Mercier 06 Jan '15

06 Jan '15

Hello, I was wondering if it is possible to improve template programming and operator overloading with XQUERY 3.? 1) template programming : XQUERY provides a quite natural template mechanism using type switch and instance: declare function yell($animal) { if ($animal instance of element()) then return typeswitch($animal) case element(cow) return "Moo" default return "..." else "this is not an animal" }; However, I wonder whether it could be possible to provide such a mechanism directly at the interpreter level ? For instance, the following code seems coherent and concise : declare function yell($animal as element(cow) ) {"Moo"} declare function yell($animal as element() ) {".."} declare function yell($animal ) {"This is not an animal"} As far as I understand, XQUERY functions identification seems to be achieve through a couple (name, arity). Would it be difficult to extend it with (name,signature or predicate) ? 2) Operator overloading (quite linked to the previous question) I don't know how to overload operators with XQUERY. For instance I would like to define something like declare function array:operator+($left,$right){array:for -each-pair($left,$right,function($a + $b) { $a + $b } )} Is it possible ? I know that I can survive writing declare function array:plus($left,$right){array:for-each-pair($left,$right, function($a + $b) { $a + $b } )} but when one start to write algorithmic code, it is far easier and understandable to write $a+$b than array:plus($a,$b)

2 1

copy-namespaces declaration
by Alex R. Loayza 06 Jan '15

06 Jan '15

Hello, I've found this same issue on version 7.9: https://mailman.uni-konstanz.de/pipermail/basex-talk/2012-May/003294.html Is there now some fix I'm not aware of? Thank you. Alex

2 1

Out Of Memory
by Mansi Sheth 06 Jan '15

06 Jan '15

Hello, I have a use case, where I have to extract lots in information from each XML in each DB. Something like, attribute values of most of the nodes in an XML. For such, queries based goes Out Of Memory with below exception. I am giving it ~12GB of RAM on i7 processor. Well I can't complain here since I am most definitely asking for loads of data, but is there any way I can get these kinds of data successfully ? mansi-veracode:BigData mansiadmin$ ~/Downloads/basex/bin/basexhttp BaseX 8.0 beta b45c1e2 [Server] Server was started (port: 1984) HTTP Server was started (port: 8984) Exception in thread "qtp2068921630-18" java.lang.OutOfMemoryError: Java heap space at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1857) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2073) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:744) -- - Mansi

5 17

Python client and BaseX 8.0
by Marc van Grootel 06 Jan '15

06 Jan '15

Hi, Anyone using the python client. I used it before, I think it was with 7.9, where it was still working. But now on a recent snapshot I cannot get my example to work anymore. I will report to maintainer but I would like to know if anyone could confirm having difficulties too. Anything changed in the protocol? I'm running on Python 2.7.8 :: Anaconda 2.0.1 (x86_64), on Mac that is. The error message occurs when trying to set up a session: /Users/marcvangrootel/Dropbox/notebooks/BaseXClient.py in __init__(self, host, port, user, password, receive_bytes_encoding, send_bytes_encoding) 129 # evaluate success flag 130 if not self.server_response_success(): --> 131 raise IOError('Access Denied.') 132 133 def execute(self, com): IOError: Access Denied. Thanks, --Marc

4 9

What's the best way to do some qualitative benchmarking?
by Marc van Grootel 06 Jan '15

06 Jan '15

Hi, For a few new blog posts I'm working on some examples where I compare traditional XQuery code with an Origami example. I am not looking for scientific answers but I do want to be able to do a more or less fair comparison. I noticed that when running code from the GUI timings can be quite different from running basex. Currently I do something like basex -V example/foo.xq or basex -r10 -V example/foo.xq What would be a honest way of including timings in an article. It's mainly for a qualitative assessment and comparing a traditional solution with an Origami solution. What to look for in the breakdown between parsing/compiling/evaluating and printing. Tips? Thanks, and I hope you are all having a good Xmas break. --Marc

2 1

Design finding "almost" duplicate xml files
by Mansi Sheth 02 Jan '15

02 Jan '15

Hello, I am trying to come up with a design, which just before insert a xml file into database, will warn us, that there is almost an identical xml file (with different name and different size) already stored in the database. "Almost identical" would be based on few elements of xml file such as: <root> <A name=""> <C name=""/> <C name=""/> . . </A> . . . </root> "A" and "B" from above snippet but different "C". Element "A" could be repeated 100s of time in single xml file. Any pointers ? - Mansi

2 3

Fwd: Error Undefined Schema Name While Using schema-element(xbrli:item)
by ankit kumar 31 Dec '14

31 Dec '14

Ankit Kumar ---------- Forwarded message ---------- From: ankit kumar <ankitforstudy(a)gmail.com> Date: Wed, Dec 31, 2014 at 2:10 PM Subject: Error Undefined Schema Name While Using schema-element(xbrli:item) To: christian.gruen(a)uni-konstanz.de, alexander.holupirek(a)uni-konstanz.de, cg(a)basex.org, ms(a)basex.org *I am trying to run this xquery script using base-x gui. But i am getting this error as show by gui* *"Undefined Schema name Q{**http://www.xbrl.org/2003/instance <http://www.xbrl.org/2003/instance>**}item" by schema-element(xbrli:item) function.* *Please help me out.* *declare namespace xfi = "http://www.xbrl.org/2008/function/instance <http://www.xbrl.org/2008/function/instance>";* *declare namespace xbrli="http://www.xbrl.org/2003/instance <http://www.xbrl.org/2003/instance>";* *declare function xfi:test($a as schema-element(xbrli:item)) { let $b := 1 return $a};let $context :=//*return //*/xfi:test(.)* Ankit Kumar

1 0

Performance and benchmark
by Florent Gallaire 29 Dec '14

29 Dec '14

Hello, I have evaluated XML databases for an important project, and I'm very happy with BaseX which has great fonctionnalities and is very easy to use. I'm now about deploying my project in production, so I have new questions. My database has 400 000 XML resource files for a total 2.4 Go size, so it should work fine (http://docs.basex.org/wiki/Statistics). 1) What are the CPU/RAM quantityies recommanded to use BaseX in production ? 2) Is there a benchmark to run to have an evaluation of the CPU/RAM quantities from a BaseX point of view ? 3) What about the JVM options ? In particular should I change the "-Xmx512m" default value ? What will be the best value ? 4) With my database, on a t2.micro EC2 instance (1 VCPU + 1 Go RAM), BaseX is unusable. On a powerful shared server (16 VCPU + 16 Go RAM) it works fine (until I exceed my 512Mb personal hosting limitation). On my shared server, a request like "curl 'http://mydomain/rest/database/file.xml?query=/a/direct/path' takes 1.5 second. It seems to me that it's a lot and should be faster as I give the good resource file and the complete path to BaseX, so there is no computation to do. What do you think about it ? 5) How many simultaneous requests can handle BaseXHTTP ? Best regards Florent -- FLOSS Engineer & Lawyer

3 6

element or attribute
by Florent Gallaire 29 Dec '14

29 Dec '14

Hello, When creating a XML file for usage as database with BaseX, is there some policy about when it's good to use an element and when it's good to use an attribut ? Cheers, Florent -- FLOSS Engineer & Lawyer

2 2

Reg : Multi Threading While Adding XMLS
by Chandrasekhar 28 Dec '14

28 Dec '14

Dear Christian We are adding the xml's to the collection database one by one(sequentially),For we are implemented multithreading concept to add the xml's to database,but at basex server level the server is adding the xmls sequentially. In our case we are opening the database and adding xmls,and also from that directory(which is having the xml's ) ,this directory can have the thousand's of xml's ,from this we need to add some particular xml's to the database, that's the reason we are adding one by one xml's to the database. Is their any way to achieve the multithreading to add xml's to database for improving the performence while adding the xml's to the database. Thanks Chandrasekhar The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited.

1 0

Editor Background Color
by Alex R. Loayza 25 Dec '14

25 Dec '14

Hello, I’m new toBaseX, and I’m trying to customize the GUI, by changing the background color ofthe Editor panel. I always use dark themes in IDEs because brightwhites hurt my eyes. However,I’ve found no way to change that setting in Base X. The Options / Colorssetting in the main menu effects changes to panels other than the Editor. Is there away to customize this? Thank youvery much for any help, and especially for having built this great product andmaking it open source! Bestregards, Alex

3 5

Preparing for Origami 0.4: new blog post on HTML templating
by Marc van Grootel 24 Dec '14

24 Dec '14

Hi all, I am preparing for a new release for Origami[1]: 0.4. This will make it almost feature complete. I introduce the most important new function which helps to build an HTML view from HTML templates. An example is described in detail in a new blog post. See http://xokomola.com/2014/12/23/xquery-origami-3.html Merry Christmas, --Marc [1] https://github.com/xokomola/origami

1 0

Use xquery:eval or not?
by Marc van Grootel 23 Dec '14

23 Dec '14

Hi, I would like to ask some advice. For my Origami[1] library I'm exploring different ways of providing more powerful selectors for nodes in a template document. As this is for a templating library I would like to offer string based selectors and am considering if I should go for CSS-style selectors or XPath selectors. I started on CSS selectors to be able to select nodes with selectors like "p#id" and "li.class" but to do this properly it involves quite a bit of string manipulation in order to support more advanced stuff. Being in a language that already has XPath in it, it would be much more natural and powerful to use XPath selectors. This led me to look at xquery:eval. So far I resisted using eval but the code is a lot simpler that way. Implementing CSS-style selectors would probably lead me down a path with a lot more ugly string manipulation. I'm aware that it's better to express these things in XQuery code and that is also possible. But it's so much nicer to read this: xf:extract(xf:select('li'), $doc) then this: xf:extract(function($node) { $node/li }, $doc) I'm interested in pros and cons of using xquery:eval with XPath string-selectors. How is the performance? Other downsides (such as being non-standard)? Thanks --Marc [1] https://github.com/xokomola/origami

4 11

Need help to query
by Florent Gallaire 23 Dec '14

23 Dec '14

Hello, I need some help to query a database of thousands of xml files. They are structured only with elements with no attributes : file1.xml <A> <C>value</C> <D> <E>text</E> </D> </A> file2.xml <A> <C>value</C> <D> <E>text</E> </D> </A> and so on file3.xml, file4.xml.... Having the value of the <C> element, I would like to have the text of the <E> element. What is the good query to do this ? Best regards Florent -- FLOSS Engineer & Lawyer

3 4

Disk full caused by huge index
by Ludovic Kuty 23 Dec '14

23 Dec '14

Dear users, Can I remove the file txtl.basex and still get an access to my data ? BaseX of course will be stopped. My HDD is full and I can't optimize all because BaseX times out. Then I would like to deactivate the UPDINDEX option on my already existing DB. I already read the thread https://mailman.uni-konstanz.de/pipermail/basex-talk/2014-July/006942.html ( UPDINDEX and ever growing index size ) and was well aware of the problem but I didn't know that it would grow so fast. total 214335012 -rw-r--r-- 1 root root 1620720 Dec 23 02:54 atv.basex -rw-r--r-- 1 root root 2300192644 Dec 23 02:54 atvl.basex -rw-r--r-- 1 root root 337605 Dec 23 02:54 atvr.basex -rw-r--r-- 1 root root 15 Dec 23 02:54 idp.basex -rw-r--r-- 1 root root 137256 Dec 23 02:54 inf.basex -rw-r--r-- 1 root root 631959552 Dec 23 02:54 tbl.basex -rw-r--r-- 1 root root 1220641 Dec 23 02:54 tbli.basex -rw-r--r-- 1 root root 146626967 Dec 23 02:54 txt.basex -rw-r--r-- 1 root root 216396026742 Dec 23 02:54 txtl.basex -rw-r--r-- 1 root root 838900 Dec 23 02:54 txtr.basex TIA, Ludovic Kuty

2 3

file names with [] and base-uri
by Andy Bunce 22 Dec '14

22 Dec '14

Hi, If I create a database with a file with [] in the name e.g. *bracket[in]name.xml *then /*/base-uri(.) gives the error [XQST0046] URI 'bracket[in]name/bracket[in]name.xml' is invalid. Query: /*/base-uri(.) Is this correct behavior? /Andy

2 2

Bug in array:fold-left?
by Marc van Grootel 21 Dec '14

21 Dec '14

Hi, I think I've spotted a bug in array:fold-left (using 20141219 snapshot) array:tail([1,2,3]) => [2,3] array:fold-left( array:tail([1,2,3]), (), function($result, $fn) { ($result, $fn) } ) => (1,2) Using array:fold-right in the above example returns (2,3). -- --Marc

2 2

fn:apply will be in XQuery 3.1
by Marc van Grootel 20 Dec '14

20 Dec '14

Just in: Michael Kay gave fn:apply it's blessing and it will be in XQuery 3.1. Great news. Thanks to the WG and all who participated. I don't want to sound too pushy, but ... eh ... when can we have it ;-)) See: https://www.w3.org/Bugs/Public/show_bug.cgi?id=26585 --Marc

5 7

tiny issue
by Rob Stapper 19 Dec '14

19 Dec '14

Hi, Concerning latest snapshot: color scheme doesn't apply correctly on datatype-declarations of function parameters ( as xs:string). It still works fine on the datatype-declaration of the function itself. See attachment Rob --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. http://www.avast.com

2 1

BaseX Users Meetup, XML Prague 2015
by Christian Grün 18 Dec '14

18 Dec '14

Hi all, We are pleased to officially announce our BaseX Users Meetup 2015, collocated with the XML Prague Conference. It will take place on Friday, 14:30 – 18:00. The current sessions and the list of speakers are listed on Lanyrd and, soon, the conference site: http://lanyrd.com/2015/basex http://www.xmlprague.cz/pre-conference-friday XML Prague continues to be the biggest, most interesting and, at the same time, very affordable XML event in Europe. If you haven't gone there yet, this is your chance! Looking forward to seeing you; merry BaseXMas, Christian

1 0

What is the best way to implement an async REST method?
by Marc van Grootel 16 Dec '14

16 Dec '14

How would you implement an async REST pattern? I don't remember having seen anything that makes me do this using plain XQuery. Does this mean the only way to tackle this would be via Java and events maybe? I would post to a URL, this URL would accept the post and return a temporary URL where I can check the status of my request, then once it's ready this temp URL will redirect me to the location of the real resource. That's the gist of the async REST pattern that I would like to implement. What would be the best approach to experiment with this? --Marc

5 9

RESTXQ - access to InitParameters of ServletContext and/or ServletConfig
by Fabrice Etanchaud 16 Dec '14

16 Dec '14

Hi all BaseX users , Inside a RESTXQ function, I would like to get InitParameters of the ServletContext or ServletConfig instances, I cannot find that in a module. Did I miss something ? Best regards, Fabrice Etanchaud

2 6

BaseX 8.0: DBA
by Christian Grün 15 Dec '14

15 Dec '14

Hi all, This is yet another update on the next, upcoming version of BaseX: We have added a new database administration (DBA) web frontend to our ZIP and EXE distributions. It allows you to create and administrate databases, evaluate queries in realtime, view log files, manage users, and more. It was completely written with XQuery and RESTXQ, apart from a few lines of Javascript code. The new frontend was added due to the known shortcomings of the Java GUI, which cannot be used to run queries on other servers. As a consequence, we have removed the server dialog from the desktop application. We hope that DBA will help you to write your own web applications based on RESTXQ! Some more infos and design goals can be found in our Wiki [1]. Have fun, we are looking forward to your feedback, Christian PS: NO, the web frontend does not provide all the cute features of the desktop gui, like the treemap, scatterplot, syntax highlighting, etc… But it may provide some of them in future. Your contributions are welcome! [1] http://docs.basex.org/wiki/DBA

5 12

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk