- BaseX-Talk - mailman.uni-konstanz.de

BaseX 8.4.1
by Christian Grün 03 Mar '16

03 Mar '16

Hi! A new patch release of BaseX has been available [1]. It includes minor bug fixes and, as usual, beta versions of the features that are announced in the documentation for the next minor/major release [2]. Have fun! Christian [1] http://basex.org/products/download [2] http://docs.basex.org/index.php?search=%22Version+8.5%22

1 0

Re: [basex-talk] out of main memory
by michele.greco2＠email.it 03 Mar '16

03 Mar '16

I Christian, i tried and this is my error: Error:Could not create the Java Virtual Machine Error:A fatal Exception has occured.Program will exit   --------- Original Message -------- Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> To: Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>om>, "BaseX" <basex-talk(a)mailman.uni-konstanz.de> Oggetto: Re: [basex-talk] out of main memory Data: 03/03/16 11:57 > I tried to do what i suggested Costantine with the value 1024m, but my > memory is alwais 495m. Please try the following: 1. Download the ZIP version of BaseX (http://files.basex.org/releases/latest/) 2. Unzip it, edit bin/basexgui.bat (provided that you are using Windows): OLD: set BASEX_JVM=-Xmx512m %BASEX_JVM% NEW: set BASEX_JVM=-Xmx4g %BASEX_JVM% 3. Start bin/basexgui.bat 4. Click on the lower right memory bar and compare it with the attached screenshot. Does this help? Christian On Thu, Mar 3, 2016 at 12:26 PM, <michele.greco2(a)email.it> wrote: > > --------- Original Message -------- > Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> > To: > Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>om>, "BaseX" > <basex-talk(a)mailman.uni-konstanz.de> > Oggetto: Re: [basex-talk] out of main memory > Data: 03/03/16 11:03 > > > >> queries work. The Query on the extraction of text from files. I 495m >> available and would like increase it to at least 1024m. > > So which value did you assign via -Xmx? Do you start BaseX via basexgui.bat? > > >> Michele >> >> --------- Original Message -------- >> Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> >> To: >> Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>om>, "BaseX" >> <basex-talk(a)mailman.uni-konstanz.de> >> Oggetto: Re: [basex-talk] out of main memory >> Data: 03/03/16 10:42 >> >> >> >> Hi Michele, >> >> How much memory do you have available on your system, and how much did >> you assign? What kind of operations are triggering the out-of-memory >> errors? >> >> Christian >> >> >> >> On Thu, Mar 3, 2016 at 9:34 AM, <michele.greco2(a)email.it> wrote: >>> Hi Constantine, thanks for your answer. >>> I'm trying to do what you suggested but all my tests so far have had no >>> effect. >>> Do you have any particular which have to be careful as you suggest? >>> >>> Regards, >>> Michele >>> >>> --------- Original Message -------- >>> Da: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com> >>> To: "basex-talk(a)mailman.uni-konstanz.de" >>> <basex-talk(a)mailman.uni-konstanz.de> >>> Oggetto: RE: [basex-talk] out of main memory >>> Data: 01/03/16 10:07 >>> >>> Hi Michele, >>> >>> 1. Go to <BASEX_INSTALL>/bin >>> 2. Open basexgui.bat (Windows) or basexgui (*nix) >>> 3. Increase value of Xmx in BASEX_JVM variable, from 512 MB to >>> something that your system can support >>> 4. Restart the GUI >>> >>> For example, here is the line from my config I allocate almost 5 >>> Gigabytes of memory >>> >>> set BASEX_JVM=-Xmx5000m %BASEX_JVM% >>> >>> Regards, >>> Cosntantine >>> >>> >>> From: basex-talk-bounces(a)mailman.uni-konstanz.de >>> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of >>> michele.greco2(a)email.it >>> Sent: 01 March 2016 10:48 >>> To: basex-talk(a)mailman.uni-konstanz.de >>> Subject: [basex-talk] out of main memory >>> >>> Hi, i get the error "out of main memory", when i run my query on a >>> collection in basex gui. >>> How can i fix? >>> How can i increase my memory? >>> Regards >>> Michele >>> >>> ---- >>> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >>> tutti i dettagli clicca qui >>> >>> Sponsor: >>> Idee regalo classiche o alternative? Trova l'offerta migliore in un click >>> Clicca qui >>> >>> ________________________________ >>> Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The >>> Netherlands, Registration No. 33156677, Registered in The Netherlands. >>> >>> >>> ---- >>> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >>> tutti i dettagli clicca qui >>> >>> Sponsor: >>> Idee regalo classiche o alternative? Trova l'offerta migliore in un click >>> Clicca qui >> >> >> ---- >> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >> tutti i dettagli clicca qui >> >> Sponsor: >> Caselle con tuo dominio su piattaforma Zimbra, fino a 30 GB di spazio, >> sincronizzazione dati e backup >> Clicca qui > > > ---- > ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per > tutti i dettagli clicca qui > > Sponsor: > Soluzioni di email hosting per tutte le esigenze: dalle caselle gratuite a > quelle professionali su piattaforma Zimbra, da quelle su proprio dominio a > quelle certificate PEC. Confronta le soluzioni > Clicca qui -- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli Clicca qui http://posta.email.it/caselle-di-posta-z-email-it/?utm_campaign=email_Zimbr… Sponsor: Registra i domini che desideri ed inizia a creare il tuo sito web Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=13323&d=3-3

2 1

Import module with xquery:eval() function
by Ioan Fericel 03 Mar '16

03 Mar '16

Hi, I try to import a module dynamically, as in this example from a earlier discussion: > let $s:='import module namespace p="xquery-30" at "../repotest/test.xqm"; > p:parse-XQuery("2+2")' > return xquery:eval($s) It gives me the following error: > . . . [XQST0059] Could not retrieve module '/home/repotest/test.xqm'. I run BaseX 8.3 as war in Tomcat container. If the module is imported in static way: > import module namespace p = "xquery-30 " at "../repotest/test.xqm"; everything works correctly. From where does the problem with malformed relative path, what solution could be found? Thanks, Ioan

2 2

Re: [basex-talk] out of main memory
by michele.greco2＠email.it 03 Mar '16

03 Mar '16

I tried to do what i suggested Costantine with the value 1024m, but my memory is alwais 495m. --------- Original Message -------- Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> To: Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>om>, "BaseX" <basex-talk(a)mailman.uni-konstanz.de> Oggetto: Re: [basex-talk] out of main memory Data: 03/03/16 11:03 > queries work. The Query on the extraction of text from files. I 495m > available and would like increase it to at least 1024m. So which value did you assign via -Xmx? Do you start BaseX via basexgui.bat? > Michele > > --------- Original Message -------- > Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> > To: > Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>om>, "BaseX" > <basex-talk(a)mailman.uni-konstanz.de> > Oggetto: Re: [basex-talk] out of main memory > Data: 03/03/16 10:42 > > > > Hi Michele, > > How much memory do you have available on your system, and how much did > you assign? What kind of operations are triggering the out-of-memory > errors? > > Christian > > > > On Thu, Mar 3, 2016 at 9:34 AM, <michele.greco2(a)email.it> wrote: >> Hi Constantine, thanks for your answer. >> I'm trying to do what you suggested but all my tests so far have had no >> effect. >> Do you have any particular which have to be careful as you suggest? >> >> Regards, >> Michele >> >> --------- Original Message -------- >> Da: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com> >> To: "basex-talk(a)mailman.uni-konstanz.de" >> <basex-talk(a)mailman.uni-konstanz.de> >> Oggetto: RE: [basex-talk] out of main memory >> Data: 01/03/16 10:07 >> >> Hi Michele, >> >> 1. Go to <BASEX_INSTALL>/bin >> 2. Open basexgui.bat (Windows) or basexgui (*nix) >> 3. Increase value of Xmx in BASEX_JVM variable, from 512 MB to >> something that your system can support >> 4. Restart the GUI >> >> For example, here is the line from my config I allocate almost 5 >> Gigabytes of memory >> >> set BASEX_JVM=-Xmx5000m %BASEX_JVM% >> >> Regards, >> Cosntantine >> >> >> From: basex-talk-bounces(a)mailman.uni-konstanz.de >> [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of >> michele.greco2(a)email.it >> Sent: 01 March 2016 10:48 >> To: basex-talk(a)mailman.uni-konstanz.de >> Subject: [basex-talk] out of main memory >> >> Hi, i get the error "out of main memory", when i run my query on a >> collection in basex gui. >> How can i fix? >> How can i increase my memory? >> Regards >> Michele >> >> ---- >> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >> tutti i dettagli clicca qui >> >> Sponsor: >> Idee regalo classiche o alternative? Trova l'offerta migliore in un click >> Clicca qui >> >> ________________________________ >> Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The >> Netherlands, Registration No. 33156677, Registered in The Netherlands. >> >> >> ---- >> ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per >> tutti i dettagli clicca qui >> >> Sponsor: >> Idee regalo classiche o alternative? Trova l'offerta migliore in un click >> Clicca qui > > > ---- > ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per > tutti i dettagli clicca qui > > Sponsor: > Caselle con tuo dominio su piattaforma Zimbra, fino a 30 GB di spazio, > sincronizzazione dati e backup > Clicca qui -- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli Clicca qui http://posta.email.it/caselle-di-posta-z-email-it/?utm_campaign=email_Zimbr… Sponsor: Registra i domini che desideri ed inizia a creare il tuo sito web Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=13323&d=3-3

2 1

Re: [basex-talk] out of main memory
by michele.greco2＠email.it 03 Mar '16

03 Mar '16

Hi, i created a collection of XML files, and so everithing is right. The error out of memory happens when i run some queries about my collection,other queries work. The Query on  the extraction of text from files. I 495m available and would like increase it to at least 1024m. Michele --------- Original Message -------- Da: "Christian GrÃ¼n" <christian.gruen(a)gmail.com> To: Cc: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com>om>, "BaseX" <basex-talk(a)mailman.uni-konstanz.de> Oggetto: Re: [basex-talk] out of main memory Data: 03/03/16 10:42 Hi Michele, How much memory do you have available on your system, and how much did you assign? What kind of operations are triggering the out-of-memory errors? Christian On Thu, Mar 3, 2016 at 9:34 AM, <michele.greco2(a)email.it> wrote: > Hi Constantine, thanks for your answer. > I'm trying to do what you suggested but all my tests so far have had no > effect. > Do you have any particular which have to be careful as you suggest? > > Regards, > Michele > > --------- Original Message -------- > Da: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com> > To: "basex-talk(a)mailman.uni-konstanz.de" > <basex-talk(a)mailman.uni-konstanz.de> > Oggetto: RE: [basex-talk] out of main memory > Data: 01/03/16 10:07 > > Hi Michele, > > 1. Go to <BASEX_INSTALL>/bin > 2. Open basexgui.bat (Windows) or basexgui (*nix) > 3. Increase value of Xmx in BASEX_JVM variable, from 512 MB to > something that your system can support > 4. Restart the GUI > > For example, here is the line from my config I allocate almost 5 > Gigabytes of memory > > set BASEX_JVM=-Xmx5000m %BASEX_JVM% > > Regards, > Cosntantine > > > From: basex-talk-bounces(a)mailman.uni-konstanz.de > [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of > michele.greco2(a)email.it > Sent: 01 March 2016 10:48 > To: basex-talk(a)mailman.uni-konstanz.de > Subject: [basex-talk] out of main memory > > Hi, i get the error "out of main memory", when i run my query on a > collection in basex gui. > How can i fix? > How can i increase my memory? > Regards > Michele > > ---- > ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per > tutti i dettagli clicca qui > > Sponsor: > Idee regalo classiche o alternative? Trova l'offerta migliore in un click > Clicca qui > > ________________________________ > Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The > Netherlands, Registration No. 33156677, Registered in The Netherlands. > > > ---- > ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per > tutti i dettagli clicca qui > > Sponsor: > Idee regalo classiche o alternative? Trova l'offerta migliore in un click > Clicca qui -- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli Clicca qui http://posta.email.it/caselle-di-posta-z-email-it/?utm_campaign=email_Zimbr… Sponsor: Idee regalo classiche o alternative? Trova l'offerta migliore in un click Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=13327&d=3-3

2 1

Re: [basex-talk] out of main memory
by michele.greco2＠email.it 03 Mar '16

03 Mar '16

Hi Constantine, thanks for your answer. I'm trying to do what you suggested but all my tests so far have had no effect. Do you have any particular which have to be careful as you suggest?   Regards, Michele --------- Original Message -------- Da: "Hondros Constantine ELS-AMS" <C.Hondros(a)elsevier.com> To: "basex-talk(a)mailman.uni-konstanz.de" <basex-talk(a)mailman.uni-konstanz.de> Oggetto: RE: [basex-talk] out of main memory Data: 01/03/16 10:07 Hi Michele,   1.       Go to <BASEX_INSTALL>/bin 2.       Open basexgui.bat (Windows) or basexgui (*nix) 3.       Increase value of –Xmx in BASEX_JVM variable, from 512 MB to something that your system can support 4.       Restart the GUI   For example, here is the line from my config –  I allocate almost 5 Gigabytes of memory   set BASEX_JVM=-Xmx5000m %BASEX_JVM%   Regards, Cosntantine     From: basex-talk-bounces(a)mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of michele.greco2(a)email.it Sent: 01 March 2016 10:48 To: basex-talk(a)mailman.uni-konstanz.de Subject: [basex-talk] out of main memory   Hi, i get the error "out of main memory", when i run my query on a collection in basex gui. How can i fix? How can i increase my memory? Regards  Michele   ---- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli clicca qui Sponsor: Idee regalo classiche o alternative? Trova l'offerta migliore in un click Clicca qui Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands. -- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli Clicca qui http://posta.email.it/caselle-di-posta-z-email-it/?utm_campaign=email_Zimbr… Sponsor: Idee regalo classiche o alternative? Trova l'offerta migliore in un click Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=13327&d=3-3

2 1

Parallel execution
by Klāvs Priedītis 02 Mar '16

02 Mar '16

Hello, BaseX team and community! Disclosure: I am not yet an expert on these issues and I have not fully studied the specs of XQuery. Now that Async Module is incorporated, I am wondering about some basic properties of xquery. Example: ```xquery for $i in 1 to 10 return fetch:text("http://example.com/magic-story/for-number/" || $i) ``` It feels to me that the xquery processor is broken if it cannot figure out on its own that the fetch can be done in parallel. Why does this not work out of the box? Can it work out of the box? Wouldn't it be able to do this in parallel if a flag `can-be-optimized-for-parallel-execution` was defined for the `fetch:text`? -- Veiksmi vēlot, Klāvs Priedītis

3 6

BaseX in a Multi-User VM
by Cecil Tarasoff 02 Mar '16

02 Mar '16

Hello, The environment that I am using BaseX GUI in is on a Virtual Workstation that is shared by multiple users. I am finding that my preferences end up being global and any user of BaseX on this workstation seems to be using the preferences that I have defined. Is it possible to install BaseX onto a multi-user workstation so that we each have our own config and repositories and keep our own data private? Thanks. Cec Tarasoff.

2 1

client:query freeze
by Strasser Pablo 01 Mar '16

01 Mar '16

Hi, Thanks for your great software. Using basex 8.4. I found the following strange behaviour in client:query call. Calling a repository function which use client:query to call another repository function freeze the basex server if done on the server. The only way to stop the server is with a kill -9. However using the standalone client it work normally. Trying to reproduce this problem I managed to get a similar problem using 2 repository test and test2. Where the first repo use a function of the second. Do you have some idea how to investigate this problem? Is it a bug? Code of test: module namespace m = 'http://strasser-family.ch/test'; declare variable $m:c := client:connect('localhost', 1984, 'admin', 'admin'); declare function m:test(){ client:query($m:c, "import module namespace m = 'http://strasser-family.ch/test2';m:test()") }; Code of test2: module namespace m = 'http://strasser-family.ch/test2'; declare function m:test() { <dummy/> }; usage code: import module namespace m = 'http://strasser-family.ch/test'; m:test() Have a nice evening. Pablo

2 5

out of main memory
by michele.greco2＠email.it 01 Mar '16

01 Mar '16

Hi, i get the error "out of main memory", when i run my query on a collection in basex gui. How can i fix? How can i increase my memory? Regards  Michele -- ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i dettagli Clicca qui http://posta.email.it/caselle-di-posta-z-email-it/?utm_campaign=email_Zimbr… Sponsor: Soluzioni di email hosting per tutte le esigenze: dalle caselle gratuite a quelle professionali su piattaforma Zimbra, da quelle su proprio dominio a quelle certificate PEC. Confronta le soluzioni Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=13326&d=1-3

2 1

About user names
by France Baril 29 Feb '16

29 Feb '16

Hi, I'm looking at the user module because I want to implement a login feature on one of my web application. More specifically, I'm looking at the restrictions: http://docs.basex.org/wiki/Valid_Names#Valid_Names For web applications, it is very usual for people to use an email as a username. The restriction on the period character is very annoying in that regard. I understand why it's a restriction on database names, but for user names it creates a limit on the ability for less frequent users to access the site. Most people remember which email they used or have only 2 or 3 to try. Passwords are saved in keychains. However, remembering a username... pfff! Is there any way that the restrictions on user names could be adapted to allow for characters that are valid for an email address in the near future? -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

3 2

Re: [basex-talk] basex-xqj source?
by Kendall Shaw 29 Feb '16

29 Feb '16

Oh, it’s here: http://xqj.net/basex/download.jsp Kendall From: <basex-talk-bounces(a)mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de>> on behalf of Kendall Shaw <kendall.shaw(a)workday.com<mailto:kendall.shaw@workday.com>> Date: Monday, February 22, 2016 at 3:27 PM To: "basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>" <basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>> Subject: [basex-talk] basex-xqj source? Hi, Where can I find source code for the basex xqj implementation? I want to see what it is doing. Thanks, Kendall

2 2

Re: [basex-talk] Add docments and retrieve results using java
by Kendall Shaw 23 Feb '16

23 Feb '16

It looks like the basex-test projects shows how to do what I want. Using QueryProcessor and the Xdm classes. Kendall From: <basex-talk-bounces(a)mailman.uni-konstanz.de<mailto:basex-talk-bounces@mailman.uni-konstanz.de>> on behalf of Kendall Shaw <kendall.shaw(a)workday.com<mailto:kendall.shaw@workday.com>> Date: Monday, February 22, 2016 at 4:56 PM To: "basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>" <basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de>> Subject: [basex-talk] Add docments and retrieve results using java How can I incrementally import XML documents into the database, using java? And, how can I incrementally retrieve query results and XML documents from the database, using java? Either I am missing it, or XQJ does not allow you to store data into a database. Should I use xmldb API? Or, should I use the native API? If so how do I incrementally read and write, instead of loading everything into a string or reading and writing to/from files? Kendall

1 0

Add docments and retrieve results using java
by Kendall Shaw 22 Feb '16

22 Feb '16

How can I incrementally import XML documents into the database, using java? And, how can I incrementally retrieve query results and XML documents from the database, using java? Either I am missing it, or XQJ does not allow you to store data into a database. Should I use xmldb API? Or, should I use the native API? If so how do I incrementally read and write, instead of loading everything into a string or reading and writing to/from files? Kendall

1 0

basex-xqj source?
by Kendall Shaw 22 Feb '16

22 Feb '16

Hi, Where can I find source code for the basex xqj implementation? I want to see what it is doing. Thanks, Kendall

1 0

Benchmarking and caching in BaseX
by Bram Vanroy | KU Leuven 21 Feb '16

21 Feb '16

Dear all My name is Bram Vanroy, and I am an intern at the Centre for Computational Linguistics (CCL; http://www.arts.kuleuven.be/ling/ccl [Dutch]) at the University of Leuven. My supervisor, Vincent Vandeghinste, has had contact with this mailing list some time ago, more specifically with Dirk Kirsten. My intership is titled "Fine-tuning the GrETEL Treebank Query Engine". GrETEL stands for Greedy Extraction of Trees for Empirical Linguistics; available at http://gretel.ccl.kuleuven.be/gretel-2.0/. Its goal is to provide users with a fast, user-friendly on-line tool to search through text corpora backed by treebanks. Accessibility is an important point for us: users do not need to be proficient with any programming languages, strict formalisms, or treebank specific annotations; every query can be executed by using an intuitive graphical interface. More advanced users can use XPath to write the representation of the syntactic structure that they are looking for. BaseX is our tool of choice as a database for our corpora in XML format. Initially, GrETEL provided access to smaller corpora such as CGN (9 million words) and Lassy Small (1 million words). We would like to expand the searchable corpora by also making the full Sonar corpus available (500 million words). This is already partially possible in GrETEL 2.0 but due to efficiency reasons, capabilities are restricted: users can only search in one component at a time, and the largest component in the corpus is not available due to its size (15 million sentences). We have applied these restrictions because the search time for the whole corpus was too long, which in turn would decrease the user-friendliness of the tool drastically. Steps have already been taken to improve search times in larger corpora. (See "Making a Large Treebank Searchable Online. The SoNaR Case." by Vincent Vandeghinste, and Liesbeth Augustinus; http://nederbooms.ccl.kuleuven.be/documentation/LREC2014-GrETELSoNaR.pdf.) To spare you the effort to go through the whole article, I hereby quote the most relevant citation from that article for this email: The general idea behind our approach is to restrict the search space by splitting up the data in many small databases, allowing for faster retrieval of syntactic structures. We organise the data in databases that contain all bottom-up subtrees for which the two top levels (i.e. the root and its children) adhere to the same syntactic pattern. When querying the database for certain syntactic constructions, we know on which databases we have to apply the XPath query which would otherwise have to be applied on the whole data set. We have called this method GrETEL Indexing (GrInd). (p. 17) So to optimise searching, the data has been pulled apart - in a sense - which would make the search space smaller and subsequently the search time shorter. In the future we would like to apply this technique on parallel corpora as well. We have not tested yet what influence this change has made to query time which is what I am going to find out during my internship. I have already analysed the XPath queries that users have made since GrETEL saw its first user and found that the queries are ten embedded levels deep at the most, but most are between one and five. The amount of nodes per query varies between one and 24, but most searches are for structures that contain between one and eight nodes. Based on this information, I am writing example XPaths that I am going to pull through BaseX as a sort of benchmark. I can then compare the query speeds between the split-up corpus, and the regular one. The problem that I have encountered is that BaseX seems to cache very efficiently. Obviously this is not a problem on production websites but for benchmarking it may not be ideal. My first question to you, then, is: is it possible to disable caching when testing queries locally? And how exactly does BaseX handle the caching? Or more specifically, if I enter a query: what is cached, and for how long? This information me be useful to analyse our logs with. If you have any feedback on GrETEL, or the new approach of GrInding, or if you have any ideas to improve search time for large corpora - I would love to hear from you, you can contact me via this email address or on LinkedIn. I reply to each email as extensively as possible. Thank you in advance, Kind regards Bram Vanroy https://be.linkedin.com/in/bramvanroy

3 4

from xquery:parse to data paths ...
by Hans-Juergen Rennau 20 Feb '16

20 Feb '16

Dear BaseX team, perhaps I hear an opinion about the following idea. (1) The function xquery:parse is already immensely useful as it allows to *validate* XPath expressions, as for example used in configuration data. (A practical example: the XPath expressions used in JMeter [1] test plans in order to extract message data and define test assertions.) (2) Given the XSDs of a Web Service (or other application), it is possible to determine all valid data paths (e.g. /a/b/c is valid, /a/b/C is not valid). (3) If there were a reliable way to map the output of xquery:parse to the implied data paths, then one could use xquery:parse to validate xpath expression not only for syntactic correctness (which is already very much!), but also for consistency with application XSDs. This would be extremly usefuly as XSDs evolve and a checking if configuration-based XPath expressions must be adapted is otherwise very difficult to achieve in any systematic way. (4) Looking at the xquery:parse output, it seems certainly feasible to write such a mapping (xquery:parse output => data paths); but the problem I see is that the format is not guaranteed to be stable, as it is no standard. (And it is probably not described.) If BaseX provided such a functionality as an additional extension function (e.g. xquery:data-paths($query as xs:string)) you would simply be - heros. Cheers,Hans-Jürgen [1] http://jmeter.apache.org/ PS: Illustrative example. let $query := "/descendant::OutboundFlight[Status = 'Booked'][1]/FlightDeparture/DepartureTime"return xquery:parse($query)=><MainModule updating="false"> <QueryPlan compiled="false"> <CachedPath> <Root/> <IterPosStep axis="descendant" test="OutboundFlight"> <CmpG op="="> <CachedPath> <IterStep axis="child" test="Status"/> </CachedPath> <Str value="Booked" type="xs:string"/> </CmpG> <Int value="1" type="xs:integer"/> </IterPosStep> <IterStep axis="child" test="FlightDeparture"/> <IterStep axis="child" test="DepartureTime"/> </CachedPath> </QueryPlan> </MainModule> let $query := "/descendant::OutboundFlight[FlightDeparture/DepartureDate][1]/FlightDeparture/DepartureDate"return xquery:data-paths($query)=>//OutboundFlight/FlightDeparture/DepartureTime,//OutboundFlight/Status

2 6

Header problems and xmlns
by Ana Victória Ladeira 20 Feb '16

20 Feb '16

Good afternoon! I currently have a dataset with some files that use the following header (I will call it Header1): <?xml version="1.0" encoding="ISO-8859-1"?><CURRICULO-VITAE SISTEMA-ORIGEM-XML="LATTES_OFFLINE" xmlns:lattes=" http://www.cnpq.br/2001/XSL/Lattes" NUMERO-IDENTIFICADOR="X" DATA-ATUALIZACAO="07122015" HORA-ATUALIZACAO="010008" xsi:schemaLocation=" http://www.cnpq.br/2014/XSD/Lattes/Versao1 CurriculoLattes.xsd" xmlns=" http://www.cnpq.br/2014/XSD/Lattes/Versao1" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance"> Other files use this other header (Header2): <?xml version="1.0" encoding="UTF-8"?><CURRICULO-VITAE SISTEMA-ORIGEM-XML="LATTES_OFFLINE" DATA-ATUALIZACAO="13072015" HORA-ATUALIZACAO="145233" NUMERO-IDENTIFICADOR="X" xmlns:lattes=" http://www.cnpq.br/2001/XSL/Lattes"> Using the same query, I can extract data from files with Header2, but now with Header1. I'm only using FLWOR expressions, and the queries are very simple. I've tried with multiple queries, and the result is always the same: I'm able to extract the information I want from files with Header2, but not from the ones with Header1. I think its a problem with the namespace, but I don't know how to solve it. Can anyone please help me with that? Thank you in advance, Ana Victória Ladeira

2 1

Odd Behavior from Ruby Client
by Eliot Kimber 20 Feb '16

20 Feb '16

I have my Ruby-based server-side commit hooks working but now I'm seeing a failure result I can't figure out yet. I have a large sample data set that I'm using for testing (https://hub.docker.com/r/ditacommunity/demo-content/). This content is non-trivial DITA and all the files are valid (e.g., Oxygen validates the maps and topics). However, when I load the files via my Ruby script that uses the BaseXClient.rb file, a few of the files fail to load with the error indicating that something in the BaseX client/server chain is trying to interpret the XML data as a file path. As far as I can tell it's the same documents every time, so I suspect there is something about the documents' details that is causing this. I guess my question is: what could cause this behavior and how might I go about debugging and correcting it? The Ruby script is here: https://github.com/dita-for-small-teams/dfst-git-commit-hooks/blob/develop/ server-side/post-receive I probably have to rewrite the script to use the REST API so that I have more control over the encoding details (per earlier discussion of how to handle UTF-16 documents) but I'd still like to what's causing this as it points up to either a really subtle user error or a flaw in the basic API mechanism. Cheers, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

1 0

Automated Docker Build: basex/basexhttp
by Jens Erat 19 Feb '16

19 Feb '16

Dear BaseX community, over the last weeks, interest in Docker utilization with BaseX hevily increased, and several images have been proposed. I've already made some experience running a BaseX pet project in BaseX for about a year now, and together with the BaseX core team created an image based on those experiences and current best practices. basex/basexhttp Docker Image ---------------------------- Finally there's the official basex/basexhttp Docker image readily available on the Docker Hub. It's an automated build directly from source, with nightly builds and tagged builds for future BaseX releases (starting with 8.4.1/8.5, whatever will come first). It is derived from the Maven base image (and thus Debian), and is automatically updated when the base images experience updates (like Java security fixes). - https://hub.docker.com/r/basex/basexhttp/ - https://github.com/BaseXdb/basex/blob/master/Dockerfile While the image is named basexhttp and always includes the HTTP server, it can also be used for running the plain basexserver or even basexclient. DBA Application Container ------------------------- As an example for deriving your own application images and also for interfacing BaseX for administrative tasks and ad-hoc queries, the DBA is also made available as a container. - https://hub.docker.com/r/basex/dba/ - https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp/dba Documentation ------------- As you're used to, documentation is available in the BaseX wiki. - http://docs.basex.org/wiki/Docker - - - If you've got any feedback, questions or proposals, feel free to get in touch with me or the core BaseX team on the usual ways. Kind regards from Lake Constance, Germany, Jens -- Jens Erat [phone]: tel:+49-151-56961126 [mail]: mailto:email@jenserat.de [jabber]: xmpp:jabber@jenserat.de [web]: http://www.jenserat.de OpenPGP: 0D69 E11F 12BD BA07 7B37 26AB 4E1F 799A A4FF 2279

2 4

Getting element counts in the visualizations?
by Steinar Bang 19 Feb '16

19 Feb '16

Platform: Intel i7 Windows 7 Enterprise BaseX 8.4 Today I opened an 80MB XML file in the BaseX GUI, and I was amazed at the speed of BaseX, compared to e.g. trying to open, count, and extract stuff from the same file in emacs. What I needed from BaseX was finding the size of the contents of the second level elements (ie. the elements immediately below the root element). The visualizations were great at navigating in the file and the Map visualization showed the sizes I was interested in. I was comparing two files with the same structure, one file with size 60MB, the other with size 80MB, and tried to figure out where the 20MB had gone. On thing I was looking for, but did not find, was the number of child elements of a given element. I was hoping for a tooltip when hovering over an element in the visualizations (map, folder and tree), or a properties dialog when right clicking the elements in the visualization. Is the information present somewhere? (I know the element child count has to be there somewhere, because without it, the map visualization couldn't be rendered...?) How hard is this information to get at? Thanks! - Steinar PS what I ended up doing, was selecting the elements I wanted to find the size of in map and tree, and then saving the contents of the Result view for each element. A bit more clumsy than I would have liked, but much faster than doing it in emacs.

2 6

Re: [basex-talk] Sending Bytes, Not Strings, To BaseX Using the Ruby Client
by Eliot Kimber 19 Feb '16

19 Feb '16

I turned my UTF-8 file into a UTF-16 file and trying to commit it to BaseX via the Ruby client it did not work: BaseXClient.rb:50:in `execute': Resource "/opt/basex/?" not found. (RuntimeError) Where "?" is some kind of "unrecognized character" indicator Cheers, E. ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com On 2/18/16, 10:26 AM, "Eliot Kimber" <basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of ekimber(a)contrext.com> wrote: >I'm implementing server-side git hooks for use in GitLab under Docker >where Java is not available (at least that I can see). The hooks load or >delete files from databases in BaseX. > >I'm trying to implement the hooks in Ruby (which is much more pleasant >than bash scripting in any case) and I'm using the BaseXClient.rb from >https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby > >I need to create or replace files by sending the bytes--I'd rather not >read the input file into a Ruby string and send that since I don't trust >Ruby to not hose up the data (even when it's UTF-8 I still don't trust it, >but I only started using Ruby yesterday so maybe my mistrust is >misplaced?). > >Using the AddExample.rb as guide, I'm doing this: > >(Earlier code to open or create database, which works). > >file = File.new("../../" + path, "rb") > bytes = file.read > file.close > puts "file=/#{bytes}/" > @basex.add(path, "#{bytes}") > >I also tried: > >@basex.add(path, bytes) > > > >And I get this result (I added some debugging messages to sendCmd()): > >ensureDatabase(): Checking database "_dfst^metadata^temp^master"... >BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53 ms. >Added or modified file: "test-newname.xml" >file=/<test>This is a test 20</test> >/ > >*** sendCmd(): >cmd= >arg=test-newname.xml >input=<test>This is a test 20</test> >BaseXClient.rb:110:in `sendCmd': "test-newname.xml.xml" (Line 1): >Premature end of file. (RuntimeError) > > from commit-hooks/git/server-side/BaseXClient.rb:64:in `add' > from commit-hooks/git/server-side/post-receive:80:in `block in update' > from commit-hooks/git/server-side/post-receive:74:in `each' > from commit-hooks/git/server-side/post-receive:74:in `update' > from commit-hooks/git/server-side/post-receive:111:in `block in <main>' > from commit-hooks/git/server-side/post-receive:103:in `each' > from commit-hooks/git/server-side/post-receive:103:in `<main>' >Eliots-MBP:hooks ekimber$ > >A couple of things here: > > >Where is the extra ".xml" in the target filename coming from? > >What is causing the premature end of file? It feels like it's trying >interpret the second argument as a filename rather than the data to be >loaded. > >If I use basex.execute("add to #{path} #{bytes}") it works but of course I >get duplicate files if I run the command twice. > >If I try: > >@basex.execute("replace #{path} #{bytes}") > >Then I get the same failure. > > >So something is not right. > >My Docker container is running 8.4.1 beta. > >What am I missing? > >Thanks, > >Eliot >---- >Eliot Kimber, Owner >Contrext, LLC >http://contrext.com > > > >

2 5

Setting base-dir for xquery:eval
by Marc van Grootel 19 Feb '16

19 Feb '16

Hi, I am having quite some fun with xquery:eval. I am working on a little module for executable documentation (similar to Python doctest). I want to write documentation in asciidoc, parse it and execute source code blocks inside the asciidoc. Parsing works by calling asciidocj and so far I can run simple queries, evaluate them, compare them with an expected outcome and write the result back as evaluated asciidoc. Now I want to tackle more advanced queries that import modules. But when running xquery:eval with a string it will try to resolve paths relative to the code module and not relative to the file the string came from. Ideally I would like to be able to set the base-dir for xquery:eval. The asciidoc file isn't always in the correct path relative to the module that I need to import. Other options I'm considering but are less ideal. - Manipulate the module import paths in the query string using regexps - Writing out to a temp file and then use xquery:parse-uri (but this is tricky as I may not want to clutter existing directories with tempfiles only to have the imports work). Unless there are better options I think I'll opt for string massaging using regexps. But maybe someone has a better idea. --Marc

3 5

Sending Bytes, Not Strings, To BaseX Using the Ruby Client
by Eliot Kimber 18 Feb '16

18 Feb '16

I'm implementing server-side git hooks for use in GitLab under Docker where Java is not available (at least that I can see). The hooks load or delete files from databases in BaseX. I'm trying to implement the hooks in Ruby (which is much more pleasant than bash scripting in any case) and I'm using the BaseXClient.rb from https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby I need to create or replace files by sending the bytes--I'd rather not read the input file into a Ruby string and send that since I don't trust Ruby to not hose up the data (even when it's UTF-8 I still don't trust it, but I only started using Ruby yesterday so maybe my mistrust is misplaced?). Using the AddExample.rb as guide, I'm doing this: (Earlier code to open or create database, which works). file = File.new("../../" + path, "rb") bytes = file.read file.close puts "file=/#{bytes}/" @basex.add(path, "#{bytes}") I also tried: @basex.add(path, bytes) And I get this result (I added some debugging messages to sendCmd()): ensureDatabase(): Checking database "_dfst^metadata^temp^master"... BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53 ms. Added or modified file: "test-newname.xml" file=/<test>This is a test 20</test> / *** sendCmd(): cmd= arg=test-newname.xml input=<test>This is a test 20</test> BaseXClient.rb:110:in `sendCmd': "test-newname.xml.xml" (Line 1): Premature end of file. (RuntimeError) from commit-hooks/git/server-side/BaseXClient.rb:64:in `add' from commit-hooks/git/server-side/post-receive:80:in `block in update' from commit-hooks/git/server-side/post-receive:74:in `each' from commit-hooks/git/server-side/post-receive:74:in `update' from commit-hooks/git/server-side/post-receive:111:in `block in <main>' from commit-hooks/git/server-side/post-receive:103:in `each' from commit-hooks/git/server-side/post-receive:103:in `<main>' Eliots-MBP:hooks ekimber$ A couple of things here: Where is the extra ".xml" in the target filename coming from? What is causing the premature end of file? It feels like it's trying interpret the second argument as a filename rather than the data to be loaded. If I use basex.execute("add to #{path} #{bytes}") it works but of course I get duplicate files if I run the command twice. If I try: @basex.execute("replace #{path} #{bytes}") Then I get the same failure. So something is not right. My Docker container is running 8.4.1 beta. What am I missing? Thanks, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

1 0

Where Is Documentation On Module Lookup?
by Eliot Kimber 17 Feb '16

17 Feb '16

I'm trying to find the documentation on how BaseX looks up modules and I can't find it in the current documentation. I know I found it in the past. Where should I be looking? Thanks, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

2 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk