BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5250 discussions

Profiling queries
by Iwan Briquemont 25 May '17

25 May '17

Hello, I was wondering if there are ways to access how much time is spent in parts of a query/program, to find out inefficient functions or expressions? Something like the total time spent in functions would be quite useful. I saw the Profiling module, but it requires you to change the code and it also won't aggregate results when you run it in a loop. It's still doable if you add labels and then do some parsing on stderr to aggregate the results yourself, but it's quite cumbersome. Some thoughts as a side note: A nice way to avoid having to change the code to do profiling (or other debugging like breakpoints) would be to change or add hooks into the AST of the query. That way you could for example add a prof:time() around all the function bodies, with the function name as label. And to help interpret the results of debugging functions like prof:time or trace, it would be nice to have a way to format them and output them as a stream of items (or even multiple streams). Then you could create XQuery scripts that consume the streams to aggregate the information in a human readable way. Best regards, Iwan

2 3

Approximating ML's Reverse Queries Mechanism?
by Eliot Kimber 23 May '17

23 May '17

I’m working on a project that currently depends on MarkLogic’s reverse query mechanism. This is a feature whereby you store documents that contain MarkLogic-specific queries (queries in their cts namespace). These get indexed in some way (no idea how the index works). You can then search for these stored queries in the context of specific nodes and ML will return those query documents that would match the specified nodes. Their driving use case is alerting-type applications where when new docs get added you see what queries they apply to and then use those queries to do something. My use case is classification: given a node to be classified, find all queries that match it and from the query get the classification details (preferred term, variant forms, associated taxonomy, etc.). This process definitely depends on MarkLogic’s full-text search features, for example to match any form of a non-preferred term to a full-text search that would match it. I have a large corpus to classify (approximately 45 million objects at current count). The processing is inherently parallelizable so I’m looking at non-ML options that would allow us to scale to the limits of our hardware budget. Even if each node was less efficient than ML we would be able to implement massive throughput for this classification operation. In theory we could scale to one processor per object if budget were unlimited (it is not unlimited), so even a brute-force solution would perform well at larger scales. So I guess I have two questions really: 1. Can anyone share ways they use BaseX for doing classification in general? I’ve so far just been focused on analyzing the current system to find performance bottlenecks so I haven’t yet had a chance to think through the classification process in general, but there must be well-understood strategies. I suspect that one could build the equivalent of ML’s reverse query index in BaseX. 2. Is there a direct equivalent to ML’s reverse query facility in BaseX or an obvious route to building one? Thanks, Eliot -- Eliot Kimber http://contrext.com

1 0

Querying documents by a full path pattern
by Jacob Borisov 19 May '17

19 May '17

Hello. Is it possible to select documents in a database by a path pattern? For example, there is a database storing following documents: dir1/a.xml dir1/b.xml dir2/a.xml dir2/b.xml I want to select documents named 'b.xml' only. I guess something like db:open('dbname', '*/b.xml')/* but it returns empty result. Thanks!

2 1

BaseX 8.6.4
by Christian Grün 18 May '17

18 May '17

Dear all, Another patch release of BaseX release is available. It includes one critical fix for fn:tail. If you use this function, an update is definitely recommended. Cheers, Christian

1 0

Re: [basex-talk] Could not reserve enough space for object heap
by Christian Grün 17 May '17

17 May '17

Dear Jim, Did you already try to change/reduce the value of the -Xmx flag in the basexhttp.bat start script? Cheers, Christian Am 17.05.2017 00:40 schrieb "Jim Saiya" <jsaiya(a)formatdata.com>: I'm having the same issue Cong Hui has described in this Stack Overflow question: http://stackoverflow.com/q/43331077/4852675 Namely, that I get this error when trying to run basexhttp.bat ---- Error occurred during initialization of VM Could not reserve enough space for 1433600KB object heap ---- My system is running Windows 10 Home version 1703 build 15063.296 (just updated), with 4.00 GB RAM Is there a solution? *Jim Saiya* XML Analyst Developer Detroit, MI ✆ *703.731.5119* ✉ jsaiya(a)formatdata.com [in] www.linkedin.com/in/jimsaiya

2 2

Could not reserve enough space for object heap
by Jim Saiya 17 May '17

17 May '17

I'm having the same issue Cong Hui has described in this Stack Overflow question: http://stackoverflow.com/q/43331077/4852675 Namely, that I get this error when trying to run basexhttp.bat ---- Error occurred during initialization of VM Could not reserve enough space for 1433600KB object heap ---- My system is running Windows 10 Home version 1703 build 15063.296 (just updated), with 4.00 GB RAM Is there a solution?   Jim Saiya XML Analyst Developer Detroit, MI ✆ 703.731.5119 ✉ jsaiya(a)formatdata.com [in] www.linkedin.com/in/jimsaiya

1 0

lib/custom directory
by Christian Grün 16 May '17

16 May '17

Hi Andy, hi Vincent, I have just added a custom directory to the ZIP and EXE distributions [1,2]. The Windows distribution is still work in progress, because I need to find a way to dynamically import all available libraries [3]. Your testing feedback is welcome, Christian [1] https://github.com/BaseXdb/basex/issues/1456 [2] http://files.basex.org/releases/latest/ [3] https://github.com/BaseXdb/basex/issues/1457 On Fri, May 12, 2017 at 12:51 AM, Lizzi, Vincent <Vincent.Lizzi(a)taylorandfrancis.com> wrote: > I like the idea of having a “custom” lib folder, also for the .zip > distribution. That would make updating to new versions of BaseX a little > easier. > > > > Sent from my Windows 10 phone > > > > From: Christian Grün > Sent: Thursday, May 11, 2017 4:09 PM > To: Andy Bunce > Cc: basex-talk(a)mailman.uni-konstanz.de > Subject: Re: [basex-talk] Java Modules - Constructor > > > >> In fact I have modified the bat to add a "lib2" folder where I can easier >> manage extension jars. >> >> set CP=%MAIN%/BaseX.jar;%MAIN%/lib/*;%MAIN%/lib2/* > > Good point. For the Windows executable, I thought about adding a > 'custom' sub-directory in the lib folder. The jars in the lib > directory could be deleted with each update, but the custom folder > would be preserved. > >> But is the original problem a bug? >> That is after installing the jar via REPO install t:new("aaa") fails >> although it works when added to lib/ > > Be sure I’ll check this out later this week. >

1 0

Configuring basexgui - keeping directory context
by Bridger Dyson-Smith 15 May '17

15 May '17

Hi all -- apologies for the strange title: I'm not quite sure how to word my question. I'm having a problem with maintaining the correct context of basexgui, particularly with testing some .bxs files. I have a directory where I've been keeping my BaseX installs: /usr/home/bridger/src/basex = $baseXhome. I've been in the habit of starting new tests and projects in the repo directory in $baseXhome; e.g. $baseXhome/repo/new-test. In the new-test directory, I have the following: new-test/ data/ resources/ add.xq .basex .basexgui .basexhome check.bxs add.bxs check.bxs is a simple script: CHECK foo RUN add.bxs CLOSE add.bxs runs an xquery script: RUN resources/add.xq and the `add.xq` is (surprisingly) another simple script: let $msg := file:current-dir() return $msg The problem I'm having is that, unless I `cd ~/src/basex/repo/new-test && basexgui`, `basexgui` isn't able to determine the relative file path(s) of any of the secondary scripts (add.bxs and resources/add.xq). While it wouldn't be a terrible burden to launch the GUI from a new terminal (thank heavens for multiplexers!), the problem there is that basexgui seems to default to (re)opening the last directory where it was launched; in other words 1. `cd ~/src/basex/repo/new-test && basexgui` and the GUI launches and things look great. 2. in a new terminal window: `cd ~/src/basex/repo/old-test && basexgui` and the GUI launches and opens the new-test directory. So, I guess that's two problems, but they're both somewhat related to the relative directory path context of the GUI. I've tried adding .basexhome (manually) and .basexgui (manually and letting the GUI add them automatically), but those files don't seem to help matters. Is there anything else that I can try to resolve this? I know that I can use absolute file paths, but I have a simple project that I want to share with a colleague. Is the solution to add some XQuery that will determine file paths, and add/insert that into the command .bxs files? Thanks so much for your time and trouble. Best, Bridger

2 3

Re: [basex-talk] Java Modules - Constructor
by Andy Bunce 15 May '17

15 May '17

Great. I believe the windows exe has a hard coded class path so ignores any additions. /Andy On 8 May 2017 at 13:24, kovacic <kovacic(a)dke.uni-linz.ac.at> wrote: > Thank you Andy. It works by starting the GUI using the basexgui.bat. > > Regards > Ilko > > Am 08.05.2017 um 14:21 schrieb Andy Bunce: > > > 1. download + unzip http://files.basex.org/releases/8.6.3/BaseX863.zip > 2. copy hello-test.jar to lib/ > 3. run bin/basexgui.bat > 4. execute: > > declare namespace t='org.basex.modules.Hello'; > let $obj := t:new('works for me') > return t:hello($obj) > > Regards > /Andy > > On 8 May 2017 at 13:16, kovacic <kovacic(a)dke.uni-linz.ac.at> wrote: > >> Hi Andy, >> >> I followed the instructions from here http://docs.basex.org/wiki/Rep >> ository#Java. >> >> I copied the jar to the basex/lib/ folder but it still does not work. >> Which version of BaseX are you using? >> >> Regards >> Ilko >> >> Am 08.05.2017 um 13:43 schrieb Andy Bunce: >> >> Hi Ilko, >> >> >The repo folder was completely empty. >> So you want to install it in the repository, using repo install? >> >> My test was installing via copying the jar to basex/lib/ where it works >> (for me). >> If I use repo install I get the same problem as you. >> >> This may be a bug/feature. >> It is possible that using the full EXPath Packaging [1] xar would work >> although I have not tried it. >> This would involve adding an XQuery file with expath-pkg.xml and >> basex.xml files to create a xar wrapping the jar, structured as here [2] >> The issue [3] may be relevant. >> >> Regards >> /Andy >> >> hello-test.jar >> <https://drive.google.com/file/d/0B_HWsgek7hAeTjFnSFg3cGVpMXc/view?usp=drive…> >> >> [1] http://docs.basex.org/wiki/Repository#Java_2 >> [2] https://github.com/expkg-zone58/metadata-extract/tree/master/src/main >> [3] https://github.com/BaseXdb/basex/issues/1413 >> >> >> >> On 8 May 2017 at 11:44, kovacic <kovacic(a)dke.uni-linz.ac.at> wrote: >> >>> Hi Andy, >>> >>> I have removed all other existing modules before installing this module. >>> The repo folder was completely empty. >>> >>> Regards >>> Ilko >>> >>> Am 05.05.2017 um 19:29 schrieb Andy Bunce: >>> >>> Hi IIko >>> >>> Your code and: >>> >>> declare namespace t='org.basex.modules.Hello'; >>> let $obj := t:new('works for me') >>> return t:hello($obj) >>> >>> Returns: >>> >>> Hello works for me >>> >>> Could be be that you have multiple versions of the jar on the path? >>> >>> Regards >>> /Andy >>> >>> On 5 May 2017 at 11:20, kovacic <kovacic(a)dke.uni-linz.ac.at> wrote: >>> >>>> Hi all, >>>> >>>> I am running BaseX 8.6.3 currently and struggling with the inclusion of >>>> modules written in Java. I have extended the Hello World example: >>>> >>>> package org.basex.modules; >>>> >>>> public class Hello { >>>> >>>> private String str = "BaseX"; >>>> >>>> public Hello() {} >>>> >>>> public Hello(String str) { >>>> this.str = str; >>>> } >>>> >>>> public String hello(final String world) { >>>> return "Hello " + world; >>>> } >>>> >>>> public String hello() { >>>> return "Hello " + str; >>>> } >>>> } >>>> >>>> I can pack it into a jar-file and install it successfully. My jar-file >>>> is structured as following: >>>> >>>> hello.jar >>>> - META-INF >>>> - MANIFEST.MF >>>> - org >>>> - basex >>>> - modules >>>> - Hello.class >>>> >>>> Here is the content of my manifest.mf: >>>> >>>> Manifest-Version: 1.0 >>>> Main-Class: org.basex.modules.Hello >>>> >>>> The methods of the Hello-class can be called without any problems when >>>> the module is imported: >>>> >>>> import module namespace t='org.basex.modules.Hello'; >>>> >>>> t:hello('text') >>>> >>>> However, when I try to explicitly create the object by calling the >>>> constructor (new) with arguments it fails: >>>> >>>> declare namespace t='org.basex.modules.Hello'; >>>> >>>> let $obj := t:new('test') >>>> >>>> return '' >>>> >>>> When I try to call the constructor I receive the error [XPST0017] >>>> Unknown function: Q{org.basex.modules.Hello}new. >>>> >>>> Since I am able to execute the FileWriter example ( >>>> http://docs.basex.org/wiki/Java_Bindings#Namespace_Declarations) which >>>> calls a constructor with an argument I would like to know if this behavior >>>> is a bug or if I am missing an important point. >>>> >>>> Regards, >>>> Ilko >>>> >>>> >>>> >>> >>> -- >>> *Ilko Kovacic* >>> Wissenschaftlicher Mitarbeiter >>> Institut für Wirtschaftsinformatik - Data & Knowledge Engineering >>> >>> >>> *JOHANNES KEPLER UNIVERSITÄT LINZ* >>> Altenberger Straße 69 >>> Science Park III, S3 111 >>> 4040 Linz, Österreich >>> T +43 732 2468 4286 <+43%20732%2024684286> >>> ilko.kovacic(a)jku.at >>> www.dke.jku.at >>> >> >> >> -- >> *Ilko Kovacic* >> Wissenschaftlicher Mitarbeiter >> Institut für Wirtschaftsinformatik - Data & Knowledge Engineering >> >> >> *JOHANNES KEPLER UNIVERSITÄT LINZ* >> Altenberger Straße 69 >> Science Park III, S3 111 >> 4040 Linz, Österreich >> T +43 732 2468 4286 <+43%20732%2024684286> >> ilko.kovacic(a)jku.at >> www.dke.jku.at >> > > > -- > *Ilko Kovacic* > Wissenschaftlicher Mitarbeiter > Institut für Wirtschaftsinformatik - Data & Knowledge Engineering > > > *JOHANNES KEPLER UNIVERSITÄT LINZ* > Altenberger Straße 69 > Science Park III, S3 111 > 4040 Linz, Österreich > T +43 732 2468 4286 <+43%20732%2024684286> > ilko.kovacic(a)jku.at > www.dke.jku.at >

3 7

Broken URLs for RPM-based repos
by Jacob Borisov 15 May '17

15 May '17

Hello. I've found that all the URLs for rpm-based repositories listed at http://basex.org/products/download/all-downloads/ are broken. For example, http://download.opensuse.org/repositories/home:/basex:/opensuse/CentOS_7/ho… URL returns "Object not found!" error.

2 5

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk