- BaseX-Talk - mailman.uni-konstanz.de

Cant add raw files explicitly or text index ms office docs
by E. Wray Johnson 26 Nov '15

26 Nov '15

Here is what I want to do: For a given folder and all its subfolders on my physical dive, mirror its contents including the contents of archives, parsing xml, json,html, text, etc. using their respective parser skipping invalids, and adding all other files as raw. I want archive files (*.zip, *.doxc) to be added as raw, however I want the text inside archive files like docx (ms-word) to be indexed and any files in the archives files that match a filter to be indexed. Note: It would be nice if there was a single db:add method that allowed me to specify a map of filters to parsers with options, where all files that do not match a filter (or are invalid) will be optionally added as raw.

3 3

file:read-binary performance
by Jérôme Chauveau 25 Nov '15

25 Nov '15

Hi, My restxq app is very slow to deliver raw files (css, images, js). For example, the restxq method takes around 600ms to load a png file in my browser when it takes only 2ms in a "direct" or "static" way. --------------------------------------- declare %rest:path("/editions/static/{$project}/{$file=.+}") function max.file:file( $project as xs:string, $file as xs:string ) as item()+ { let $path := file:parent(file:base-dir()) || 'editions/' || $project || '/ui/' || $file return ( web:response-header(map { 'media-type': web:content-type($path) }), file:read-binary($path) ) }; --------------------------------------- Why such a difference? How can I improve it? I am using BaseX 8.3 (war version). Regards, Jérôme -- Jérôme Chauveau - CERTIC-DSI - Campus I Université de Caen Normandie

2 2

Assigning incrementing number to 100ks of elements
by Chuck Bearden 24 Nov '15

24 Nov '15

BaseX 8.3.1, installed on Linux today from zip archive. I have about 137,000 target elements, each having an average of 550 descendant elements, and each taking about 45k of space on disk. The target elements each have an @uuid value, and I'd like to extract each @uuid value and assign it a unique number from an incrementing sequence of integers. I'd like to be able to start the sequence at an arbitrary value. I tried a recursive function I borrowed and modified from a StackOverflow post: declare function local:loop($seq, $count) { if(empty($seq)) then () else let $prod := $seq[1], $count := $count + 1 return ( <result> <id>{$count}</id> <uuid>{data($prod)}</uuid> </result>, local:loop(tail($seq), $count) ) }; and calling it like so: return <results>{ local:loop($pubs, 0) }</results> where $prod is a sequence of all the @uuid attributes (not the values, and not the whole elements). My query fails with the error: [bxerr:BASX0005] Stack Overflow: Try tail recursion? I'm using -Xmx6g and -Xss4m as flags for Java in the basexgui script. Is there some way to get BaseX to use tail recursion? Or is there another approach to doing what I want? Many thanks, Chuck

2 1

issue with stack trace in query-info window
by Rob Stapper 24 Nov '15

24 Nov '15

Hi Christian, I found that in case of a runtime error the links in the stack trace in the Query info window to the different locations in the program don't work anymore. I use of course the latest snapshot: BaseX 8.4 beta 3b6f6e0 Regards, Rob --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 1

basex performance for high request number
by Filippo Gaudenzi 23 Nov '15

23 Nov '15

Hi all, I’m running some scripts to solve a NP-problem in order to check trees compatibility and i’m using Basex as a database for my hierarchy structure, indeed for each path/each node i run a query on the basexserver. This brings to have for a tree with 7 paths a 7! permutations and therefore at least 7!*4=21.000 queries (the query below) in a lapse of 5minutes. If i use the db in my localhost environment i see that basexserver goes over 150% of cpu usage. I was wondering how to increase the performance without causing the server crash. basex is running on a 2,6 GHz Intel Core i5 with 16gb of Ram, OS X 10.10.5 basex version is 8.2 Here the code i use: public class BaseXOntologyManager{ private final int PORT; private final String PASSWORD; private final String USERNAME; private final String ADDRESS; private final String DBNAME; /** * Constructor of the Basex ontology Manager * @param address The address of the DB * @param port The port to use to connect to the DB * @param username The username to use to connect to the DB * @param password The password to use to connect to the DB * @param dbName The db Name */ public BaseXOntologyManager(String address, int port, String username, String password, String dbName){//,boolean connected) { this.ADDRESS = address; this.PORT = port; this.USERNAME = username; this.PASSWORD = password; this.DBNAME = dbName; } private BaseXClient getOpenConnection() { try { BaseXClient session = new BaseXClient(this.ADDRESS, this.PORT, this.USERNAME, this.PASSWORD); return session; } catch (IOException e) { return null; } } /** * Return the subClasses of the element based on the ontology in the DB, * if the direct is true returns only the parents otherwise also the descendants * @param keyName The name of the Ontology to use * @param element the element to search in the param * @param direct true if you want only the parents * @return A list of subClasses of the ontology * @throws Exception when element doesn't belong to Ontology */ public ArrayList<String> getSubClasses(String keyName, String element, boolean direct) throws Exception { BaseXClient client = this.getOpenConnection(); if (client != null) { if(!checkExistence(element)) throw new Exception("notFound"+element); ArrayList<String> result = new ArrayList<String>(); String input = null; if (direct == true) { input = "for $n in db:open('" + this.DBNAME +"', '/" + keyName + "')//*[parent::" + element + "] return $n/name()"; } else { input = "for $n in db:open('" + this.DBNAME +"', '/" + keyName + "')//*[ancestor::" + element + "] return $n/name()"; } try { final BaseXClient.Query query = client.query(input); while (query.more()) { result.add(query.next()); } query.close(); result.add(element); return result; } catch (IOException e) { return null; } finally { try { client.close(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); } } }else{ System.out.println("CLIENT NULL"); } return null; } Thanks, Filippo

2 2

issue with the latest snapshot
by Rob Stapper 23 Nov '15

23 Nov '15

Gute Morgen, After installing the latest Basex snaphot (8.4) this morning, the new version couldn't resolved function calls to function that are located in imported modules while compiling. I get an error messages: "Unknown function '...'; similar: '...' ." After installing the previous version (8.3) everything worked fine again. I'm using windows 7 and basex is installed on my L:-drive. Hope this is enough information. Rob Stapper --- Dit e-mailbericht is gecontroleerd op virussen met Avast antivirussoftware. https://www.avast.com/antivirus

2 12

Curious "Stack Overflow: Try tail recursion?" error message
by Andy Bunce 20 Nov '15

20 Nov '15

Hi, Using a fresh install on 8.3.1 and running basexgui.bat on windows. I install this expath package https://github.com/expkg-zone58/ex-xparse/releases/download/v0.0.5/ex-xpars… Then, paste the following code into an editor window.. ------------------------ import module namespace xp="http://expath.org/ns/xparse"; "http://www.xqueryfunctions.com/xq/functx-1.0-doc-2007-01.xq" !unparsed-text(.) !xp:parse(.,map{"lang":"xquery"}) ----------------------- The result is the error... Stack Overflow: Try tail recursion? The curious part is.. if I keep retrying by clicking the execute button - after around 10 tries it starts working and seems to continue to work every time after that within that GUI instance. I have reproduced this on another pc. /Andy

2 2

Re: [basex-talk] Counting Occurences
by Christian Grün 20 Nov '15

20 Nov '15

Hi Ana (cc to the list), > I've tried using the "group by" clause, however, it only returns to me the > list of the present provinces, without the frequence, or, if I use the > "count" function, in only returns 1 (Since every curriculum on the > collection only contains a single occurence the province) Did you combine group by and count (as shown in the Wiki)? How does your current query look like? Christian > > Do you have any idea of how could I list the frequence of each province? > > Thank you for your help. > > 2015-11-20 15:17 GMT-02:00 Christian Grün <christian.gruen(a)gmail.com>: >> >> Hi Ana, >> >> The "group by" clause will help you here [1]. Hope this helps; feel >> free to ask if you need more examples. >> >> Christian >> >> [1] http://docs.basex.org/wiki/XQuery_3.0#group_by >> >> >> On Fri, Nov 20, 2015 at 6:06 PM, Ana Victória Ladeira >> <ladeira.ana(a)gmail.com> wrote: >> > Good afternoon! >> > >> > I have a collection of XML files in which all files contain the >> > attribute >> > //ENDERECO-PROFISISONAL/@UF >> > >> > In which UF is the province in which the person to whom the file makes >> > reference to lives. >> > >> > I want to develop a query that returns to me the number of occurences of >> > each province in the collection. I've tried using count and >> > distinct-values, >> > but it always return the value "1" for me. >> > >> > The query I'm trying to use is the following: >> > for $x in >> > >> > collection('Teste500')/CURRICULO-VITAE/DADOS-GERAIS/ENDERECO/ENDERECO-PROFISSIONAL >> > let $y := count(distinct-values($x/@UF)) >> > return $y >> > >> > The desired result would be, in a collection of documents in which 5 >> > people >> > live in SP, 3 in DF and 9 in AM would be the following: >> > >> > SP: 5 >> > DF: 3 >> > AM:9 >> > >> > Can somebody please help me with that? I'm still struggling with xQuery. >> > >> > > >

2 1

Counting Occurences
by Ana Victória Ladeira 20 Nov '15

20 Nov '15

Good afternoon! I have a collection of XML files in which all files contain the attribute //ENDERECO-PROFISISONAL/@UF In which UF is the province in which the person to whom the file makes reference to lives. I want to develop a query that returns to me the number of occurences of each province in the collection. I've tried using count and distinct-values, but it always return the value "1" for me. The query I'm trying to use is the following: for $x in collection('Teste500')/CURRICULO-VITAE/DADOS-GERAIS/ENDERECO/ENDERECO-PROFISSIONAL let $y := count(distinct-values($x/@UF)) return $y The desired result would be, in a collection of documents in which 5 people live in SP, 3 in DF and 9 in AM would be the following: SP: 5 DF: 3 AM:9 Can somebody please help me with that? I'm still struggling with xQuery.

2 1

Applicative Cache
by Marco Lettere 20 Nov '15

20 Nov '15

Hi all, I went though the emails talking about mainmemory databases and related but I've not been able to find an answer to this loud thinking ... Would it be possible to create an in memory database with a RESTXQ call and keep it accessible for other RESTXQ invocations as if it was a disk db throughout the lifecycle of a basexhttp server? Application level cacheing of (possibly distributed) dbs or management of transient volatile data items like tokens are the scenarios I have in mind. Just wondering whether this makes any sense. Whether it's feasible or whether someone already did this with other means. Thanks a lot! Marco.

3 6