BaseX-Talk June 2015

basex-talk@mailman.uni-konstanz.de

28 participants
43 discussions

Module parsing performance
by Alexander Shpack 22 Jun '15

22 Jun '15

Hi BaseX team! How to increase performance of parsing and executing on custom xqm module? I have 10K lines of sxq code and it makes huge CPU load on the parsing process. Any ideas? -- s0rr0w

2 5

Fwd: Modifying CSV node
by Noam Green 22 Jun '15

22 Jun '15

Sorry - Just noticed I sent it to Christian only instead of to the group as well. Resending. ==== Hi Christian, Thanks for the quick reply. The attacked CSV is very small. I've also included the test query file. As mentioned, I'd like to encrypt the CK field (so that it's saved to the file as encrypted). Thanks, Noam On Mon, Jun 22, 2015 at 11:30 AM, Christian Grün <christian.gruen(a)gmail.com> wrote: > Hi Noam, > > Could you please provide us with a self-contained example (the > smaller, the better [1])? > > Thanks in advance, > Christian > > [1] http://sscce.org/ > > > > > > > On Mon, Jun 22, 2015 at 10:09 AM, Noam Green <green.noam(a)gmail.com> wrote: > > Hi all, > > > > I'm using the following xquery (after being helped by Christian). > > > > declare variable $in external; > > declare variable $out external; > > declare variable $vendor external; > > > > let $options := map { 'header': true() } > > let $text := file:read-text($in) > > let $xml := csv:parse($text, $options) > > > > let $result := <csv>{ > > for $record in $xml//record[contains(VENDORS, $vendor) and > > not(contains(HOST, "Non external")) and not(empty(HOST))] > > > > return <record>{ > > > > $record/(HOST, PATH, ATTACKCOUNT, TIME_STAMP,CK) > > }</record> > > > > }</csv> > > return file:write-text($out, csv:serialize($result, $options)) > > > > I would like to have one of the returned records encrypted. How do I > change > > that record before saving it? > > I tried: > > return <record>{ > > > > crypto:encrypt($record/CK, 'symmetric','keykeyke','DES'), > > $record/(HOST, PATH, ATTACKCOUNT, TIME_STAMP,CK) > > }</record> > > > > But that doesn't really change the CK field. I can't seem to modify > anything > > in the $record/(HOST, PATH, ATTACKCOUNT, TIME_STAMP,CK). > > > > Any thoughts? > > Thanks, > > Noam >

2 2

Modifying CSV node
by Noam Green 22 Jun '15

22 Jun '15

Hi all, I'm using the following xquery (after being helped by Christian). declare variable $in external; declare variable $out external; declare variable $vendor external; let $options := map { 'header': true() } let $text := file:read-text($in) let $xml := csv:parse($text, $options) let $result := <csv>{ for $record in $xml//record[contains(VENDORS, $vendor) and not(contains(HOST, "Non external")) and not(empty(HOST))] return <record>{ $record/(HOST, PATH, ATTACKCOUNT, TIME_STAMP,CK) }</record> }</csv> return file:write-text($out, csv:serialize($result, $options)) I would like to have one of the returned records encrypted. How do I change that record before saving it? I tried: return <record>{ crypto:encrypt($record/CK, 'symmetric','keykeyke','DES'), $record/(HOST, PATH, ATTACKCOUNT, TIME_STAMP,CK) }</record> But that doesn't really change the CK field. I can't seem to modify anything in the $record/(HOST, PATH, ATTACKCOUNT, TIME_STAMP,CK). Any thoughts? Thanks, Noam

2 1

A question about context for ancestor::*
by France Baril 22 Jun '15

22 Jun '15

In the following example, both $yo and $yo2 are obtained in the context of $all, but $yo2 is interpreted in the context of $t2b. let $t2a := <topic><title id="yo">2.1</title></topic> let $t2b := db:open('test')/*[@id='topic'] let $all := <map><title>Map</title>{$t2a}{$t2b}</map> let $yo := $all//*[@id='yo'] let $yo2 := $all//*[@id='yo2'] return ($all, count($yo/ancestor::*), count($yo2/ancestor::*)) returns <map> <title>Map</title> <topic> <title id="yo">2.1</title> </topic> <topic id="topic"> <title id="yo2">2.2</title> </topic> </map> 2 1 I would have expected both ancestor::* clauses to return 2. If I ask for the last ancestor of $yo2, I get <topic id="topic">, again wrong context. The only way for me to force $yo2 to pick the node from the context of $all is to create a copy of $all, and then work from the copy. Why would yo get into context, but not yo2?

2 1

Can not use a foreign namespace function in BaseX
by Robert Berger 22 Jun '15

22 Jun '15

I go to http://www.xqueryfunctions.com/ and download functx-1.0-doc-2007-01.xq I open that xq file in the editor and run it (Green Arrow). But if I try to use any function it fails !! [image: Inline image 1] Thanks, Bob Berger

2 1

Performing queries with a local function takes too much time
by serdar ceran 20 Jun '15

20 Jun '15

Hi, BaseX team. I have a problem; Performing queries with a local function takes too much time to get the result. Please have a look at the Stackoverflow <http://stackoverflow.com/questions/30865291/basex-query-optimization-on-join> for more details. Best regards; Serdar CERAN

2 2

Re: [basex-talk] Strange path problem (revisited)
by James Ball 18 Jun '15

18 Jun '15

Kristian, It’s interesting your raise this - I was having trouble just yesterday working out why my colleague kept getting locked out of the server (I have fail2ban installed). Turns out he was going to http://ourdomain.com/dba/ (with a trailing slash) It all works if you go to http://ourdomain.com/dba (without a trailing slash) From my testing it looks like ‘dba’ and ‘dba/‘ both get caught by %rest:path(‘/dba’) which is redirected to ‘dba/databases’. However this means without the slash the browser interprets this as a redirect to http://ourdomain.com/dba/databases With the slash it is interpreted as a redirect to http://ourdomain.com/dba/dba/databases (which doesn’t exist and so after some attempts he was blocked). I only tested briefly yesterday but I couldn’t see immediately how to fix this (adding another %rest:path('/dba/') didn’t seem to work. But then I don’t know if the regex support affects that? Interested to hear others’ experience. Regards, James > > Message: 1 > Date: Mon, 01 Jun 2015 18:57:06 +0300 > From: Kristian Kankainen <kristian(a)keeleleek.ee> > To: BaseX <basex-talk(a)mailman.uni-konstanz.de> > Subject: [basex-talk] Strange path problem (revisited) > Message-ID: <556C80D2.1050101(a)keeleleek.ee> > Content-Type: text/plain; charset=utf-8; format=flowed > > Hello! > > I noticed that I get different results if I visit the following two urls: > http://myhost.domain/webappname/ > http://myhost.domain/webappname > > The only difference is the trailing slash. My RESTXQ has a path for '/' > but not for ''. My setup is with Tomcat. The difference seems to be the > basepath for the webapp. Static files etc are not found. This also > applies for the BaseX DBA: > > http://myhost.domain/webappname/dba/login resolves correctly > http://myhost.domain/webappname/dba/login/ doesn't find it's static > files etc. > > For my own webapp, it's the other way around --- with the trailing slash > everything works fine, without it the static files etc aren't found. > > Is it yet another Tomcat problem or can it be reproduced on your setups? > > Cheers! > Kristian

3 2

performance of preceding/following axis
by Schopper, Daniel 18 Jun '15

18 Jun '15

Hi, I'm trying to use BaseX for linguistic queries on a TEI document containing annotated tokens (i.e. tei:w-elements with attributes). I'm specifically interested in distance queries that allow to search for combinations of token features within a given window (e.g. all nouns that have an adjective ending with 'lein' within a distance of 3.) Theoretically, this is rather easy to formulate with an XPath or XQuery expression, but performance is poor when the dataset gets a bit larger (in my case, I have a total of 190.000 tokens in my test document, attribute and text indexes created). This is what I essentially try to do as a simple XPath: declare default element namespace "http://www.tei-c.org/ns/1.0"; //w[@type = "NN"][(subsequence(preceding::w, 1, 3), subsequence(following::w, 1, 3))/@type = "ADJA"] Since tokens may be interwoven with markup, I have to use preceding::* or following::* A simple XQuery returning all matches including their context would look like this: declare default element namespace "http://www.tei-c.org/ns/1.0"; let $window := 3 let $matches := //w[@type = "NN"] return for $m in $matches let $pre := subsequence($m/preceding-sibling::w, 1, $window) let $next := subsequence($m/following-sibling::w, 1, $window) return if (($pre,$next)[@type = "ADJA"]) then <conc> <pre>{$pre}</pre> <match>{$m}</match> <next>{$next}</next> </conc> else () With $matches being a sequence of ca. 50.000 elements, a FLOWR is a bit too costly, I fear; limiting $matches to ~ 1.000 items performs within 5600ms (returning 154 items), but performance decreases rapidly after that (not to speak about setting a larger distance) . So, my question is: Is there a way to improve performance on operations like these (without resorting to changing the input document)? I'd be glad to provide my dataset off list, if this helps. Thanks, Daniel

3 3

Gui
by meumapple 18 Jun '15

18 Jun '15

Hallo, I point out that In the last release, the number of repetions of a for clause is not displayed anymore on the top right corner of the GUI. This was very useful. Thanks, Joseph

2 1

Low/high caps...
by France Baril 17 Jun '15

17 Jun '15

Hi, I moved a project from Windows to Mac (might be relevant or not) and after working with content for a while, I went to the GUI to create backups. Some of the DBs appears as duplicated and are listed with different caps. Ex. - D-archie - D-Archie The DBs listed in the GUI have the same size, they have the same content when opened in, even the same list of available backups, they look like they point to the same repository. In fact, the data repository only shows 1 folder and db:list() returns no duplicate, just the original names with the original caps. The WebDAV data source connection in Oxygen shows no duplicate either. I'm afraid of deleting the duplicate from GUI, as there is only one match in the data folder. I'll look in my code for a place where I might have typed the name with the wrong caps (I'm guessing a db:add or db:replace operation with the wrong caps would be the source of the problem). The request is for GUI and xquery to handle things in a similar fashion on a given platform. Either the caps make a difference, which means that trying to save content in a DB that doesn't exists (different caps = different names = different DBs) returns an error, or GUI handles caps as irrelevant and shows only 1 instance of the DB. Regards, France P.S. I'm on 8.2. -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk June 2015