BaseX-Talk

basex-talk@mailman.uni-konstanz.de

13 participants
5244 discussions

BaseX 9.5.3
by Christian Grün 27 May '21

27 May '21

Dear all, A new BaseX patch release is available. It comes with minor bug fixes and little enhancements: • RESTXQ responses now come with a Server-Timing HTTP header (motivated by Reece) • A context file can be specified via the Bindings Dialog of the XQuery editor (motivated by Hans-Jürgen) • The current HTTP context will now be propagated if jobs:eval is called • The DBA CodeMirror and Jetty libraries have been updated Find the new version on our web site: https://basex.org/ Have fun, Christian

1 0

Running XQuery Update via apply Ant task
by Zimmel, Daniel 26 May '21

26 May '21

Hi, is there anyone using BaseX in an Apache Ant scenario? I have the following target for XQuery Updating some files (no database access), but I am too dumb to make an apply task that reads every file in a fileset. Usually you place the <srcfile/> where you need it to be in your external command, but unfortunately the BaseX command-line syntax won't allow this to be easily placed: the filename is supposed to follow directly suffixed to "-i" where I cannot use an element, of course. <target name="xquery_update_default"> <echo level="info" message="XQuery Update (Default): ${param.xquery-update-uri}"></echo> <apply executable="java" > <arg value="-cp"/> <arg value="BaseX95.jar"/> <arg value="org.basex.BaseX"/> <arg line="-u"/> <arg line="-w"/> <arg line="-c'set exporter omit-xml-declaration=no,indent=no'"/> <arg line="-b'path=${param.folder}'"/> <arg line="-i${dummy.xml}"/> <srcfile/> <fileset dir="../${dir.xquery}" includes="${param.xquery-update-uri}"/> </apply> </target> As a workaround, I set a dummy.xml for input and bind the current folder as a parameter and in my Query I can use file:list() which then is updating all files in the directory, but this makes the Query dependent on the parameter, and updates are obviously not written back unless all files have been processed in memory. I would prefer to have a standalone query operating only on the current document-uri (so I can better reuse it elsewhere). Can someone give me a hint if it is possible in plain Ant to apply a CCL-BaseX to any file in a given fileset? Other than that, BaseX performs well as a file processor (if you ignore the time related to file handling). Thanks, Daniel

1 1

BUG node comparison
by ydyxeb＠post.cz 26 May '21

26 May '21

Dear BaseX, according to github instructions I’m sending a bug report via email: Using BaseX 9.5.1, cmd-line “basex.bat -i datafile.xml queryfile.xq” leads to no result at all. I’m convinced there should be result equivalent to the one of the online xquery tester (https://www.videlibri.de/cgi-bin/xidelcgi) For a modified query using string comparison based on fn:path(), BaseX gives the expected result. %%% query begin %%% let $rootElement := /child::* for $x in //Section[parent::* != $rootElement] return fn:path($x) %%% query end %%% %%% data begin %%% <root> <Section id="should not be reported 1" /> <Section id="should not be reported 2"> <Section id="should be reported 1"/> <a> <Section id="should be reported 2"/> </a> </Section> <a> <Section id="should be reported 3"/> </a> </root> %%% data end %%% %%% modified query begin %%% let $rootElementPath := fn:path(/child::*) for $x in //Section[fn:path(parent::*) != $rootElementPath] return fn:path($x) %%% query end %%% Kind regards Jan Červák

3 5

CSV and automatic encoding detection
by Kristian Kankainen 24 May '21

24 May '21

Hi folks, I am aware that with the HTML module you can let it guess a file's encoding by itself by providing it in binary format: > If the input encoding is unknown, the data to be processed can be passed on in its binary representation. The HTML parser will automatically try to detect the correct encoding: > > Query > html:parse(fetch:binary("https://en.wikipedia.org")) But is there a way to guess encoding of CSV files? So far I have tried with the fetch and CSV module with no results. I have a huge bunch of CSV files and they are all in different encodings. Maybe it is possible to pipe the content of the fetch:binary to a system command for guessing the encoding, and use this to read in the csv? Best regards, Kristian Kankainen

3 3

Re: [basex-talk] automatic encoding
by Philippe Pons 11 May '21

11 May '21

Dear Christian, Thank you very much for this solution! It works perfectly. I use it on fairly short texts, so it is very fast. However, I have a subsidiary question: I sometimes have dates (for example 1600) or groups of words. In these two cases only, the automatic encoding does not work? Do you have any tips? Thank you so much. Philippe Le 06/05/2021 à 16:37, Christian Grün a écrit : > Hi Philippe, > > here’s one way to solve your challenge: > > let $csv := csv:doc('exemple.csv') > let $doc := doc('exemple1.xml') > return $doc update { > for $text in .//text() > return replace node $text with ( > for $token in analyze-string($text, '\p{L}+')/* > let $record := $csv//record[entry[1] = $token] > return if ($record) then ( > element { data($record/entry[2]) } { data($token) } > ) else ( > text { $token } > ) > ) > } > > In the update block, all text nodes are processed and replaced by a > sequence of new element and text nodes. The text is tokenized with > fn:analyze-string, such that the non-letter characters will not get > lost. If a CSV entry exists for a single token, it is replaced by a > new element that contains the token. Otherwise, the original token > will be adopted. > > If large texts need to be processed, it may be recommendable to > organize all CSV entries in a map: > > let $words := map:merge( > csv:doc('exemple.csv')/csv/record > ! map:entry(data(entry[1]), data(entry[2])) > ) > return doc('exemple1.xml') update { > .//text() ! (replace node . with ( > for $token in analyze-string(., '\p{L}+')/* > let $name := $words($token) > return if ($name) then ( > element { $name } { data($token) } > ) else ( > text { $token } > ) > )) > } > > Alternative solutions are appreciated. > > Hope this helps, salutations, > Christian > > > > On Thu, May 6, 2021 at 3:40 PM Philippe Pons > <philippe.pons(a)college-de-france.fr> wrote: >> Hi, >> >> This may be more of an xquery question than a BaseX question, but I'm trying to ask it here anyway. I can of course remove it if necessary. >> >> However, I have an XML document with a sequence of paragraphs and a CSV spreadsheet (examples 1 and 2 below). The csv gives a list of terms in the first column (present in the XML), and the name of the index in which we want to integrate this term in the second. >> >> I use BaseX and XQuery a lot, and I'm looking for a way to automatically encode the terms in the TEI file without success so far. >> >> I've tried to rely on the ft:mark function to achieve this, but it doesn't completely fit my need. >> >> Maybe you could give me some advice to get the result I'm looking for (example 3)? >> >> With kind regards, >> Philippe >> >> Exemple 1, XML : >> >> <div> >> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc quis nisl ultrices, viverra erat id, volutpat magna. Mauris cursus tellus non nisi commodo faucibus. Pellentesque condimentum feugiat sem quis interdum. Vestibulum tempus lectus a augue viverra molestie. Ut condimentum vehicula nisi, vel tincidunt mauris accumsan malesuada. Aliquam quis facilisis justo. Proin convallis eget enim vel eleifend. Nullam faucibus ultricies diam, iaculis feugiat odio condimentum non. Cras vitae dignissim lectus, in pellentesque est. Vivamus pharetra semper magna, sed sodales dui porttitor in. Pellentesque eget sodales quam, et dignissim velit. Aliquam vulputate pulvinar cursus. Phasellus commodo nibh a diam imperdiet cursus. Maecenas dui orci, aliquet quis porttitor non, auctor fringilla leo.</p> >> <p>Vestibulum venenatis velit in imperdiet iaculis. Vivamus consectetur mollis augue, ac efficitur nisl ultrices eu. Cras commodo eleifend mi a luctus. Vivamus sed odio maximus, laoreet mauris quis, tincidunt lorem. Duis non elementum tortor. Nam hendrerit dolor ac interdum condimentum. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Aenean vitae massa commodo, condimentum quam in, malesuada massa. Maecenas finibus convallis erat at aliquet. Aliquam erat volutpat. Praesent ligula nisi, tempus id arcu id, scelerisque condimentum dolor. Sed quis tincidunt sem. Nulla ac ex >> hendrerit, ullamcorper leo et, sodales nulla.</p> >> </div> >> >> Exemple 2, CSV : >> >> Vestibulum, placeName >> Cras, persName >> condimentum, persName >> >> >> Exemple 3, Results : >> >> <div> >> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc quis nisl ultrices, viverra erat id, volutpat magna. Mauris cursus tellus non nisi commodo faucibus. Pellentesque <persName>condimentum</persName> feugiat sem quis interdum. <placeName>Vestibulum</placeName> tempus lectus a augue viverra molestie. Ut <persName>condimentum</persName> vehicula nisi, vel tincidunt mauris accumsan malesuada. Aliquam quis facilisis justo. Proin convallis eget enim vel eleifend. Nullam faucibus ultricies diam, iaculis feugiat odio <persName>condimentum</persName> non. <persName>Cras</persName> vitae dignissim lectus, in pellentesque est. Vivamus pharetra semper magna, sed sodales dui porttitor in. Pellentesque eget sodales quam, et dignissim velit. Aliquam vulputate pulvinar cursus. Phasellus commodo nibh a diam imperdiet cursus. Maecenas dui orci, aliquet quis porttitor non, auctor fringilla leo.</p> >> <p><placeName>Vestibulum</placeName> venenatis velit in imperdiet iaculis. Vivamus consectetur mollis augue, ac efficitur nisl ultrices eu. <persName>Cras</persName> commodo eleifend mi a luctus. Vivamus sed odio maximus, laoreet mauris quis, tincidunt lorem. Duis non elementum tortor. Nam hendrerit dolor ac interdum <persName>condimentum</persName>. <placeName>Vestibulum</placeName> ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Aenean vitae massa commodo, <persName>condimentum</persName> quam in, malesuada massa. Maecenas finibus convallis erat at aliquet. Aliquam erat volutpat. Praesent ligula nisi, tempus id arcu id, scelerisque <persName>condimentum</persName> dolor. Sed quis tincidunt sem. Nulla ac ex hendrerit, ullamcorper leo et, sodales nulla.</p> >> </div>

2 1

Getting profile information in server responses.
by Reece Dunn 10 May '21

10 May '21

Hi all, In BaseX, is there a way to get the profile timings (compile, run, print, etc.) in the response of a HTTP request via the Server-Timing header ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing)? Kind regards, Reece

2 2

automatic encoding
by Philippe Pons 06 May '21

06 May '21

Hi, This may be more of an xquery question than a BaseX question, but I'm trying to ask it here anyway. I can of course remove it if necessary. However, I have an XML document with a sequence of paragraphs and a CSV spreadsheet (examples 1 and 2 below). The csv gives a list of terms in the first column (present in the XML), and the name of the index in which we want to integrate this term in the second. I use BaseX and XQuery a lot, and I'm looking for a way to automatically encode the terms in the TEI file without success so far. I've tried to rely on the ft:mark function to achieve this, but it doesn't completely fit my need. Maybe you could give me some advice to get the result I'm looking for (example 3)? With kind regards, Philippe Exemple 1, XML : <div> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc quis nisl ultrices, viverra erat id, volutpat magna. Mauris cursus tellus non nisi commodo faucibus. Pellentesque condimentum feugiat sem quis interdum. Vestibulum tempus lectus a augue viverra molestie. Ut condimentum vehicula nisi, vel tincidunt mauris accumsan malesuada. Aliquam quis facilisis justo. Proin convallis eget enim vel eleifend. Nullam faucibus ultricies diam, iaculis feugiat odio condimentum non. Cras vitae dignissim lectus, in pellentesque est. Vivamus pharetra semper magna, sed sodales dui porttitor in. Pellentesque eget sodales quam, et dignissim velit. Aliquam vulputate pulvinar cursus. Phasellus commodo nibh a diam imperdiet cursus. Maecenas dui orci, aliquet quis porttitor non, auctor fringilla leo.</p> <p>Vestibulum venenatis velit in imperdiet iaculis. Vivamus consectetur mollis augue, acefficitur nisl ultrices eu. Cras commodo eleifend mi a luctus. Vivamus sed odio maximus, laoreet mauris quis, tincidunt lorem. Duis non elementum tortor. Nam hendrerit dolor ac interdum condimentum. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Aenean vitae massa commodo, condimentum quam in, malesuada massa. Maecenas finibus convallis erat at aliquet. Aliquam erat volutpat. Praesent ligula nisi, tempus id arcu id, scelerisque condimentum dolor. Sed quis tincidunt sem. Nulla ac ex hendrerit, ullamcorper leo et, sodales nulla.</p> </div> Exemple 2, CSV : Vestibulum, placeName Cras, persName condimentum, persName Exemple 3, Results : <div> <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc quis nisl ultrices, viverra erat id, volutpat magna. Mauris cursus tellus non nisi commodo faucibus. Pellentesque <persName>condimentum</persName>feugiat sem quis interdum. <placeName>Vestibulum</placeName>tempus lectus a augue viverra molestie. Ut <persName>condimentum</persName>vehicula nisi, vel tincidunt mauris accumsan malesuada. Aliquam quis facilisis justo. Proin convallis eget enim vel eleifend. Nullam faucibus ultricies diam, iaculis feugiat odio <persName>condimentum</persName>non. <persName>Cras</persName>vitae dignissim lectus, in pellentesque est. Vivamus pharetra semper magna, sed sodales dui porttitor in. Pellentesque eget sodales quam, et dignissim velit. Aliquam vulputate pulvinar cursus. Phasellus commodo nibh a diam imperdiet cursus. Maecenas dui orci, aliquet quis porttitor non, auctor fringilla leo.</p> <p><placeName>Vestibulum</placeName>venenatis velit in imperdiet iaculis. Vivamus consectetur mollis augue, ac efficitur nisl ultrices eu.<persName>Cras</persName>commodo eleifend mi a luctus. Vivamus sed odio maximus, laoreet mauris quis, tincidunt lorem. Duis non elementum tortor. Nam hendrerit dolor ac interdum <persName>condimentum</persName>. <placeName>Vestibulum</placeName>ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Aenean vitae massa commodo, <persName>condimentum</persName>quam in, malesuada massa. Maecenas finibus convallis erat at aliquet. Aliquam erat volutpat. Praesent ligula nisi, tempus id arcu id, scelerisque <persName>condimentum</persName>dolor. Sed quis tincidunt sem. Nulla ac ex hendrerit, ullamcorper leo et, sodales nulla.</p> </div>

2 1

Clarification on registering a job service
by Zimmel, Daniel 06 May '21

06 May '21

Hi, can someone clarify on the following behaviour? The docs say: "Job services can be updated: If a new job is registered, and if there is already a job with the same id, the old entry will be replaced." https://docs.basex.org/wiki/Jobs_Module#Services So when I try to update an existing job service with the following query, I get an unexpected error "Job id already exists: myquery.": jobs:eval( xs:anyURI('myquery.xq'), (), map { 'id':'myquery', 'start':'19:00:00', 'interval':'P1D', 'service': true() } ) Am I reading the documentation wrong? Thanks Daniel

2 3

Re: [basex-talk] feature request: EXI support
by Christian Grün 06 May '21

06 May '21

Hi Mikhail (cc to the list), If your queries generate hundreds of Megabytes, you could check if the full result is actually required. If yes, you may need to check which step is responsible for letting you wait: Is it the pure transmission over the network, is it the evaluation time or is it the client that processes the query result? If it’s the data traffic, and if you use Jetty as BaseX web server, you could enable GZIP compression [1]. If you use another web server, or if your architecture is more complex, you could install a proxy in between that takes care of compressing the transferred data. Best, Christian [1] https://docs.basex.org/wiki/Options#GZIP On Wed, May 5, 2021 at 7:17 AM Mikhail Kuznetsov <mikekuznetsov(a)mail.ru> wrote: > > Hi, Christian. > > I am trying to build a central server for processing multiple QXuery tasks. In some cases the output will be used by a BI system for visualization. For example: > > the BaseX server will accumulate a lot of "raw" xml files collected from multiple entities, say their tax returns > this is great that I can have the flexibility to write new queries and adapt existing ones without modifying the underlying data (which would be the case if I had chosen an SQL database for storage) > a Power BI server gets connected to BaseX through RESTful API and retrieves two kinds of data: individual by entity and benchmark based on the average. > > Some queries may result in hundreds of Mb of xml data. My BaseX server is hosted in AWS, while the Power BI server is located internally. The communication takes too much time, which is a show stopper for me. I am looking for ways to optimize the data transfer. The BaseX server itself is perfect, many thanks for the great job. > > Regards, > Mikhail > > > Среда, 5 мая 2021, 2:45 +03:00 от Christian Grün <christian.gruen(a)gmail.com>: > > Hi Mikhail, > > > Hi there. I have recently been looking at ways to improve the efficiency of my XQuery server and have come across Efficient XML Interchange (EXI) Format. Is there a way that BaseX could support encoding/ decoding for this format? > > The internal storage of XML documents in BaseX differs quite a lot > from the original and verbose textual representation (for which EXI > provides improvements). > > What kind of performance penalties did you encounter so far? > > Best, > Christian > > > On Tue, May 4, 2021 at 10:34 PM Mikhail Kuznetsov <mikekuznetsov(a)mail.ru> wrote: > > > > > > Thanks. > > Mikhail > > > > -- > Mikhail Kuznetsov >

2 2

feature request: EXI support
by Mikhail Kuznetsov 04 May '21

04 May '21

Hi there. I have recently been looking at ways to improve the efficiency of my XQuery server and have come across Efficient XML Interchange (EXI) Format. Is there a way that BaseX could support encoding/ decoding for this format? Thanks. Mikhail

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk