BaseX-Talk June 2023

basex-talk@mailman.uni-konstanz.de

10 participants
18 discussions

BaseX 9.6: The Summer Edition
by Christian Grün 28 Nov '24

28 Nov '24

Dear all, We provide you with a new and fresh version of BaseX, our open source XML framework, database system and XQuery 3.1 processor: https://basex.org/ Apart from our main focus (query rewritings and optimizations), we have added the following enhancements: XQUERY: MODULES, FEATURES - Archive Module, archive:write: stream large archives to file - SQL Module: support for more SQL types - Full-Text Module, ft:thesaurus: perform Thesaurus queries - Fulltext, fuzzy search: specify Levenshtein limit - UNROLLLIMIT option: control limit for unrolling loops XQUERY: JAVA BINDINGS - Java objects of unknown type are wrapped into function items - results of constructor calls are returned as function items - the standard package "java.lang." has become optional - array arguments can be specified with the middle dot notation - conversion can be controlled with the WRAPJAVA option - better support for XQuery arrays and maps WEB APPLICATIONS - RESTXQ: Server-Timing HTTP headers are attached to the response For a more comprehensive list of added and updated features, look into our documentation (docs.basex.org) and check out the GitHub issues (github.com/BaseXdb/basex/issues). Have fun, Your BaseX Team

11 43

recursively used variables
by Rob Stapper 12 Aug '24

12 Aug '24

Hi, The code[1] below and send as attachment generates a error message: “Static variable depends on itself: $Q{http://www.w3.org/2005/xquery-local-functions}test”. I use these variables to refer to my private functions in my modules so I can easyly refer to them in a inheritance situation. It’s not a big problem for me but I was wondering if the error-triggering is justified or that it should work. [1]=========================================== declare variable $local:test := local:test#1 ; declare %private function local:test( $i) { if ( $i > 0) then $local:test( $i - 1) } ; $local:test( 10) =========================================== Kind regards, Rob Stapper Sent from Mail for Windows 10 -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

4 7

xslt:transform-report result
by Andy Bunce 11 Mar '24

11 Mar '24

Hi, Using BaseX 9.7.1 and saxon9he-9.9.1.jar The documentation suggests the ?result from xslt:transform-report should be *a* document-node where possible [1] This seems not quite to be the case when there are processing instructions or comments at the top level. In these cases a sequence of document-nodes is returned. /Andy [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report let $xslt:=<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="3.0"> <xsl:mode on-no-match="shallow-copy"/> </xsl:stylesheet> let $xml:=document{ <?something type="compact"?>, <foo/>} return xslt:transform-report($xml,$xslt) Returns map { "messages": (), "result": (<?something type="compact"?>, <foo/>) }

3 7

PHP Client Session Object: infinite loop in read method when bpos is null
by Tamara Marnell 13 Jul '23

13 Jul '23

Hello everyone, Once in a while, my server running BaseX will suddenly experience a big spike in write operations, the hard disk space will fill up to 100%, and the whole server will lock up. I can't say what initiates this, because the logs and metrics disprove every theory I can think of. (There's nothing unusual in the access logs or BaseX data logs, memory usage stays consistently at 45% and CPU usage at 5%, no user attempted anything like a huge bulk upload, etc.) However, one thing that always happens during these incidents is that the Apache error log blows up to a crazy size like 1 GB, filled with the message: PHP Notice: Trying to access array offset on value of type null in /[path]/client/session.php on line 164 This line is in the read method of the Session class from GitHub, and it indicates that the byte position passed to the buffer property is null. My error log probably fills up because read() is called by readString() in a while loop that never terminates. There might be extraordinary circumstances in which during construction, socket_connect returns "true" but the response is not valid or expected. -Tamara -- Tamara Marnell Program Manager, Systems Orbis Cascade Alliance (orbiscascade.org <https://www.orbiscascade.org/>) Pronouns: she/her/hers

2 5

Whitespaces not preserved in TEI documents.
by Montserrat Matías 06 Jul '23

06 Jul '23

Hello. I have a bug with BaseX version 10 and higher related to handling whitespaces in documents with mixed content elements. I'm working with narrative documents (TEI). Following the instructions in the documentation about this type of documents (containing mixed elements) I have disabled the indentation and the STRIPWS option. When I execute queries with connection to the database, the result is as explained in the documentation (blanks are preserved). However, when I use the BaseX libraries to execute queries in a local context (without a connection to the database) the spaces are sometimes removed. Same error if I run the query in the Database Administration interface. Query: db:option('stripws') Result: false Query: db:option('serializer') Result: map{"omit-xml-declaration":"yes","binary":"yes","method":"basex","use-character-maps":"","tabulator":"no","allow-duplicate-names":"no","media-type":"","doctype-public":"","escape-uri-attributes":"no","standalone":"omit","csv":map{"lax":true(),"backslashes":false(),"separator":"comma","allow":"","header":false(),"quotes":true(),"format":"direct"},"indents":2,"json-node-output-method":"xml","json":map{"escape":true(),"strings":false(),"lax":false(),"indent":(),"format":"direct","merge":false()},"doctype-system":"","item-separator":(),"indent":"no","suppress-indentation":"","byte-order-mark":"no","include-content-type":"yes","encoding":"UTF-8","newline":"\n","normalization-form":"none","html-version":"","version":"","limit":-1,"undeclare-prefixes":"no","cdata-section-elements":"","parameter-document":""} Query: let $doc := <doc><entry><sense id='1'><def>Gente grosera o vulgar.</def><cit><quote><seg>La</seg> <oRef>chusma</oRef>.</quote></cit></sense></entry></doc> return $doc/entry//sense[@id = '1'] Result: <sense id="1"><def>Gente grosera o vulgar.</def><cit><quote><seg>La</seg><oRef>chusma</oRef>.</quote></cit></sense> The whitespace contained between the </seg> and <oRef> tags has been stripped. In version 9.3.5 this problem did not occur. Can you please have a look at this? In a TEI document whitespaces are important. Thanx in advance, Montse. ___________________________________________ NOTA LEGAL El contenido de este mensaje de correo electrónico, incluidos los ficheros adjuntos, es confidencial y está protegido por el artículo 18.3 de la Constitución española, que garantiza el secreto de las comunicaciones. Si usted recibe este mensaje por error, le rogamos que se ponga en contacto con el remitente para informarle de este hecho y no difunda su contenido ni haga copias. La Real Academia Española informa de que los datos que en esta comunicación figuran, así como los que mantiene de usted y/o de su empresa, son tratados con la finalidad de mantener el contacto, así como de realizar las gestiones que en esta aparecen, y son utilizados de forma autorizada por las partes y sin cederse a terceros ajenos. Puede ejercer sus derechos a través de proteccion_de_datos(a)rae.es<mailto:proteccion_de_datos@rae.es>. Puede obtener más información sobre protección de datos en nuestra página web<https://www.rae.es/aviso-legal> o contactando directamente con nosotros (Reglamento UE 2016/679).

3 7

Performance of db:get-value() for large maps
by Eliot Kimber 03 Jul '23

03 Jul '23

I’m working on constructing DITA key spaces for our content. My current implementation builds an XQuery map that contains the key space data as well as the XML data from which the key space was constructed, which can be quite a bit (1-2 megabytes of XML all told—key space construction effectively requires constructing a single “resolved” DITA map from a tree of map documents, which I’m then storing in my key space map). A given key space might have 100,000 individual name-to-element mappings in addition to the raw DITA content. I tried saving these maps into a database using db:put-value(), which worked. But, when I went to fetch them back out, it took 30 seconds to fetch one of these maps. My expectation was that retrieving a map would be quite fast. I’m sure that it was silly to try to store all the raw markup in these maps, and I will change that aspect of my implementation to store the resolved maps separately and then use node IDs in the key space map. Worst case I can do what I have been doing with BaseX 9 and convert the XQuery map to an XML representation and use that for retrieval, which of course will work fine, at the cost of more code and a few more milliseconds of ingestion processing. Before I spend any more time trying to optimize the storage and retrieval of these maps, I wanted to ask a few questions: 1. Is my expectation that map retrieval should generally be very fast correct, modulo not having gobs of raw XML in them? 2. Is there a more efficient way to query over maps that are stored in a database? My initial attempt was to simply pull the entire map from the database and then operate on it as I would any other map, i.e.: $keySpace as map(*) := getKeySpace($rootMap) (: Pulls the map from the database :) $keydefs as element()* := $keySpace?keyscopes?($scopeKey)?keydefs?($keyName) 3. Is there some other maps-in-a-database optimization technique I’m overlooking? Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 2

Calling a Java function with varargs parameter from XQuery
by Lizzi, Vincent 03 Jul '23

03 Jul '23

Hello BaseX people, The Java language bindings in BaseX look like a very good way to use Java libraries from XQuery. I'm currently trying to use Lingua (https://github.com/pemistahl/lingua) with BaseX, and mostly have it working, but I've not been able to figure out how to call a Java function that takes a vararg (variable arguments) parameter. Here is a bit of XQuery code that shows the problem. I'm using BaseX version 10.6 and the jar for Lingua version 1.2.2 has been added to the classpath. This query should return a LanguageDetectorBuilder object. declare namespace Builder = "java:com.github.pemistahl.lingua.api.LanguageDetectorBuilder"; declare namespace Detector = "java:com.github.pemistahl.lingua.api.LanguageDetector"; declare namespace Language = "java:com.github.pemistahl.lingua.api.Language"; let $builder := Builder:fromLanguages( [ Language:ENGLISH(), Language:DUTCH(), Language:GERMAN(), Language:SPANISH(), Language:FRENCH() ] ) return $builder The above code produces this error message in the BaseX GUI: [XPTY0004] com.github.pemistahl.lingua.api.LanguageDetectorBuilder:fromLanguages(com.github.pemistahl.lingua.api.Language[]) expected, (CArray) found. If I try making the parameter a sequence instead of an array, replacing () parenthesis for the [] brackets, then this is the error message: [XPTY0004] com.github.pemistahl.lingua.api.LanguageDetectorBuilder:fromLanguages(com.github.pemistahl.lingua.api.Language[]) expected, (List) found. I've read through the Java Bindings page in the wiki and tried to rewrite the parameter in different ways, but so far have only gotten different error messages. The signature for the fromLanguages function can be seen at https://github.com/pemistahl/lingua/blob/main/src/main/kotlin/com/github/pe… Is there a way to make this work? Thanks, Vincent ______________________________________________ Vincent M. Lizzi Head of Information Standards | Taylor & Francis Group 530 Walnut St., Suite 850, Philadelphia, PA 19106 E-Mail: vincent.lizzi(a)taylorandfrancis.com<mailto:vincent.lizzi@taylorandfrancis.com> Web: www.tandfonline.com<http://www.tandfonline.com> Taylor & Francis is a trading name of Informa UK Limited, registered in England under no. 1072954 "Everything should be made as simple as possible, but not simpler." Information Classification: General

2 4

Prolog and XQuery
by Ben Engbers 29 Jun '23

29 Jun '23

Hi, My ultimate goal is to investigate the advantages of using Prolog and XQuery together when querying XML files. In doing so, I want to take advantage of BaseX's XQuery engine. Since there is no Prolog client, I initially started writing a Prolog client. But in SWI-prolog, I did not manage to connect using the webconnection tools. So I first started to write a C++ client and then start writing a bridge in Prolog that uses that client. The starting point for the C++ client was the RBaseX client I published earlier. I converted all functionalities from that client to C++. A preliminary version of the C++ client can be found at https://github.com/BenEngbers/BasexCpp. As soon as I manage to create a real shared object using CMake I will publish a final version. Regarding the server protocol for BaseX, however, I still have a question. According to that protocol, the ADD command accepts 4 arguments ({09 {name} {path} {input} ). However, neither in R nor in C++ have I succeeded in using the {name} argument. My question is whether this is a bug in the protocol? I don't know if there is a need for a C++ client that implements the full server protocol but in any case I enjoyed working on this project. Have fun, Ben Engbers

2 1

Setting up redirects and https
by Eliot Kimber 28 Jun '23

28 Jun '23

I have a “production” BaseX server for our Mirabel system that is currently served on the old 8984 port. As it’s a dedicated server, I would like to serve it on port 80. But I also have users who use the current port, and I don’t want to disrupt them more than necessary when I make this change. I’d also like to start serving using HTTPS. Is there an easy way to set up a redirect from the old port to the new port? What is involved in serving BaseX over HTTPS? I’m using the built-in Jetty server and I suspect the answer is in the Jetty docs but I thought I’d ask here first before seeing of ChatGPT can get me close to a solution. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

1 0

Dynamic path segment for websockets
by Marco Lettere 28 Jun '23

28 Jun '23

Dear all, so I just found out that it is not possible to create websockets bound to dynamic paths. If I try a websocket connection to declare %ws:connect('/notification/subject/{$subject}') function log:subject() {}; I get an error (without significant logs). Whereas declare %ws:connect('/notification/subject/one') function log:subject() {}; works well. Does this have a motivation? Is there another way of meeting the requirement of having dynamic websocket channels? Thanks in advance, Marco.

2 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk June 2023