BaseX-Talk July 2022

basex-talk@mailman.uni-konstanz.de

20 participants
22 discussions

Adding Attributes After the Fact: How To?
by Eliot Kimber 16 Jul '22

16 Jul '22

In my Mirabel project I’m loading large quantities of DITA content (40K topics per content database). I’m turning off DTD parsing because otherwise it takes hours to load the content (I haven’t had bandwidth to figure out how to use Xerces grammar caching with BaseX, which would solve the problem). DITA uses an attribute-based mechanism (@class) to associate elements with their base types. For example, a <concept> element is a kind of <topic>, as indicated by the @class value “- topic/topic concept/concept “. To find DITA elements you always search on @class values rather than tag names. In XSLT 3 this is best expressed as match=”*[contains-token(@class, ‘topic/topic’)]” to handle any topic element. These attributes are normally defaulted in grammars so they are not explicit in the source as imported and thus do not exist in my no-DTD content databases. To work around this I’ve implemented a function that hard codes the element-to-class-value mapping to enable selecting elements by @class value. (For any set of content you can know what the @class value is for any element type as its statically declared in the governing grammar, so it is possible to have this kind of hard-coded element-to-class mapping.) However, I now realize that if the attributes are in the data as stored in BaseX that lookup will be much faster using an attribute index. That is, rather than using a predicate like [dutils:class(., ‘topic/topic’)] I could do db:token($database, ‘topic/topic’, ‘class’). Because I have the element-to-class mapping I can add the attributes to the content after the initial database load. My question: How best to do that? I can have an updating function that simply adds the @class attribute to all the elements but I suspect there are some performance implications for doing that kind of bulk update to millions of nodes. So any guidance on this kind of bulk update would be helpful. Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

1 0

RESTXQ: How to Reflect Progress of Long-Running Querries
by Eliot Kimber 16 Jul '22

16 Jul '22

In my Mirabel system, I have multiple BaseX instances to handle long-running processes while a primary instance serves the web pages. For these background processes I’d like to be able to show progress in the main web app. My question: what are the options for doing this? I can imagine having a database to write progress data to and then read from that in my RESTXQ page (maybe using a simple JavaScript to poll the app). Are there better techniques? I’m trying to limit the amount of JavaScript I have to write but I can do it if I have to. Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

Possible to Speed Up This Lookup?
by Eliot Kimber 16 Jul '22

16 Jul '22

I generate a “where used” index as an XML doc in a separate database. I’m then doing lookups against this index like this where $node is the element I want to find the index entry for: collection($rkDatabase)/doc-where-used-index/where-used-entry[string(@key) eq string(db:node-id($node))] Where the markup is shown in [1] below. Using prof:track() I’m measuring a consistent 0.036 seconds per lookup, which seems very slow. Db:node-id() is instantaneous so the cost must be in the XPath itself. I’ve tried different variations on the predicate but nothing makes a measurable difference. There must be a more efficient way to do this kind of lookup or a better way to structure my index XML so that the lookup is optimized. Or is this something that could be affected by my index settings at the database level? Thanks, Eliot [1] Where-used index markup: <doc-where-used-index> <where-used-entry key="9151416" tagname="map" class="map/map" id=""> <title> <ph keyref="workplace-safety-management"/> </title> <conrefs/> <topicrefs> <noderef node-id="672471" database="tokyo" tagname="mapref" baseuri="/tokyo/qa-sushma-sundareswaran.ditamap" href="workplace-service-delivery-core.ditamap"/> <noderef node-id="898684" database="tokyo" tagname="mapref" baseuri="/tokyo/qa-ankit-jaiswal.ditamap" href="workplace-service-delivery-core.ditamap"/> <noderef node-id="9524996" database="tokyo" tagname="mapref" baseuri="/tokyo/workplace-service-delivery-suite.ditamap" href="workplace-service-delivery-core.ditamap"/> </topicrefs> <doc> <noderef node-id="9151416" database="tokyo" tagname="map" baseuri="/tokyo/workplace-service-delivery-core.ditamap"/> </doc> <xrefs/> <referencing-maps> <noderef node-id="672452" database="tokyo" tagname="map" baseuri="/tokyo/qa-sushma-sundareswaran.ditamap"/> <noderef node-id="898667" database="tokyo" tagname="map" baseuri="/tokyo/qa-ankit-jaiswal.ditamap"/> <noderef node-id="9524980" database="tokyo" tagname="map" baseuri="/tokyo/workplace-service-delivery-suite.ditamap"/> </referencing-maps> </where-used-entry> … </doc-where-used-index> _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

5 11

file:resolve-path, ".." steps
by Hans-Juergen Rennau 11 Jul '22

11 Jul '22

Dear BaseX people, extension function file:resolve-path() shows a behaviour which appears to me inconsistent and probably not intended: - if called with a single parameter, a path containing ".." is resolved as expected- if called with two parameters, the ".." steps are not resolved. I would expect the resolution of ".." steps to be always a part of path resolution. Example: cd C:\a\b\c file:resolve-path('../x') =>C:\a\b\x file:resolve-path('../x', 'file:///C:/a/b/c/') =>C:\a\b\c\..\x file:resolve-path('../x ', 'file:///C:/a/b/c/' ! file:resolve-path(.) =>C:\a\b\x Greetings, Hans-Jürgen

2 2

Explicit cacheing with store: vs. implicit cacheing in RESTXQ ?
by Majewski, Steven Dennis (sdm7g) 11 Jul '22

11 Jul '22

Thinking about use cases for the new store: cacheing module, and wondering if I understand the cacheing already provided in RESTXQ. https://docs.basex.org/wiki/RESTXQ#Preliminaries <https://docs.basex.org/wiki/RESTXQ#Preliminaries> So in a production environment where I have disabled monitoring for file changes by setting PARSERESTXQ to -1, variables declared in prolog of library modules have a long lifetime and are only recalculated when the modules are reloaded and parsed — which would, I assume, imply that if modifications are made to the database, I should call rest:init() to force a reload if I have declared values that are calculated from a collection that may have been changed. And that would also imply that I could, in many cases, use that implicit caching in place of explicit store: functions. Is that correct ? — Steve M.

2 1

Email integration
by Ben Pracht 09 Jul '22

09 Jul '22

Hi, This may seem a bit unusual, but I like the parsing capabilities of XQuery and I'd like to use it for my email. I'd like to have BaseX log into my IMAP server, fetch emails and put each email as a separate XML document. I'm willing to have it ignore attachments if necessary. Then, I'd like to query my email using normal BaseX XQuery. This isn't for normal emails, but I wanted to be able to query it for automated emails and for one-off type queries. Oh, to make things more difficult, I'm using GMail, so the login just got even more difficult. Any suggestions? Thanks, Ben Best Regards, Ben Pracht 919.809.2439 ben.pracht(a)gmail.com

3 3

Same prefix, same functions, different namespace?
by Graydon Saunders 07 Jul '22

07 Jul '22

Hello -- As part of a long and involved conversion process, the content starts in a lax version of the target vocabulary and proceeds to a strict version of the target vocabulary. By local convention, the two vocabularies use the same namespace prefix but are not the same namespace and use different namespace URIs. Ideally, I would be using the same test functions in a lot of cases, only changing the association for the prefix depending on the data. So far as I know, there isn't any way to declare a namespace dynamically; I can't create a "test this content" module and tell it which namespace to use in some parameterized way, so I'm going to have to create two versions of the module despite the only difference being a namespace declaration. Is that right, or is there a way to do this? Thanks! Graydon

3 2

Java 8 and XML Catalog
by Lizzi, Vincent 07 Jul '22

07 Jul '22

Hi Christian, I'm having trouble getting an XML Catalog to work in BaseX 9.7.2 running on Java 8. The XML Catalog is working properly in BaseX and Saxon using Java 17. However, if I keep everything the same and switch to Java 8 the XML Catalog doesn't work. The error messages when running on Java 8 indicate that the XML Catalog is not being used at all. The obvious solution of updating the environment to a more recent version of Java is not so easy in this case. I have the XML Catalog location configured as an absolute file system URI in the system properties org.basex.catfile, xml.catalog.files, and javax.xml.catalog.files. Option db:intparse is set to false and option db:dtd is set to true. I have set the class path to include both the Apache XML Resolver (xml-resolver:xml-resolver:1.2) and the xmlresolver.org XML Resolver (org.xmlresolver:xmlresolver:4.4.0). It is my understanding that BaseX will use Norm Tovey-Walsh's XML Resolver from xmlresolver.org if it is found on the class path, or fall back to Java's built in XML resolver. I see the logic for this check on which catalog resolver to use in org.basex.util.Resolver. Do you have any suggestions on what to try, or is this pointing to a possible bug somewhere? Thank you, Vincent ______________________________________________ Vincent M. Lizzi Head of Information Standards | Taylor & Francis Group 530 Walnut St., Suite 850, Philadelphia, PA 19106 E-Mail: vincent.lizzi(a)taylorandfrancis.com Web: www.tandfonline.com Taylor & Francis is a trading name of Informa UK Limited, registered in England under no. 1072954 "Everything should be made as simple as possible, but not simpler." Information Classification: General

4 5

RESTXQ: Serve images that are not within the web app dir
by Eliot Kimber 07 Jul '22

07 Jul '22

In my RESTXQ application, I have the XML content in a database but the images referenced from that content is only in a local clone of the git repository from which the XML content is loaded into BaseX. I would like to serve the images from RESTXQ-generated HTML Pages but I’m not sure how to do that other than to use URLs that include the base64 image data directly. What I think I need to do is implement a RESTXQ end point that handles image URLs that I know how to convert into image files on the file system but I don’t see how to then return the raw bytes as the return result from the RESTXQ handler function. I’m trying to avoid copying the images because there’s no other reason to have them in a database or otherwise copy them somewhere else on the file system. I suppose I could symlink the git directories under the webapp directory but that seems kind of icky too. Thanks, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

4 5

Re: [basex-talk] Sym links in BaseX 9.7.2
by Maud Ingarao (IHRIM - ENS de Lyon) 07 Jul '22

07 Jul '22

Hi Christian :-) The prior version we were using was 9.3.2 and everything worked fine. Now with 9.7.2 I get this error : Stopped at /path/to/basex/webapp/synopsx/workspace/bvm/models/mixed.xqm, 38/55: [XQST0059] Could not retrieve module: /path/to/basex/webapp/synopsx/models/synopsx.xqm. although nothing changed in our webapp. Thanks a lot, and all the best :-) Maud

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk July 2022