- BaseX-Talk - mailman.uni-konstanz.de

How to store text files in a data base
by Hans-Juergen Rennau 13 Nov '22

13 Nov '22

Hello, I wonder what is the recommended way how to store text files in a database. What occurs to me is the use of db:put-value / db:get-value: Storage: db:put-value($db, $filePath ! file:read-text(.), $dbPath) Retrieval: db:get-value($db, $dbPath) Would you recommend a different approach, e.g. storing and retrieving as a binary? With kind regards,Hans-Jürgen

1 0

GUI doesn't quit on Mac
by Majewski, Steven Dennis (sdm7g) 10 Nov '22

10 Nov '22

I’ve been having a problem for a while where after attempting to quit the Mac GUI, it doesn’t actually exit and has to be force quit. I’m sorry I wasn’t able to track how this tracks with MacOS / JDK / BaseX updates and when it actually started happening. Wondering if anyone else has had this issue, and whether it’s Mac specific or JDK version specific. MacOS 12.6.1 but problem goes back at least to earlier 12.6 versions Openjdk from home-brew install — currently openjdk/19.0.1, but was happening on earlier versions. And the current and last several versions of BaseX — Steve M.

4 7

Reported memory usage in GUI with Store Module
by Tim Thompson 08 Nov '22

08 Nov '22

Hello, I've been experimenting with the new Store module, which seems very handy. I have a custom store that is about 2GB in size. In the GUI (BaseX 10.3 beta), when I try to load it into memory with store:read(), the reported memory usage appears to max out (with 32GB of available memory): Total Memory: 32 GB Reserved Memory: 32 GB Used Memory: 32 GB However, monitoring memory usage with htop shows a smaller increase in memory usage: from 5GB at baseline, it peaks around 15GB, then goes down to 9GB, all while the BaseX GUI appears to be frozen. After a couple of minutes, the store does load, but GUI performance seems to be degraded, and it continues to report max memory usage. Any thoughts? Tim -- Tim A. Thompson (he, him) Librarian for Applied Metadata Research Yale University Library www.linkedin.com/in/timathompson

2 1

basex client implementation
by Erik Peterson 06 Nov '22

06 Nov '22

The CLI basexclient uses the same client api as other clients found here yes? https://docs.basex.org/wiki/Clients from basexclient CLI >BINARY GET lib/example.jpg as expected, returns the jpg I PUT from there from my client app. However, the same command BINARY GET from my client app, using COMMAND {command} {result} {info} \00 does not return the jpg. The bytes going to the basex server, at least the command 'BINARY GET lib/example.jpg' part are identical in both cases.

3 4

Testing that a value is a UUID
by Graydon Saunders 05 Nov '22

05 Nov '22

Hello -- You can (I think) test if some attribute value is an RFC 4122 UUID by using a regular expression: let $regexp as xs:string := '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-5][0-9a-fA-F]{3}-[089abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$' for $uuid in //@GUID let $value as xs:string := $uuid/string() where not(matches($value,$regexp)) return $value Is there a better way to do this? thanks! Graydon

4 4

Client protocol updated
by Ben Engbers 04 Nov '22

04 Nov '22

Hi, Thanks to the mail from Erik Peterson on the basex client implementation, I learned that the client protocol has been updated and that 'replace/store' have been renamed to 'put/putBinary'. In my RBaseX-client, I used the 'retrieve' command to read binary data. I can't remember where I found that 'retrieve' was used for retrieving binary data. But while implementing 'put/putBinary' I noticed that 'retrieve' is no longer accepted as command. I have added a remark on this to the Clients page. Ben Engbers PS. RBaseX is loaded on average 10 times a day. I have never had any feedback on this package so I don't know in howfar it is really used. Any feedback is welcome!

1 0

is this a bug?
by Rob Stapper 02 Nov '22

02 Nov '22

Hi, I stumbled upon this behavior. Is this a bug? I’m still using Basex version 9.7. Code[1]: let $e := <E>abc</E> update { replace value of node . with ./data() || "def" } return ( $e, $e/data() ) Result: <E>abcdef</E> abc Expecting: <E>abcdef</E> abcdef [1] attachment: “replace-value-of-bug.xq” Pleased to hear from you, Rob Stapper Sent from Mail for Windows -- This email has been checked for viruses by Avast antivirus software. www.avast.com

2 1

Full-text bug (?) - all words / any word + stemming
by Hans-Juergen Rennau 28 Oct '22

28 Oct '22

Hello, I encountered a behaviour of full-text search which appears to me a bug: (1) OKbasex "'adverse' contains text 'adverse' using stemming" => true (2) NOTOKbasex "'adverse' contains text 'adverse' all words using stemming"=> false (3) NOTOKbasex "'adverse' contains text 'adverse' any word using stemming" => false (4) OK basex "'fox' contains text 'fox' any word using stemming" => true BaseX version is 10.3. With kind regards,Hans-Jürgen

2 1

Client auth debugging
by Erik Peterson 28 Oct '22

28 Oct '22

I'm implementing a client per basex api shown here: https://docs.basex.org/wiki/Server_Protocol#Authentication I'm working on digest auth and I can get back the realm and timestamp. I'm getting an access denied however when I send the username and token. My questions are: 1) What tips are there for debugging authentication implementation? I've set logs.debug to true in .basex and and tail them. I can see access denied but there's no info to help me debug. I'd like to at least see what the server is receiving. Any way to do that? BTW, my token creation is correct per the example in the docs. 2) It's taking a long time, 60s, for the test to run. Any way to speed that up?

3 2

xqDoc - potential updates
by Loren Cahlander 24 Oct '22

24 Oct '22

Hello folks, I am looking at maybe doing some updates to the xqDoc spec and the codebase. I have code that transforms the xqDoc resources with RestXQ into an OpenAPI 3 API documentation. The things that I want to figure out is: Figure out how to specify the schema of the parameters and the return Document the various HTTP return codes and the response content for each return code I would like to get the input from everyone using xqDoc Thank you, Loren

3 2

Lookup operator, conformance
by Hans-Jürgen Rennau 24 Oct '22

24 Oct '22

Hello, I noticed that the lookup operator (?) accepts as key expression also non-parenthesized expressions, e.g.: $map?$key It seems to me that the XQuery spec does not allow that - the key specifier must be an NCName, an integer, * or a parenthesized expression [1]: [126] KeySpecifier ::= NCName | IntegerLiteral | ParenthesizedExpr | "*" So only this should be allowed: $map?($key) But I am not quite sure if I do not overlook something. *My question*: is the acceptance of non-parenthesized key expressions conformant behaviour, or a (certainly intended) laxness? Kind regards, Hans-Jürgen [1] https://www.w3.org/TR/xquery-31/#id-lookup

2 2

How to check if a node is from a database
by Hans-Jürgen Rennau 08 Oct '22

08 Oct '22

Dear BaseX people, what would be an efficient way of checking if a given node is a database node? According to the documentation, function db:name() throws an error if the node is not a database node. However, that is not what I observe.(The same applies to db:node-id(), db:node-pre(), db:path()). Therefore I resorted to this way of checking: try {db:name($node) ! db:exists(.)} catch * {false()} That is clumsy and unreliable: if applied to a node in a file, db:name() returns the base name of the file, which might accidentally be a database name. What would you recommend instead? Kind regards, Hans-Jürgen

2 1

ft:mark and differing result sets on 10.2
by Chris Yocum 05 Oct '22

05 Oct '22

Hi, I had a chance to come back to basex after a while when I needed to do some querying on my data set. I updated to 10.2 and loaded the files. This was all fine. I then ran this query: for $x in db:get($db)/sample/entry return ft:mark($x[descendant::text() contains text {'fas'} using wildcards]) which ran in 2528.78ms and return 170 results. This seemed rather slow so I started to work on it. I also ran this: for $x in db:get($db)/sample/entry return $x[descendant::text() contains text {'fas'} using wildcards] which ran in 57.77ms and returned 35 results. I was very suprised by this as I had expected it to run in the same time and return the same result set. I, then, started look deeper and it seems like ft:mark does more than add an XML tag around the matched token? My question is this: does ft:mark do more than what I had expected it to do? Am I misunderstanding or miswriting this query? Thank you and all the best, Chris

3 3

Vulnerabilities in docker image
by Huib Verweij 05 Oct '22

05 Oct '22

Hi, we have built a docker image based on the 10.1 war (and 10.2 war). In Harbor, part of the OpenShift K8s platform that we use, the image is scanned for vulnerabilities. There are quite a few Criticals and Highs! Here’s a list of the most pressing CVEs: CVE package version Fixed in version ================================================================== CVE-2013-7285 com.thoughtworks.xstream:xstream 1.4.2 1.4.7 CVE-2021-21342 com.thoughtworks.xstream:xstream 1.4.2 1.4.16 and 5 more Criticals and a lot more Highs in XStream. There’s also a High in JDOM: CVE-2021-33813 org.jdom:jdom 1.1 The Aqua-Trivy plugin for the Docker Desktop-app reports the same CVEs. Especially the Criticals that are (presumably) easy to fix by upgrading the XStream package are in the way of deployment. Our image is based on tomcat:9-jre17-temurin, when I leave out the BaseX stuff it has just two Low CVEs. Btw, an up to date docker image on Docker Hub would be much appreciated. If it’s also available for the linux/arm64/v8 architecture that would be perfect! Kind regards, Huib.

2 1

Indent result for unit tests
by Tim Thompson 05 Oct '22

05 Oct '22

Hello, I've noticed that the "Indent results" button in the GUI does not seem to be working when running a test module from the GUI. I'm using BaseX 10.3 beta. All best, Tim -- Tim A. Thompson (he, him) Librarian for Applied Metadata Research Yale University Library www.linkedin.com/in/timathompson

2 1

Error: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms.
by ykhabins＠bellsouth.net 30 Sep '22

30 Sep '22

Hello, I am using BaseX v.10.2 More specifically launching basexserver.bat and using c# samples from the following link: https://github.com/BaseXdb/basex/tree/9/basex-api/src/main/c%23 Everything is work fine when a computer with the Windows 10 OS has the following security policy disabled: System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing https://docs.trendmicro.com/all/ent/sc/v3.0/en-US/cmcolh/t_fips.html Alas, I am working for the company where that security policy is enabled. And I am getting the following error: "This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms." This particular function is erroring out. private string MD5(string input) { MD5CryptoServiceProvider MD5 = new MD5CryptoServiceProvider(); ... } I did some research and collected the following useful links: https://stackoverflow.com/questions/14509354/this-implementation-is-not-part -of-the-windows-platform-fips-validated-cryptogra https://docs.trendmicro.com/all/ent/sc/v3.0/en-US/cmcolh/t_fips.html https://stackoverflow.com/questions/6652850/enforcefipspolicy-flag-in-web-co nfig-doesnt-seem-to-working-for-web-application https://techcommunity.microsoft.com/t5/microsoft-security-baselines/why-we-a mp-8217-re-not-recommending-amp-8220-fips-mode-amp-8221/ba-p/701037 I also tried to add the following to the .config file to no avail: <configuration> <runtime> <enforceFIPSPolicy enabled="0" /> </runtime> </configuration> Is it possible to modify c# samples to work on the computers where mentioned above security poilsy is enabled? Regards, Yitzhak Khabinsky

1 0

BaseX 10.1 bug in GET command?
by Gabriel Lorenzo 27 Sep '22

27 Sep '22

Hello, I think I've found a possible bug. When I use the GET command the document retrieved is not always the same I requested. It only happens with the GET command, as using db:get() from xquery always works as expected. I haven't observed it happening with BINARY GET. I've reproduced it with the simplest example: BaseX 10.1 [Client]Try 'help' to get more information. > create db sample Database 'sample' created in 14.0 ms. > open sample Database 'sample' was opened in 0.03 ms. > add to hello.xml <hello>world</hello> Resource(s) added in 2.25 ms. > add to goodbye.xml <goodbye>world</goodbye> Resource(s) added in 2.22 ms. > get hello.xml <goodbye>world</goodbye>Query executed in 0.18 ms. > get goodbye.xml <hello>world</hello>Query executed in 0.19 ms. > xquery db:get('sample','hello.xml') <hello>world</hello> Query "BaseX" executed in 0.64 ms. > xquery db:get('sample','goodbye.xml') <goodbye>world</goodbye> Query "BaseX" executed in 0.23 ms. > I'm using BaseX 10.1 on Windows 10 with OpenJDK Temurin-17.0.4.1+1 Regards, Gabriel Lorenzo

2 1

Full text search with non-Latin scripts
by Andreas Hartmann 26 Sep '22

26 Sep '22

Hi, I have problems with full text search in 10.1, when non-Latin script is involved. Expressions such as "βάναυσος" contains text "67644567" using fuzzy "67644567" contains text "βάναυσος" using fuzzy "humbug" contains text "βάναυσος" using fuzzy all yield "true". It's the same with Hebrew strings. It does not matter which Greek/Hebrew word is used, as long as the word-length does not differ too much from the compared string. Is this really the expected behavior or could it be a bug? Best greetings Andreas

2 1

file:descendants() returns relative paths
by Hans-Jürgen Rennau 26 Sep '22

26 Sep '22

Dear BaseX people, functions file:descendants and file:children return relative paths, if input parameter $dir is a relative folder path. I regard this as a bug - relative folder paths should be resolved against the current working directory, so that a relative path and the corresponding absolute path are equivalent parameters. The functions should always return absolute paths, as the documentation promises. After all, a key use case is to find files which to access, and the accessor functions (doc(), json:doc(), unparsed-text() etc.) do *not* resolve relative paths against the current working dir. Kind regards, Hans-Jürgen

4 9

Full Text bug (?) - window ... words
by Hans-Juergen Rennau 26 Sep '22

26 Sep '22

Dear BaseX people, it seems to me there is a bug concerning Full Text Search, using option "window": let $text1 := '1 The usability of a Web site is how well the site' let $text2 := '2 The usability of a Web site is how well the sitx'let $text3 := '3 The usability of a Web site is how well the site'return ( $text1[. contains text 'usability web site' all words window 5 words], $text2[. contains text 'usability web site' all words window 5 words], $text3[. contains text 'usability web site' all words window 10 words] ) This query should return all three, $text1, $text2 and $text3, but it only returns $text2 and $text3. So it seems to me that the implemented logic is: "all matches of the "all words" search must be within a 5-words window, but it should be "there is a match of the "all words" search which is within a 5-words window. More detailed argument in PS. Kind regards,Hans-Jürgen PS: Compare https://www.w3.org/TR/xpath-full-text-30/#ftwindow "A window selection examines the matches generated by the preceding portion of the FTSelection, and selects those for which the matched tokens and phrases (more precisely, the individual StringIncludes of that match) are all found within a window whose size is a specified number of FTUnits (words, sentences, or paragraphs); for each such window, the window selection then generates a match containing the merge of those StringIncludes, plus any StringExcludes that fall within the window." (Italic added by me) The detailed semantics are given in 4.2.4 FTWords [1]. Pseudo-function fts:applyFtWordsAllWord() constructs for each of the words an fts:allMatches element and then performs conjunction of these elements, based on recursive application of 4.3.6.2 FTAnd [2]. Each fts:allMatches element contains one fts:match element for each occurrence of the word in question. The "ANDing" of two fts:allMatches is described by pseudo-function fts:ApplyFTAnd(), which creates one match for each pair of matches found in the operands - in other words, all combinations of operand matches are considered. As an example consider: "foo bar" all words and assume "foo" occurs two times and "bar" occurs two times. The fts:allMatches element representing the result is the result of applying fts:ApplyFTAnd() to two fts:allMatches elements, one obtained for word "foo", one obtained for word "bar". Schematically: $fts:allMatches_foo: <fts:match>foo(1)</fts:match> <fts:match>foo(2)</fts:match> $fts:allMatches_bar: <fts:match>bar(1)</fts:match> <fts:match>bar(2)</fts:match> fts:applyFTAnd($fts:allMatches_foo, $fts:allMatches_bar) is a single fts:allMatches containing four matches, each one of which is a combination of matches found in the operands: <fts:allMatches_allwords> = <fts:match>foo(1), bar(1)</fts:match> <fts:match>foo(1), bar(2)</fts:match> <fts:match>foo(2), bar(1)</fts:match> <fts:match>foo(2), bar(2)</fts:match> Now extend the query to "foo bar" all words window 5 words The result is true, if there is at least one combination of occurrences of "foo" and "bar" found in a window of at most 5 for word "bar". Referring to the semantic intermediates: if <fts:allMatches_allwords> contains at least one <fts:match> element for which all contained matches satisfy the window condition. [1] https://www.w3.org/TR/xpath-full-text-30/#tq-ft-fs-FTWords[2] https://www.w3.org/TR/xpath-full-text-30/#tq-ft-fs-FTAnd

2 1

Java 11
by Giuseppe G. A. Celano 13 Sep '22

13 Sep '22

Hi All, On the BaseX website (https://basex.org/download/ <https://basex.org/download/>), it is specified that “Java 11 is required to install and run BaseX (10.1)” . Since it is not specified “Java 11 or higher”, I am wondering whether this requirement is actually meant to be strict. Thanks! Best, Giuseppe

2 1

current-dateTime() function precision
by ykhabins＠bellsouth.net 09 Sep '22

09 Sep '22

Hello, I am using BaseX v.10.1 It seems that the current-dateTime() function precision is off. I am validating an XML file against an XSD 1.1 file via Xerces 2.12.2 validator. And trying to measure a timing of it. Here is my code. xquery version "4.0"; let $xml := '\\...\AForm-XSD-20211013\PD220224062681.XML' let $xsd := '\\...\AForm-XSD-20211013\Miami-ws-AForm.xsd' let $before_datetime := current-dateTime() let $result := validate:xsd-report($xml, $xsd, map { 'http://apache.org/xml/features/validation/cta-full-xpath-checking': true() }) return <root><metadata> <result>{data($result/status)}</result> <errors>{count($result/message)}</errors> <startTime>{$before_datetime}</startTime> <endTime>{current-dateTime()}</endTime> <xmlFile>{$xml}</xmlFile> <xsdFile>{$xsd}</xsdFile> <xsdProcessor>BaseX {data(db:system()//version)}, EE-Java, {validate:xsd-processor()} 2.12.2</xsdProcessor> <xsdVersion>{validate:xsd-version()}</xsdVersion> </metadata> <messages>{$result/message}</messages> </root> It is emitting the following output: <root> <metadata> <result>invalid</result> <errors>3</errors> <startTime>2022-09-09T11:32:29.667-04:00</startTime> <endTime>2022-09-09T11:32:29.667-04:00</endTime> <xmlFile>\\...\AForm-XSD-20211013\PD220224062681.XML</xmlFile> <xsdFile>\\...\AForm-XSD-20211013\Miami-ws-AForm.xsd</xsdFile> <xsdProcessor>BaseX 10.1, EE-Java, Xerces 2.12.2</xsdProcessor> <xsdVersion>1.1</xsdVersion> </metadata> <messages> ... </messages> </root> The question is why the before and after timing is the same: <startTime>2022-09-09T11:32:29.667-04:00</startTime> <endTime>2022-09-09T11:32:29.667-04:00</endTime> They are identical up to a millisecond. My expectations were that at least the millisecond time portion would be different. Regards, Yitzhak Khabinsky

3 8

Copy Modify Return - base URI preserved?
by Hans-Juergen Rennau 07 Sep '22

07 Sep '22

Dear BaseX people, I just observed that a document obtained by applying Copy/Modify/Return to a source document preserves its base URI, that is, base-uri($resultDoc) = base-uri($sourceDoc). That is marvellous, but can I rely on this as a feature? (In comparison - if I obtained the result document in a "conventional" way by recursive processing of the source document, the base URI would be lost, unless I added an @xml:base.) Kind regards,Hans-Jürgen

2 1

Basex query for checking size of node
by Sarath Surisetty 03 Sep '22

03 Sep '22

Problem statement: one of the project stores the logs in BaseX, and size of the baseX increases nearly 1gb to 10gb . when the size of BaseX increases, while performing queries to baseX, noticed that the CPU utilization exceeded more than 200% and performance is degraded drastically, But when some of logs data removed from BaseX manually BaseX performance is good. could u help in , 1)We are getting issues from customers env , where we don't have access to analyze which node is consuming more storage or which query is consuming more cpu and customer are not supposed to use basex gui. Could you give the query(Xquery) to print list of nodes(xmls) and its sizes ? 2)could you share any suggestions to resolve the performance issues. thanks sarath Regards Sarath

1 0

Am I using proc:exec wrong?
by Graydon Saunders 03 Sep '22

03 Sep '22

Hello -- This is with BaseX 10.1 on a linux box: 18:02 graydon % java -version openjdk version "17.0.4" 2022-07-19 OpenJDK Runtime Environment (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8) OpenJDK 64-Bit Server VM (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8, mixed mode, sharing) I'm trying to invoke CoffeePot, https://coffeepot.nineml.org/ which is an ixml processor. In the BaseX GUI, proc:execute('/usr/bin/java','-version') gives me the expected version information as an error, despite a return code of zero: <result><error>openjdk version "17.0.4" 2022-07-19 OpenJDK Runtime Environment (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8) OpenJDK 64-Bit Server VM (Red_Hat-17.0.4.0.8-1.fc36) (build 17.0.4+8, mixed mode, sharing) </error><code>0</code></result> >From a console, /usr/bin/java -cp /home/graydon/bin/coffeepot-package/coffeepot-1.99.8/coffeepot-1.99.8.jar org.nineml.coffeepot.Main works fine, in as much as I get the list of coffeepot usage information I expect if it's run without arguments. proc:execute('/usr/bin/java','-cp /home/graydon/bin/coffeepot-package/coffeepot-1.99.8/coffeepot-1.99.8.jar org.nineml.coffeepot.Main') returns <result><error>Unrecognized option: -cp /home/graydon/bin/coffeepot-package/coffeepot-1.99.8/coffeepot-1.99.8.jar org.nineml.coffeepot.Main Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. </error><code>1</code></result> I suspect I'm doing something wrong, but it's not at all clear to me what. Anyone have suggestions? Thanks! Graydon

3 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk