- BaseX-Talk - mailman.uni-konstanz.de

Issues with X11 display
by Graydon Saunders 30 Apr '24

30 Apr '24

Hello -- After upgrading to Fedora 40, I find that I can not run the BaseX GUI. This is against BaseX110-20240426.163243.zip or 10.7 17:28 bin % ./basexgui /home/graydon/bin/basex/basex/.basex: writing new configuration file. Exception in thread "main" java.awt.HeadlessException: No X11 DISPLAY variable was set, or no headful library support was found, but this program performed an operation which requires it, at java.desktop/java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:166) at java.desktop/java.awt.Window.<init>(Window.java:553) at java.desktop/java.awt.Frame.<init>(Frame.java:428) at java.desktop/java.awt.Frame.<init>(Frame.java:393) at java.desktop/javax.swing.JFrame.<init>(JFrame.java:180) at org.basex.BaseXGUI.splash(BaseXGUI.java:96) at org.basex.BaseXGUI.<init>(BaseXGUI.java:54) at org.basex.BaseXGUI.main(BaseXGUI.java:37) 17:23 bin % java -version openjdk version "21.0.2" 2024-01-16 OpenJDK Runtime Environment (Red_Hat-21.0.2.0.13-4) (build 21.0.2+13) OpenJDK 64-Bit Server VM (Red_Hat-21.0.2.0.13-4) (build 21.0.2+13, mixed mode, sharing) Various attempts to set DISPLAY have not been helpful. Trying to use OpenJDK 17 produced effectively the same error message at greater length. Anyone got any suggestions on how to fix this? Thanks! Graydon

4 4

Request and JSON parsing
by Günter Dunz-Wolff 26 Apr '24

26 Apr '24

Hi there, via RESTXQ I request a vectorstore. The response is an JSON-Object. For example: With the following curl: curl https://vector.io/query-data \ -H "Authorization: Bearer ABcFMGhetc" \ -d '{ "data": "Du hast dem Menschen ein so freies, herrliches und üppiges Leben bestimmt", "topK": 2, "includeVectors": false, "includeMetadata": true }' I'm getting this object: { "result" : [ { "id" : "ba02277", "score" : 0.9949091, "metadata" : {"genre":"ba","blatt":"1810-01","autor":"Kleist","titel":"Zoroaster.","link":"https://kleist-digital.de/etc","text":"freies, herrliches und üppiges Leben bestimmt."} }, { "id" : "ba01711", "score" : 0.8891792, "metadata" : {"genre":"ba","blatt":"1810-29","autor":"Ompteda","titel":"Fragmente","link":"https://kleist-digital.de/etc","text":"So mit dem Menschen im Moralischen."} } ] } 1. Is the request in my RESTXQ in the right form? http:send-request( <http:request method="post" href="{$prod:url}/query-data"> <http:header name="Authorization" value="Bearer {$prod:token}"/> <http:body media-type="application/json"> {{ "data": "{$query}", "topK": 20, "includeVectors": false, "includeMetadata": true }} </http:body> </http:request> ) 2. I'm getting a full Object (but I dont know, if its the raw JSON-Object like above), but I am not able, to parse it, to get a list in html. In the documentation https://docs.basex.org/wiki/JSON_Module is an example: let $input := '{ "Title": "Drinks", "Author": [ "Jim Daniels", "Jack Beam" ] }' let $data := json:parse($input, map { 'format': 'xquery' }) return map:for-each($data, function($k, $v) { $k || ': ' || string-join($v, ', ') }) But my Object is more complex and I don't know, how to get two and more datasets with $k and $v out of the object. I want to have the following list: <div> <li> <ul> <li>Score: result score</li> <li>Blatt: result blatt</li> <li>Autor: result autor</li> <li>Titel: result titel</li> <li>Text: result text</li> <li>Link: result link</li> </ul> </li> <ul> <li></li> </ul> etc. </div> Thanks for any help Günter

3 5

Performance issue with BaseX CLI
by ANDRADE Antonio 22 Apr '24

22 Apr '24

Hie, For the purposes of European Water Framework Directive reporting, I compared the performances of the Saxon and BaseX XQuery engines. I observe a performance gap of a factor of 100 to 200 depending on the use case (see functions test_xquery_monitoring() and test_xquery_multischema_2022() in scripts test_saxoncee.py and test_basex.sh available at https://outil-transferts.ofb.fr/?107ae461a144d0b) Can you please help me understand the reasons for such gaps ? Thanks in advance, Antonio Andrade Date engineer

5 9

file:path-to-native() throws an error if its argument does not exist
by Imsieke, Gerrit, le-tex 22 Apr '24

22 Apr '24

I have a file:// URI that corresponds to a directory that I need to create (using svn mkdir, therefore file:create-dir() is not an option here) if it doesn’t exist. Calling file:path-to-native() on it results in a file:not-found error. Is there a fundamental reason why the file needs to exist before transforming its URI into the OS-native representation? Using BaseX 10.7. Gerrit

3 3

.xar deployment
by Régis WITZ 11 Apr '24

11 Apr '24

Hi all, I'm kind of a XQuery noob, but I'm guilty of the creation of a little BaseX module, Heimdall (10.5281/zenodo.10638084 <https://zenodo.org/records/10638084>). It's automatically built, unit tested and documented using GitLab CI. All build results, including the .xar file, are hosted using GitLab Pages. Thus, the module .xar archive is available here : |https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar| <https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar>. As you see, all this is hosted in Huma-Num's self-hosted GitLab instance (Huma-Num is a french research infrastructure). However, this mail is /not/ (just) a shameless plug. My issue is that I cannot install my .xar directly with |REPO INSTALL| : 1. I cannot install it with the CLI: basex -c"<repo-install path='https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'/>" [repo:not-found] Package 'https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar' not found. Things are perfectly fine with for example the functx package : |basex -c"<repo-install path='https://files.basex.org/modules/expath/functx-1.0.xar'/>"| 2. For gigs, I tried to change the protocol from HTTPS to HTTP, and here is the error that I get : basex -c"<repo-install path='http://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar'/>" [repo:parse] heimdall.xar: Resource "expath-pkg.xml" not found.. So, maybe there is a problem with my expath-pkg.xml file ? I didn't find any -but, as I said, I'm no pro. If needed, here is the file that is wrapped in the .xar : https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/raw/main/expath-pkg… <https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/raw/main/expath-pkg…> 3. However, when I download my .xar file from the URL, and then REPO INSTALL it using the local path, stuff works fine : wget --no-check-certificate https://datasphere.gitpages.huma-num.fr/heimdall/xquery/heimdall.xar basex -c"<repo-install path='./heimdall.xar'/>" basex -c"<repo-list />" Name Version Type Path --------------------------------------------------------------------------- http://heimdall.huma-num.fr 2.1 EXPath http-heimdall.huma-num.fr-2.1 1 package(s). So I suppose the problem /might/ not be within the .xar itself, but the way it is hosted ? Maybe REPO INSTALL doesn't like my disturbing lack of SSL certificate ? I tried to search for some clue, but didn't find any clue in the documentation <https://docs.basex.org/wiki/Repository>. Maybe I'm blind, maybe that's not the problem at all. It's not exactly a blocking issue, but my noobiness regarding the basex ecosystem (and xquery in general) prevents me to understanding what happens, my README contains false info, and I'm not at ease with that. If someone had any intel, I'd be thankful. If needed, here is the repository : https://gitlab.huma-num.fr/datasphere/heimdall/xquery/ <https://gitlab.huma-num.fr/datasphere/heimdall/xquery/> And here is the corresponding issue (in French, sorry) : https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/issues/1 <https://gitlab.huma-num.fr/datasphere/heimdall/xquery/-/issues/1> Have a nice day, sunshine, flowers and all, Régis

4 3

some eq versus =
by Leo Studer 10 Apr '24

10 Apr '24

Hello are the following queries equivalent? 1. //city[some $city in following::city satisfies string(.) eq string($city)] 2. //city[. = following::city ] Thanks in advance, Leo

3 4

Single command expected
by Ветошкин Владимир 08 Apr '24

08 Apr '24

2 2

hof:until is gone?
by Graydon Saunders 28 Mar '24

28 Mar '24

Hello -- So I'm trying to do this: import module namespace xcs = "http://www.xcential.com/xquery/utils/script" at "same-words-same-order-script.xqm"; (: we don't need xc computationally but there are external variables in that namespace in scope :) import module namespace xc = "http://www.xcential.com/xquery/utils" at 'same-words-same-order.xqm'; declare function xc:dropTableLines($in as node()*,$toggle as xs:boolean) as node()* { switch (true()) case empty($in) return () case starts-with(head($in),':stab') return (<line/>,xc:dropTableLines(tail($in),true())) case starts-with(head($in),':rtab') return (<line/>,xc:dropTableLines(tail($in),false())) case $toggle return (<line/>,xc:dropTableLines(tail($in),$toggle)) default return (head($in),xc:dropTableLines(tail($in),$toggle)) }; let $test as element() := <text> <line></line> <line></line> <line>:stab</line> <line>weasels</line> <line>:stab</line> <line>weasels</line> <line></line> <line></line> <line>:rtab.</line> <line></line> <line>asparagus</line> <line></line> <line></line> <line>:stab</line> <line></line> <line>weasels</line> <line>:rtab.</line> <line>asparagus</line> <line>asparagus</line> <line>asparagus</line> <line>asparagus</line> <line>asparagus</line> <line>:stab</line> <line>:stab</line> <line></line> <line></line> <line>:stab</line> <line></line> <line>weasels</line> <line></line> <line>:rtab.</line> <line></line> <line>asparagus</line> <line></line> <line></line> <line></line> </text> return element {'text'} { xc:dropTableLines($test/line,false()) } Only at scale the stack blows up and I get the "try tail recursion?" suggestion. I would have tried hof:until for that, since I have to pass the current state of "are we dropping or not dropping intervening content?", but it looks like it's been removed? And the available hof functions in 4 look like they're strictly positional which is actively unhelpful in this case. (At least with whatever brain cells I currently have.) What's the appropriate pattern for "process a sequence, toggling an action on or off based on the last member of the sequence we looked at?" Thanks! Graydon

6 8

error Access Denied
by Synnott, Stephane 28 Mar '24

28 Mar '24

Hi, I have a C# .net application on my local machine, I have started the BaseX server and I want to use BaseX in my .Net application but as soon as I want to create a Session: Session session = new Session("localhost", 1984, "admin", "admin"); I also tried with no username and password but it is the same result. I get an "Access Denied". What am I missing? Thanks in advance, Stephane ________________________________ IMPORTANT NOTICE: This message may contain confidential or privileged information and is intended only for the individual named. If you are not the intended recipient you should not disseminate, distribute or copy this email. Please notify the sender immediately if you have received this email by mistake and delete it from your system. AVIS IMPORTANT: Le pr?sent courriel peut contenir des renseignements confidentiels ou privil?gi?s et est strictement r?serv? ? l'usage du destinataire pr?vu. Si vous n'?tes pas le destinataire pr?vu, vous ne devez pas diffuser, distribuer ou copier ce courriel. Veuillez aviser imm?diatement l'exp?diteur si vous avez re?u ce courriel par erreur et supprimez-le de votre syst?me. ________________________________

2 2

BaseX and NFS
by Marco Lettere 20 Mar '24

20 Mar '24

Dear all, we have an instance of BaseX running on top of an NFS exported disk. Yes yes I know it's not the best possible scenario thus I was expecting a slight performance decrease. Anyway, when comparing local disk to NFS disk, for a tiny operation like storing a very small document into a database (without optimize or indexes) we get orders of magnitude of difference. And I'm saying from few hundreds of ms to several seconds in some cases. Does anyone have experience with cases like these? Or a solid motivation that explains this degradation? Thank you very much as usual. Marco.

2 1

Out of Main Memory
by Murray, Gregory 15 Mar '24

15 Mar '24

Hello, I’m working with a database that has a full-text index. I have found that if I iteratively add XML documents, then optimize, add more documents, optimize again, and so on, eventually the “optimize” command will fail with “Out of Main Memory.” I edited the basex startup script to change the memory allocation from -Xmx2g to -Xmx12g. My computer has 16 GB of memory, but of course the OS uses up some of it. I have found that if I exit memory-hungry programs (web browser, Oxygen), start basex, and then run the “optimize” command, I still get “Out of Main Memory.” I’m wondering if there are any known workarounds or strategies for this situation. If I understand the documentation about indexes correctly, index data is periodically written to disk during optimization. Does this mean that running optimize again will pick up where the previous attempt left off, such that running optimize repeatedly will eventually succeed? Thanks, Greg Gregory Murray Director of Digital Initiatives Wright Library Princeton Theological Seminary

4 8

validate:xsd() questions
by Murray, Gregory 11 Mar '24

11 Mar '24

Hello, What is the default value of the “cache” option for the validate:xsd() function? The default doesn’t seem to be indicated in the documentation for that function. Also, the following code raises an error message saying “Item expected, empty sequence found” but that seems odd since the second argument to validate:xsd() is supposed to be optional, thereby telling the validation process to fall back to using the schema indicated in the document itself. let $options := map {'cache' : true()} for $doc in collection() return validate:xsd($doc, (), $options) Is this a bug or have I misunderstood something? Thanks, Greg Gregory Murray Director of Digital Initiatives Wright Library Princeton Theological Seminary

2 1

xslt:transform-report result
by Andy Bunce 11 Mar '24

11 Mar '24

Hi, Using BaseX 9.7.1 and saxon9he-9.9.1.jar The documentation suggests the ?result from xslt:transform-report should be *a* document-node where possible [1] This seems not quite to be the case when there are processing instructions or comments at the top level. In these cases a sequence of document-nodes is returned. /Andy [1] https://docs.basex.org/wiki/XSLT_Module#xslt:transform-report let $xslt:=<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="3.0"> <xsl:mode on-no-match="shallow-copy"/> </xsl:stylesheet> let $xml:=document{ <?something type="compact"?>, <foo/>} return xslt:transform-report($xml,$xslt) Returns map { "messages": (), "result": (<?something type="compact"?>, <foo/>) }

3 7

Troubles with ft:mark
by Jack Steyn 07 Mar '24

07 Mar '24

Hi BaseXers, Consider the following query run on a database 'factbook' containing factbook.xml with full-text indexing: declare function local:search( $database as xs:string, $query as xs:string ) { let $country-search := ft:search($database, $query)/ancestor::country let $city-search := ft:search($database, $query)/ancestor::city/ancestor::country let $other-search := ft:search($database, $query)/parent::*[name() = ('ethnicgroups', 'languages')]/ancestor::country let $country-mark := $country-search[.//name[text() contains text { $query }]] => ft:mark() let $city-mark := $city-search[.//city[text() contains text { $query }]] => ft:mark() let $other-mark := $other-search[.//*[name() = ('ethnicgroups', 'languages')][text() contains text { $query }]] => ft:mark() return ( $country-mark, $city-mark, $other-mark ) }; local:search('factbook', 'German') The first result of this query is the entry for Austria. I would expect both of the instances of the word 'German' in that entry to be surrounded by tags. However only the first instance is. Some changes that produce the correct result for the Austria entry: - replacing $database with 'factbook' in the definition of $other-search, or - deleting either $country-mark or $city-mark in the sequence returned. If you make either of the above changes, but additionally change the predicate [text() contains text { $query }] to [ft:contains(text(), $query)], the incorrect result returns. Any idea what might be causing this? I'm running BaseX 10.7. Cheers, Jack

2 5

Inconsistent behavior for root context in BaseX GUI
by Thompson, Timothy 06 Mar '24

06 Mar '24

I’ve been noticing an inconsistent behavior in the GUI with BaseX 11 beta (build 17d8426). I have a simple query on a database created from a CSV file. Sometimes it works, but sometimes the root context is evaluated as an empty sequence: - rewrite context value: . -> () - rewrite util:root(nodes): util:root(()) -> () I notice this when I open the database as context and try to execute the query. After the query fails, if I try to click on the “home” icon in the result panel, it also returns an empty sequence, even though the database is open. If I reopen the database and run the query, it works again. Sample data: <csv> <record> <entry>http://id.loc.gov/authorities/names/n00000121</entry<http://id.loc.gov/authorities/names/n00000121%3c/entry>> <entry>http://id.loc.gov/rwo/agents/n00000121</entry<http://id.loc.gov/rwo/agents/n00000121%3c/entry>> <entry>0.11861849</entry> </record> <record> <entry>http://id.loc.gov/authorities/names/n00000122</entry<http://id.loc.gov/authorities/names/n00000122%3c/entry>> <entry>http://id.loc.gov/rwo/agents/n00000122</entry<http://id.loc.gov/rwo/agents/n00000122%3c/entry>> <entry>0.11699477</entry> </record> <record> <entry>http://id.loc.gov/authorities/names/n00000267</entry<http://id.loc.gov/authorities/names/n00000267%3c/entry>> <entry>http://id.loc.gov/rwo/agents/n00000267</entry<http://id.loc.gov/rwo/agents/n00000267%3c/entry>> <entry>0.10811427</entry> </record> </csv> Query (with DB open as context): count(distinct-values( for $rec in /csv/record where contains($rec/entry[1], "id.loc.gov") return $rec/entry[1] )) Thanks in advance, Tim -- Tim A. Thompson (he, him) Librarian for Applied Metadata Research Yale University Library www.linkedin.com/in/timathompson<http://www.linkedin.com/in/timathompson>

2 3

DBA Editor behavior in v11 not as expected
by Eliot Kimber 04 Mar '24

04 Mar '24

Using the 27-02-2024 build of v11, if I create a query in the Editor panel and then save it, the Run button is disabled, which is unexpected. If I then open the file, the Run button remains disabled. If I cut the contents of the editing panel, select “close” to clear the panel, then paste back into the panel, the Run button is enabled. I think this behavior is incorrect—as long as there is some content in the editor the Run button should be enabled. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 5

Inconsistency in base-uri()
by Eliot Kimber 04 Mar '24

04 Mar '24

Using BaseX 11 (but I think the code is the same in BaseX 10). I’m trying to understand how base-uri() behaves relative to how it should behave when the database path of a document is not a valid URI, i.e., it has a space in it. First I have this test: let $doc as document-node() := document { <root><child xml:base="temp/child-uri%20with%20space.xml">child</child></root> } return $doc/*/child ! base-uri(.) Which produces: file:///data/basex/data/.dba/temp/child-uri%20with%20space.xml Which is the correct result: it’s the value of @xml:base and the escaped spaces make it a valid URI. Replacing %20 with “ “ in the @xml:base value results in this error: Invalid URI: Illegal character in path at index 14: temp/child-uri with space.xml. Also correct as the spaces have to be escaped. This verifies that base-uri() applied to nodes with explicit @xml:base attributes work per the spec. But this test does not involve database paths. To try to test things with database paths I then created this pair of test scripts: Script to put docs in a database: let $db := 'temp' let $filename as xs:string := 'with space.xml' let $doc1 as document-node() := document {<root><child>No xml:base</child></root>} let $doc2 as document-node() := document { <root><child xml:base="{'/temp/xmlbase/doc2_' || $filename}">With xml:base unescaped</child></root> } let $doc3 as document-node() := document { <root><child xml:base="{iri-to-uri( '/temp/xmlbase/doc3_' || $filename)}">With xml:base escaped</child></root> } return (() ,db:put($db, $doc1, 'doc1_' || $filename) ,db:put($db, $doc2, 'doc2_' || $filename) ,db:put($db, $doc3, 'doc3_' || $filename) ) Script to report on them: let $db := 'temp' let $filenameBase as xs:string := 'with space.xml' return for $i in 1 to 3 let $filename := 'doc' || $i || '_' || $filenameBase let $doc := db:get($db, $filename) let $child as element() := $doc/*/child let $dbPath := db:path($doc) let $baseUriDoc := base-uri($doc) let $baseUriChild := try { base-uri($child) } catch * { $err:description } return (() ,``[ Doc "`{$dbPath}`":]`` ,$doc ,``[xml:base att: "`{$child/@xml:base}`"]`` ,``[base URI of doc: "`{$baseUriDoc}`"]`` ,``[base URI of child: "`{$baseUriChild}`"]`` ) Which returns this result: Doc "doc1_with space.xml": <root> <child>No xml:base</child> </root> xml:base att: "" base URI of doc: "/temp/doc1_with space.xml" base URI of child: "/temp/doc1_with space.xml" Doc "doc2_with space.xml": <root> <child xml:base="/temp/xmlbase/doc2_with space.xml">With xml:base unescaped</child> </root> xml:base att: "/temp/xmlbase/doc2_with space.xml" base URI of doc: "/temp/doc2_with space.xml" base URI of child: "Invalid URI: Illegal character in path at index 23: /temp/xmlbase/doc2_with space.xml." Doc "doc3_with space.xml": <root> <child xml:base="/temp/xmlbase/doc3_with%20space.xml">With xml:base escaped</child> </root> xml:base att: "/temp/xmlbase/doc3_with%20space.xml" base URI of doc: "/temp/doc3_with space.xml" base URI of child: "Invalid URI: Illegal character in path at index 15: /temp/doc3_with space.xml." Note the result for doc3: It’s reporting the base URI of the document (/temp/doc3_with space.xml), not the base URI of the child (/temp/xmlbase/doc_with%20space.xml). Why? I think the answer is that under the covers it’s doing resolve-uri(), which also checks the validity of both the base and relative parts. One observation is that base-uri() is treating the db-provided base URI differently from an xml:base-provided base URI, but only when there is no @xml:base attribute. In doc 1, the database path has a space but base-uri() does not fail when returning it even though it’s not a valid URI. Why not? In doc 2, the xml:base-supplied base URI is correctly reported as invalid, but the database-supplied base URI of the root is not reported as invalid. My expectation would be that the behavior is consistent: Either all URIs must be valid, including those coming from database paths or all are automatically escaped (as though iri-to-uri() had been applied). Finally, why do I get the result for doc 3, where it’s reporting the database path as the base URI of the child rather than the @xml:base-defined base URI (which is correctly escaped). In my code, which depends on the use of @xml:base to do DITA link resolution for “resolved” DITA maps, I’ve adjusted my code to escape URIs in @xml:base values and as far as I can tell everything works as it should. But I’m still concerned about the inconsistency in the behavior of base-uri(). I tried to trace through the code that handles base-uri() but it’s pretty twisty and does different things for files and nodes. It would obviously be very disruptive to have base-uri() start failing on database paths with spaces—I think the current behavior dates back to the very start of BaseX, but it’s still an inconsistency that can lead to trouble with the unawares. For example, consider this code: let $topicref := db:get('maps', 'map with space.ditamap')/*/topicref[@href][1] let $target as element()? := local:resolve-href($topicref) let $baseUri as xs:string := base-uri($target) ! string(.) let $newElem as element := <submap xml:base="{$baseURI}"/> return base-uri($newElem) The value of $baseUri will be “map with space.ditamap”, not “map%20with%20space.ditamap”, making the value of @xml:base on $newElem: xml:base=”map with space.ditamap”, meaning that base-uri($newElem) will throw an invalid URI exception. My expectation would be either that base-uri($target) also throws an exception or, more usefully, that it returns the iri-to-uri() result, ensuring that the values will always be treated as valid URIs, consistent with how the database paths are treated. Cheers, E.

2 2

Possible lingering issues with database optimization in v11
by Eliot Kimber 04 Mar '24

04 Mar '24

I’m continuing to test my process for loading data which depends on optimizing databases, some of which are pretty large (100+MB with 100s of 1000s of elements). I’m testing on both macOS and linux using the 27 Feb build on macOS and the 29 build on linux (just a matter of when I downloaded them). What I’m seeing is that when I test with a relatively small content set the optimization completes reliably and everything works as it should. When I test with a realistically large data set, the optimization either takes a very long time (as much as an hour to complete) or never completes with the server at 100% CPU utilization. It seems to be worse on macOS but it’s difficult to verify, partly because a test takes several hours to run. I have the BaseX source code available locally, although I’m unable to compile with maven due to internal maven issues (we have a pretty locked down maven proxy and I don’t know maven well enough to know how I might configure around that). Is there anything I can do to diagnose this issue to at least confirm or deny that there are still deadlock issues with the optimization? I assume that it should not take 10s of minutes to optimize even a large database. Here's the details for a typical database that has failed to optimize on macOS: [cid:image001.png@01DA6D49.2FC41230] Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 2

Optimize database never returns, leaves database in "opened by another process" state
by Eliot Kimber 04 Mar '24

04 Mar '24

Using Basex 10.7 on Linux. I’m running a sequence of jobs to update and optimize a set of databases following loading a number of documents created dynamically (as opposed to being read from disk). I’m seeing a new behavior, which is that the optimization step never completes but also doesn’t show any error in the log. The database shows no items and is in the locked by another process state if I try to drop it. This behavior seems to be consistently repeatable with my current code base (I’m working on some code updates, so it’s possible I’ve introduced something that would cause this behavior but I haven’t changed the code that leads up to the failing optimize). The server has plenty of disk space, etc. Optimization code is: try { if (db:exists($database)) then ( util:logToConsole('dbadmin:optimizeDatabase', ``[Optimizing database `{$database}`]``), db:optimize($database, true(), $dbadmin:dbOptimizeOptions) ) else util:logToConsole('dbadmin:optimizeDatabase', ``[Database '`{$database}`' does not exist. Nothing to optimize.]``) } catch * { util:logToConsole( 'dbadmin:optimizeDatabase', ``[Exception optimizing database '`{$database}`': `{$err:code}` - `{$err:description}`]``, 'error') } And the optimization options are: declare variable $dbadmin:dbOptimizeOptions as map(*) := (: Turn on all the indexes :) map { 'attrindex' : true(), 'tokenindex' : true(), 'textindex' : true(), 'ftindex' : true() }; This code has been working fine for a long time and I’ve been running 10.7 for a least a couple of months, so I’m wondering: A) Would would cause this behavior? B) How can I diagnose it short of debugging the Java code (which I can do but it’s non-trivial for me to set up). Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

3 10

Re: [basex-talk] Item Separator
by Leo Studer 29 Feb '24

29 Feb '24

Dear Martin Thank you for investigating. > The "text1, , text2" I only see if oXygen shows the result as a sequence in its result viewer; I would suggest to ask the oXygen support why you get that result for that case, it seems like some quirk in their result viewer (it shows two xs:string items but for reasons I can't investigate serializes the two items for the result viewer as "text1, , text2"). > This is incredible news. I will get to Oxygen about that. Really troubling … Blessings, Leo

1 0

Odd "Access denied" messages in the log
by Eliot Kimber 28 Feb '24

28 Feb '24

I have an application (our Mirabel system) running on a server inside our firewall (so not visible to the open Internet). I’ve recently started seeing messages like this in the log: Access denied: ��. Access denied: PRI * HTTP/2.0 SM . Where the value reported can be quite varied, but is often unrenderable characters or other stuff. (In this case the characters are all \uFFFD). The log messages all report the same IP address. This server does not use named users, so there’s no authentication required to access it. The IP address is not one of my own servers, so I don’t think it’s something generated by my own code. Any idea what this might be? It’s started relatively recently, which makes me think it might be some sort of penetration test. Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

Item Separator
by Leo Studer 27 Feb '24

27 Feb '24

Hello I am serialising the following expressions on oxygen/saxonEE12.3 as text (item-separator=“”) and wonder why the differente outputs let $xml:=<xml><element>text1</element><element>text2</element></xml> return $xml/element/text() gives text1text2 And let $xml:=<xml><element>text1</element><element>text2</element></xml> return $xml/element/string() text1 text2 When I set the item-separator =“, “ then I get text1, text2 Or text1, , text2 Any explanations??? Thanks, Leo

3 3

file:write, formatting, and maps in select attributes
by Graydon Saunders 24 Feb '24

24 Feb '24

Hello -- I'm trying to regularize the formatting of some XSLT files by loading them (via doc()) and writing them back out via file:write() I know about the *indent* and *indent-attributes* serialization parameters, and mostly they're doing what I want. What I'd like is to preserve a one-line-per-entry format for a map in a select attribute. For example, I'd like to keep <xsl:variable as="map(xs:integer,xs:string)" name="hexNum" select=" map { 1: '0', 2: '1', 3: '2', 4: '3', 5: '4', 6: '5', 7: '6', 8: '7', 9: '8', 10: '9', 11: 'a', 12: 'b', 13: 'c', 14: 'd', 15: 'e', 16: 'f' }" static="yes" /> instead of <xsl:variable as="map(xs:integer,xs:string)" name="hexNum" select=" map { 1: '0', 2: '1', 3: '2', 4: '3', 5: '4', 6: '5', 7: '6', 8: '7', 9: '8', 10: '9', 11: 'a', 12: 'b', 13: 'c', 14: 'd', 15: 'e', 16: 'f' }" static="yes"/> I don't think there's a way to do this with the serialization parameters, but is there a way to do this? Thanks! Graydon -- Graydon Saunders | graydonish(a)fastmail.com Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.")

3 3

basex failed to cast large numbers as xs:integer
by mx z 22 Feb '24

22 Feb '24

Hi, I found a bug in basex. Version: 1ce0618 (2024.02.22) Java: openjdk 17.0.9 2023-10-17 LTS OpenJDK Runtime Environment Corretto-17.0.9.8.1 (build 17.0.9+8-LTS) OpenJDK 64-Bit Server VM Corretto-17.0.9.8.1 (build 17.0.9+8-LTS, mixed mode, sharing) OS: Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy input.xml: <a xmlns:C="p"> <C:A id="1">1458820431</C:A>1438466813 </a> query.xq: //*[not(. castable as xs:integer)] Reproduce: java -cp basex-core/target/basex-11.0-SNAPSHOT.jar org.basex.BaseX -i input.xml query.xq It printed out <a xmlns:C="p"> <C:A id="1">1458820431</C:A>1438466813 </a> While basex should return empty, as both 1458820431 and 1438466813 are integers. Max Zhang

2 1

Found problems with map:for-each
by Vincenzo Cestone 20 Feb '24

20 Feb '24

Hi all, with the basex 10.7 version (but even in 9.6 version I have the same issue) I found that the following code wont work as expected: (: It wont work :) let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='), "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')} let $w := map:for-each($files, function($filename, $content) { file:write-binary($filename, $content, 0) }) return () that is, it will not write the two files hello1.txt and hello2.txt in the basex_home/bin folder. But, if you return $w instead: (: It works :) let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='), "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')} let $w := map:for-each($files, function($filename, $content) { file:write-binary($filename, $content, 0) }) return $w With a similar implementation, with the classic FLWOR, the issue does not arise, even if I return the empty sequence: (: It works :) let $files := map {"hello1.txt" : xs:base64Binary('SGVsbG8gd29ybGQ='), "hello2.txt" : xs:base64Binary('SGVsbG8gd29ybGQ=')} let $w := for $filename in map:keys($files) return file:write-binary($filename, map:get($files, $filename), 0) return () that is it will write two files hello1.txt and hello2.txt in basex_home/bin folder. Note that the files in the examples above are for demonstration purposes, however in my code they are actually binary files. My java version is a Oracle JDK 17 This problem also occur to others? Thanks, Vincenzo

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk