- BaseX-Talk - mailman.uni-konstanz.de

Error with function-lookup and static variable
by Amanda Galtman 20 Sep '24

20 Sep '24

Hi, all. I'm seeing an error in BaseX when I use function-lookup in both a global variable and a function, where the function relies on the variable. I reduced the situation to the following small query: xquery version "3.1"; declare variable $local:lookup := function-lookup(QName("nonexistent", "nonexistent"), 1); declare function local:myfcn() { let $f := ($local:lookup, function-lookup(QName('http://www.w3.org/2005/xpath-functions','string'), 1))[1] return $f('a') }; local:myfcn() When I run it with BaseX 11.1, I get [XQDY0054] Static variable depends on itself: $local:lookup When I run it with Saxon-HE 12, I don't get this error. Is there anything I can do in my code to avoid this error? Thanks, Amanda

4 13

Unknown function db:open
by Kendall Shaw 17 Sep '24

17 Sep '24

I'm using basex 11.2 under WSL ubuntu 24.04 when I use the command xquery count(db:open('some')/*), I see an error message about db prefix being undefined. The same thing from an editor tab. import module namespace db = 'http://basex.org/modules/db'; count(db:open('some')/*) I get the same error message. Is this to be expected? How can I supply what is needed? Thanks, Kendall Shaw

2 2

Re: [basex-talk] BaseX-Talk Digest, Vol 177, Issue 2
by Rob Stapper 12 Sep '24

12 Sep '24

> Op 11-09-2024 12:00 CEST schreef basex-talk-request(a)mailman.uni-konstanz.de: > > > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. terrible performance of webserver under BaseX11 (Rob Stapper) > 2. Re: terrible performance of webserver under BaseX11 > (Christian Gr?n) > 3. Re: terrible performance of webserver under BaseX11 > (Christian Gr?n) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 10 Sep 2024 15:31:52 +0200 (CEST) > From: Rob Stapper <r.stapper(a)lijbrandt.nl> > To: "basex-talk(a)mailman.uni-konstanz.de" > <basex-talk(a)mailman.uni-konstanz.de> > Subject: [basex-talk] terrible performance of webserver under BaseX11 > Message-ID: <1197257124.2620908.1725975112156(a)kpc.webmail.kpnmail.nl> > Content-Type: text/plain; charset="utf-8" > > Hi Christian, > > My webserver-app that performed reasenably well under BaseX10 performs terribly under Basex11. Chrome even gives me at the end of the process an 'out of memory' message. So it is not only performance but also resource-usage. I have absolutely no clou how to address this, so all suggestions are very welcome. > > mvgr. > > Rob Stapper >

2 1

Re: [basex-talk] terrible performance of webserver under BaseX11
by Christian Grün 10 Sep '24

10 Sep '24

(cc to the list) For a more helpful response, we would certainly need to look at your code and run it by ourselves. I recommend you to run your function… $SWM:dialog.new …outside your web application, e.g. in the BqseX GUI. This may help you to isolate the problematic part of the code.

1 0

terrible performance of webserver under BaseX11
by Rob Stapper 10 Sep '24

10 Sep '24

Hi Christian, My webserver-app that performed reasenably well under BaseX10 performs terribly under Basex11. Chrome even gives me at the end of the process an 'out of memory' message. So it is not only performance but also resource-usage. I have absolutely no clou how to address this, so all suggestions are very welcome. mvgr. Rob Stapper

2 1

Unexpected error: Improper use? Potential bug? Your feedback is welcome:
by Rob Stapper 03 Sep '24

03 Sep '24

Hi Christian, After the hollidaybreak I found the courage to try to port my application to BaseX's version 112. I get the error-dump shown below. Any idea what is going wrong here? error-dump: ========= Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 11.2 Java: Eclipse Adoptium, 17.0.8 OS: Windows 10, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 3 out of bounds for length 3 at org.basex.query.func.Closure.setSignature(Closure.java:418) at org.basex.query.func.StaticFuncs$FuncCache.init(StaticFuncs.java:268) at org.basex.query.func.StaticFuncs.check(StaticFuncs.java:83) at org.basex.query.QueryParser.check(QueryParser.java:270) at org.basex.query.QueryParser.parseLibrary(QueryParser.java:198) at org.basex.query.QueryContext.lambda$parseLibrary$1(QueryContext.java:236) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.parseLibrary(QueryContext.java:234) at org.basex.query.QueryContext.parse(QueryContext.java:178) at org.basex.http.web.WebModule.qc(WebModule.java:100) at org.basex.http.web.WebModule.parse(WebModule.java:57) at org.basex.http.web.WebModules.parse(WebModules.java:376) at org.basex.http.web.WebModules.parse(WebModules.java:367) at org.basex.http.web.WebModules.parse(WebModules.java:367) at org.basex.http.web.WebModules.cache(WebModules.java:336) at org.basex.http.web.WebModules.find(WebModules.java:149) at org.basex.http.web.WebModules.restxq(WebModules.java:116) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1410) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:529) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1381) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1303) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.Server.handle(Server.java:563) at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) at java.base/java.lang.Thread.run(Unknown Source) mvgr. Rob Stapper

2 1

Alternatives to JTattoo in combination with BaseX GUI?
by Zimmel, Daniel 03 Sep '24

03 Sep '24

Hi, I noticed that JTattoo is no longer working properly with BaseX 11 (which is not a surprise because it looks like an abandoned project). I liked to use it to force a dark theme to the GUI. Does anybody know of alternatives? Are there any? Thanks, Daniel

2 2

xquery insert via REST API
by Liedtke, Alexander 28 Aug '24

28 Aug '24

Hallo in die Runde, vielleicht kann mir hierzu jmd einen Tipp geben. Ich nutze die Rest API um via post xqueries (mit python requests) zu machen - das klappt mit z.B. replace auch problemlos. Gerne würde ich jedoch auch insert nutzen - allerdings bekomme ich hier grundsätzlich eine 500 response. Die query mache ich gemäß der Doku nach dem Beispielschema: insert nodes (<phone/>, <mail/>) into //name In den Logs von dba sehe ich hierzu nichts. Ich bekomme auch sonst keine Info auf der shell in Form einer Fehlermeldung. Der Test Request sieht wie folgt aus: xq = '<query><text> insert node <test/> into //object[@id="574"]</text></query>' url = 'http://localhost:8080/rest/ids/ObjectIDs.xml' header = {'Media-Type': 'application/xml', 'charset':'utf-8'} res = requests.post(url, headers=header, data=xq, auth=_auth.getBaseXAuth()) Hat hierzu vielleicht jemand Anmerkungen / Ideen? Vielen Dank im Voraus Alexander Liedtke

1 1

Basex java.lang.ArrayIndexOutOfBoundsException
by Chavez, Robert 19 Aug '24

19 Aug '24

Greetings, We are suddenly seeing the following error message when using the REST API and also when using to Web GUI when trying to list the contents of a database by clicking on the database name and also when clicking on “optimize” (screen shot of gui attached). Does anyone have experience with hw to handle this? Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64 Thank you, Robert Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de<mailto:basex-talk@mailman.uni-konstanz.de> Version: BaseX 10.4 Java: Amazon.com Inc., 17.0.12 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 2147483647 out of bounds for length 17322 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:514) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:474) at org.basex.io.random.TableDiskAccess.read5(TableDiskAccess.java:180) at org.basex.data.Data.textRef(Data.java:467) at org.basex.data.DiskData.text(DiskData.java:272) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:96) at org.basex.query.func.db.DbListDetails$2.get(DbListDetails.java:1) at org.basex.query.func.fn.FnSubsequence.value(FnSubsequence.java:108) at org.basex.query.expr.IterMap.value(IterMap.java:106) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:78) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.constr.Constr.add(Constr.java:72) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.List$1.iter(List.java:213) at org.basex.query.expr.List$1.next(List.java:181) at org.basex.query.QueryContext.next(QueryContext.java:375) at org.basex.query.expr.constr.Constr.add(Constr.java:73) at org.basex.query.expr.constr.CElem.item(CElem.java:149) at org.basex.query.expr.constr.CElem.item(CElem.java:1) at org.basex.query.expr.ParseExpr.value(ParseExpr.java:51) at org.basex.query.func.StaticFuncCall.evalArgs(StaticFuncCall.java:147) at org.basex.query.func.FuncCall.value(FuncCall.java:53) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:82) at org.basex.query.scope.MainModule$1.next(MainModule.java:55) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:79) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:72) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1589) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:840)

2 4

recursively used variables
by Rob Stapper 12 Aug '24

12 Aug '24

Hi, The code[1] below and send as attachment generates a error message: “Static variable depends on itself: $Q{http://www.w3.org/2005/xquery-local-functions}test”. I use these variables to refer to my private functions in my modules so I can easyly refer to them in a inheritance situation. It’s not a big problem for me but I was wondering if the error-triggering is justified or that it should work. [1]=========================================== declare variable $local:test := local:test#1 ; declare %private function local:test( $i) { if ( $i > 0) then $local:test( $i - 1) } ; $local:test( 10) =========================================== Kind regards, Rob Stapper Sent from Mail for Windows 10 -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

4 7

Bug report
by Csaba Fekete 12 Aug '24

12 Aug '24

Hello The following works on Basex 10.7 but fails on Basex 11.1. The command I'm trying to run is: /opt/basex/bin/basex -v -c'OPEN mydb; ADD "/tmp/ZTH4ZRO.xml"' I've attached the XML file.The output I'm getting: Database 'mydb' was opened in 54.52 ms. Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 11.1 Java: Ubuntu, 11.0.19 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: arraycopy: length -1 is negative at java.base/java.lang.System.arraycopy(Native Method) at org.basex.util.Array.insert(Array.java:99) at org.basex.util.list.ObjectList.insert(ObjectList.java:153) at org.basex.index.resource.Docs.insert(Docs.java:166) at org.basex.index.resource.Resources.insert(Resources.java:71) at org.basex.data.Data.indexAdd(Data.java:1098) at org.basex.data.Data.insert(Data.java:836) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:291) at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:275) at org.basex.core.cmd.Add.lambda$run$0(Add.java:62) at org.basex.core.cmd.ACreate.update(ACreate.java:90) at org.basex.core.cmd.Add.run(Add.java:56) at org.basex.core.Command.run(Command.java:233) at org.basex.core.Command.execute(Command.java:93) at org.basex.api.client.LocalSession.execute(LocalSession.java:131) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:94) at org.basex.core.CLI.execute(CLI.java:78) at org.basex.core.CLI.execute(CLI.java:65) at org.basex.BaseX.<init>(BaseX.java:82) at org.basex.BaseX.main(BaseX.java:44)

2 3

Typos in BaseX documentation
by Amanda Galtman 09 Aug '24

09 Aug '24

Hi, BaseX team. I found a couple of typos in the BaseX documentation. I wouldn't mind fixing them myself if I knew how to do so in the new RESTXQ format. Is the source in GitHub, and are BaseX users encouraged to contribute corrections to the documentation? Here are the details, in case you'd rather fix them yourself. Typo 1: The page https://docs.basex.org/main/Standard_Functions starts with a stray "f" ("fThis page presents") Typo 2: On the page https://docs.basex.org/main/File_Functions, the following example is missing a closing curly brace before the final parenthesis: file:write('data.bin', xs:hexBinary('414243'), { 'method': 'basex') Thanks, Amanda

2 1

Ignoring non-fatal XML syntax issues in third-party HTML from command line
by Ross Goldberg 08 Aug '24

08 Aug '24

I am trying to use an xpath passed to basex on the macOS command line to extract data from a downloaded third-party HTML file. The file isn't completely valid xml syntax (but only in elements that aren't referenced by my xpath), so basex outputs errors instead of the data that I want. How can I configure basex to ignore xml syntax issues that aren't fatal for my xpath? If it cannot be so configured: - could that option be added as a new feature? - in the meantime, what other macOS command-line xpath parsers can ignore non-fatal XML syntax issues? Which of those support the newest versions of xpath, are performant, etc.? Thanks. Sent with Proton Mail secure email.

2 1

Performance & scaling
by Csaba Fekete 29 Jul '24

29 Jul '24

Hi I have read the Advanced User Guide section in the documentation and I got a lot of useful information but I'm still a little bit stuck. I have large (1 GB+) mostly single document databases and complex queries that often take too long to execute. Apart from optimizing the queries, what are my options to speed things up? Any ideas are welcome. 1) Vertical scaling - how does a single BaseX instance respond to adding more CPUs or RAM? Which one could improve performance? Apart from indexing, are there any fine-tuning options that could help improve performance? 2) Partitioning: would it make sense to break my documents into several smaller ones, run the queries against them in parallel and then merge the results? Has anyone done such optimization? Did it work? 2) Horizontal scaling: the docs read: *"The BaseX client-server architecture offers ACID-safe transactions, with multiple readers and writers."* - Where can I get a guide on how to set up a single writer - multiple readers architecture? Thank you Csaba

2 1

Duplicate IDs and function id()
by Andrew Sales 26 Jul '24

26 Jul '24

Hello, If I understand the spec for this function[1] correctly, at most one element should be returned for a given ID value: "If several elements have the same ID value, then E is the one that is first in document order." ("Rules", item 3) BaseX behaves differently from e.g. Saxon in this respect. The query: let $doc := document{<foo xml:id='a1'><bar xml:id='a1'/></foo>} return ( 'Saxon-HE 11.6:', xslt:transform($doc, <xsl:stylesheet xmlns:xsl=" http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs" version="3.0"> <xsl:template match="/"> <xsl:sequence select="id('a1')"/> </xsl:template> </xsl:stylesheet>), 'BaseX 10.7:', $doc/id('a1') ) returns: Saxon-HE 11.6: <foo xml:id="a1"><bar xml:id="a1"/></foo> BaseX 10.7: <foo xml:id="a1"><bar xml:id="a1"/></foo> <bar xml:id="a1"/> i.e. Saxon returns only the first in document order, whereas BaseX returns both of them. Grateful for your thoughts, Andrew [1] https://www.w3.org/TR/xpath-functions-31/#func-id

2 1

BaseX 9.7 to BaseX 11: find a regex in an external file
by Josselin Morvan 19 Jul '24

19 Jul '24

Hi Everyone, I'm migrating a webapp from BaseX 9.7 to BaseX 11 and I'm having a problem with a wrapper function (just a function for adding content to an xml template file) Basically, this function looks for nodes in a template.xml file that matche regex to update the content of this file. Here is a small example : template.xml <foo> <bar>{content}</bar> </foo> Query: let $file := 'template.xml' let $wrap := fn:doc($file) let $regex := '\{(.+?)\}’ (: looking for something between curly braces :) return for $node in $data//* where $node[fn:matches(text(), $regex)] return $node This function worked with BaseX 9.7, but returns an error with BaseX 11 : [XPTY0004] Item expected, sequence found: ("
 ", "
"). However, if I put the content of template.xml in a variable, it works (even with BaseX 11): let $wrap := <foo> <bar>{{content}}</bar> (: just escaped curly braces :) </foo> let $regex := '\{(.+?)\}' return for $node in $wrap//* where $node[fn:matches(text(), $regex)] return $node (:<bar>{content}</bar>:) Do you have any idea of what I'm doing wrong or what I’m missing? Thanks for your help! Best, Josselin

2 2

Server becomes unresponsive after several large database updates (db:delete())
by Eliot Kimber 17 Jul '24

17 Jul '24

(Using 10.7 on Centos): I have some databases with many large documents (1000s of docs ranging from 6KB to 2MB). I need to clean out older files, so I wrote this simple query, which looks at the time stamps in the doc filenames: let $db := '_valrep_validation_reports_internal' let $paths := (db:open($db)/* ! db:path(.)[matches(., '2024-0[123456]')] => sort()) return for $path in $paths return db:delete($db, $path) With only the dba app deployed, I ran this query on the four databases and it ran in a few seconds and removed the files as expected, but then I ran it on one more database and the query never returned and the dba app became unresponsive (Databases view doesn’t refresh). On the server I can see that a Java process, which can only be BaseX on this server, is at 100%: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8376 eliot.k+ 20 0 9251688 1.1g 16124 S 101.0 3.5 14:02.42 java What would cause this kind of behavior? Cheers, E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer Digital Content & Design O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

1 0

Query optimization: What can I check or measure?
by Eliot Kimber 16 Jul '24

16 Jul '24

I am loading page view data exported from Adobe as CSV as XML with a structure like: <csv timestamp="2024-01-24"> <record language="en-US" family="vancouver" bundlename="bundle-platcap-platform-capabilities" topicpath="administer/general/concept/capabilities-bundle-landingpage.dita" notviewed="viewed" viewcount="120587"> <entry name="URL">docs.servicenow.com/bundle/vancouver-servicenow-platform/page/administer/ge…</entry> <entry name="Page Views">120587</entry> <entry name="Unique Visitors">50863</entry> <entry name="Bounce Rate">0.255523003</entry> <entry name="Content Velocity">3.289644475</entry> <entry name="Docs Engagement Rate">0.289549233</entry> </record> … 50K more records … </csv> For this database I have a token index and the MAXLEN value is set to 255 to ensure that the @topicpath values will be tokenized (none should be longer than 255). I’m measuring a consistent 0.3 seconds for this query: let $docRecords as element()* := db:token($analyticsmgmt:analyticsDatabase, $docPath, 'topicpath')/.. Where $docPath is a value that will match tokens in the @topicpath attribute. Based on other token-based optimizations I’ve done, I would expect faster results, at least 10-times faster. My questions: 1. What can I look for that might be making this lookup slower than expected or, conversely, how can I prove that this is the fastest the query will return? 2. Is there a better way to manage this kind of tabular data with many 1000s of records within BaseX? For example, does it matter if the records are part of a single document or would it be better to have each record be a separate document? Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer Digital Content & Design O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 4

How can I reset my password for the basex website?
by Ben Engbers 16 Jul '24

16 Jul '24

I wanted to add a link in the server clients page for my C++ shared lib but iIt seems that my password has changed. When trying to login I only see a message that I should check my login data. How can I reset that password? Ben Ben Engbers

2 3

Puzzled by timing difference of two similar queries
by Bernhard Liebl 16 Jul '24

16 Jul '24

Hello, as a rather unexperienced xquery and basex user, I'm puzzled by the timings of running the same xquery on two very similar xml files. For an academic paper, I'm trying to benchmark finding certain successive (windowed) <item> tags inside a <seq> tag. This is the full query: declare function local:check-sequence($items as element(item)*) as element(item)* { if ($items[1]/@x = $items[2]/@x and $items[2]/@x = $items[3]/@x) then if (sum($items/@y) < 0.5) then $items else () else () }; for $seq in doc("data.xml")//seq return <seq>{ for $i in 1 to (count($seq/item) - 2) let $window := subsequence($seq/item, $i, 3) return <match>{local:check-sequence($window)}</match> }</seq> Here's the puzzling thing. When I run this on an xml file that has one <seq> with 10000 <item>s, i.e. <seqs><seq> <item>[*10000] </seq></seqs>, I get the following reasonable timings: - Parsing: 0.63 ms - Compiling: 0.95 ms - Optimizing: 30.59 ms - Evaluating: 11.48 ms - Printing: 0.61 ms - Total Time: 44.27 ms However, once I run the same query on an xml file with two (!) <seq> of 10000 items each, the runtime does not double, but is now more than 50 times of that of the first query: - Parsing: 0.6 ms - Compiling: 1.3 ms - Optimizing: 48.13 ms - Evaluating: 2843.67 ms - Printing: 1.12 ms - Total Time: 2894.82 ms I don't see how I induce any exponential runtimes, and esp. check-sequence should be constant. Am I missing something fundamental here? The result size of the second version is about twice the size as the result size of the first version. Thanks in advance for any pointers. Bernhard Liebl

4 5

A new C++ client
by Ben Engbers 16 Jul '24

16 Jul '24

Before I started developing the R client for BaseX several years ago, I used to program regularly in SWI-Prolog. And I always wondered afterwards about the possible benefits of using Prolog in combination with XQuery. So after I finished developing RbaseX, I started developing ProBaseX. I did not manage to set up a connection in Swipl. Therefore, my initial plan to develop ProBasex entirely in SWI-prolog was changed to a plan to first set up a connection in C++ and then use that connection. This work eventually resulted in a linux shared library that implements the full server protocol. I will now use that library for further development work. LibBasexCpp and the test program libBasexTest.cpp are now on the Internet: https://github.com/BenEngbers/libBasexCpp I am curious to see if this library fills a need and if it makes sense to continue working on a dual platform version. Ben Engbers

2 2

feature request: control environment of external processes
by Hauke Brandes 16 Jul '24

16 Jul '24

Hello list, I would like to suggest an enhancement for the proc:system/execute/fork family of functions: it should be possible to control the environment variables of the external process. At the moment, the environment is always inherited from the calling basex process, as far as I can tell. There are several reasons why it is desirable to control the environment of external processes: * set variables to values that are not known statically when starting basex * avoid information leaking (restrict environment to the minimal required subset) * control the PATH of the external process It would be great to have an additional, optional entry in the $options map (maybe named "env" or "environment") that contains a map of environment variables for the external process. If absent, the current behavior (inherited environment) should be used. If the option is given, **only** the environment variables in the map should be used for the external process. Effectively, the "env"/"environment" option would behave as if it had a default value of `map:merge( available-environment-variables() ! map:entry(., environment-variable(.)) )` The ProcessBuilder in Java should allow for a straightforward implementation. What do you think? Cheers, Hauke

2 1

How best to handle long-running REST requests?
by Eliot Kimber 15 Jul '24

15 Jul '24

(Using 10.7 for now—upgrade to 11 is planned but not scheduled yet.) In my web app I have implemented client-side on-load JavaScript that implements deferred loading of the results of long-running queries. The JS finds elements that represent callbacks to the server and makes REST calls that are then dispatched to an appropriate worker server based on worker load. These queries can take minutes to run and return a result back to the browser, where the result is added to the HTML page, replacing the original element. In order for the server to maintain the HTTP connection while the query runs I had to turn off the server time out. This works but seems like a crude solution. My question: Is there a better way to manage this kind of client-to-server-to-client communication for long-running queries? For example, is there a way to have the response timeout be more dynamic? I didn’t see anything but I haven’t had time to push on it hard. The result I was getting in the browser with the default 30-second timeout was that when the response eventually returned, the browser reported that the connection had already been closed so it couldn’t do anything with the result. I’m using fetch() to get the result: fetch(url).then(function(response) { return response.text(); }).then(function(data) { const parser = new DOMParser(); const doc = parser.parseFromString(data, 'text/html'); … update web page … } Where “url” is the REST call back to my server. My infrastructure *should* be protecting against long-running queries blocking the main web-serving worker, but it still seems dangerous to have no time out (or a timeout set to 10 minutes or something). Thanks, Eliot _____________________________________________ Eliot Kimber Sr Staff Content Engineer Digital Content & Design O: 512 554 9368 M: 512 554 9368 servicenow.com<https://www.servicenow.com> LinkedIn<https://www.linkedin.com/company/servicenow> | Twitter<https://twitter.com/servicenow> | YouTube<https://www.youtube.com/user/servicenowinc> | Facebook<https://www.facebook.com/servicenow>

2 1

XSD 1.1 validation: base-uri(.) in the assert
by ykhabins＠bellsouth.net 14 Jul '24

14 Jul '24

Here is the latest development on the subject. I communicated with Mukul Gandhi. He is a chief maintainer of the Xerces-J XSD 1.1 project. Please see below his reply with the good news. >Hi Yitzhak, > >I'm well and hoping same for you. > >I could fix this bug, and the fix for this has been committed to Xerces-J's XML Schema 1.1 dev repos branch at, https://github.com/apache/xerces-j/tree/xml-schema-1.1-dev. > >I've also created a jira issue for this at, https://issues.apache.org/jira/browse/XERCESJ-1765. > >If you wish, you may build and use Xerces-J's XML Schema 1.1 implementation from repos url cited above. > >-- >Regards, >Mukul Gandhi Regards, Yitzhak Khabinsky

1 0

XSD 1.1 validation: base-uri(.) in the assert
by ykhabins＠bellsouth.net 11 Jul '24

11 Jul '24

Hello, We are using the final release BaseX 11.0 and Xerces 2.12.2 XSD 1.1 engine. There is a need to validate XML file and check XML file name for naming convention. Unfortunately, the assert with the base-uri(.) function is not working as expected in the assert expression. Observations: The XPath expression on the XML file emits, as expected, the following: o XPath: base-uri(.) o Results: file:/C:/Users/i300179/Downloads/ATF_NESS/base-uri/input.xml The XPath expression on the XML file emits, as expected, the following: o XPath: tokenize(base-uri(.),"/")[last()] o Results: input.xml I tried multiple expressions around base-uri(.) function. None of them worked. You will find all my attempts in the XSD file. The test case XQuery/XML/XSD is below. We are using Xerces, but the following Saxon excerpt could be pointing to the root cause of the problem. For the reference: https://www.saxonica.com/html/documentation10/functions/fn/base-uri.html "It is possible for the base URI of a document to be absent (unknown). This happens when the document is supplied to Saxon with no location information: for example when the node is represented using a tree model such as DOM that has no base URI information; or when a Source is supplied for which getSystemId() returns null. The most common example of this is a Source created using new StreamSource(new FileInputStream("input.xml")). The problem can be fixed by using the second argument of the StreamSource constructor, or by using the constructor that expects a File rather than an InputStream." XSD11_Validation.xq ======================= declare variable $xml as xs:string external; declare variable $xsd as xs:string external; let $result := validate:xsd-report($xml, $xsd, map { 'http://apache.org/xml/features/validation/cta-full-xpath-checking': true(), 'cache': false() }) return $result input.xml ======================= <root> <event> <id>1</id> </event> </root> input.xsd ======================== <?xml version="1.0" encoding="UTF-8"?> <xs:schema version="1.0.1" attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:vc="http://www.w3.org/2007/XMLSchema-versioning" xmlns:xerces="http://xerces.apache.org" vc:minVersion="1.1"> <xs:element name="root"> <xs:complexType> <xs:sequence> <xs:element maxOccurs="unbounded" ref="event"/> </xs:sequence>  <xs:assert test="tokenize(base-uri(.), '/')[last()] eq 'input.xml'" xerces:message="Rule #21: XML file name shall match file naming convention">   </xs:assert> </xs:complexType> </xs:element> <xs:element name="event"> <xs:complexType> <xs:sequence> <xs:element ref="id"/> </xs:sequence> </xs:complexType> </xs:element> <xs:element name="id" type="xs:integer"/> </xs:schema> Regards, Yitzhak Khabinsky

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk