- BaseX-Talk - mailman.uni-konstanz.de

automatically running an unknown number of queries on a variable content set
by Graydon Saunders 06 Jul '21

06 Jul '21

Hello! So the idea is to -- via some automatic process, so "the command line" -- load a content set into BaseX and then run some arbitrary list of queries on that content set, returning an aggregated report. It's important that it's easy to add a query; we expect to be adding queries as we find new ways for the content to be inappropriate. (We're guaranteed it's valid; we are not guaranteed it's sensible, in the "and just how do you expect to publish forty seven levels of nested lists?" sense.) My first thought is to have a directory of individual queries and have the main query walk through them with xquery:eval(), but there might be a better way. Anyone got any suggestions? Thanks! Graydon

3 2

Increase automatic logout time
by Josselin Morvan 02 Jul '21

02 Jul '21

Hi everyone, We are using BaseX for the development of our web application and we are having issues with automatic logouts. We have developed forms with Xforms that allow us to process historical documents and we use Basex to manage our data directly in XML. Recently we put our application online to be able to share our data and work with the whole team more easily. For the management of the users and the permissions, we relied on the BaseX doc (https://docs.basex.org/wiki/Permissions <https://docs.basex.org/wiki/Permissions>). Globally everything works fine, except that we encounter a kind of automatic logout of the users maybe about 1 hour after their login, maybe even less, it’s hard to say… In fact, our forms are quite big and it often takes more than an hour to finish processing a single historical resource, and when the user save his work, if he is logged out, he loses all what he’ve done. Would you know a way to increase this logout time? On our local BaseX instance, we tried changing the database options, including Keepalive (https://docs.basex.org/wiki/Options#KEEPALIVE). We changed the value 600 to 60, just to make sure it was this option that needed to be changed, but it didn't seem to have any effect… For the moment we set up an automatic backup system, but this solution introduces other issues and is therefore not ideal. Thank you in advance for your help!! Take good care, Josselin.

3 2

Copy/modify, Update, or file:append - help appreciated
by Bridger Dyson-Smith 02 Jul '21

02 Jul '21

Hi all - I'm trying to keep track of some HTTP POST results in a file, something akin to the following: file:write-text( "/tmp/bridger-test.csv", "book" || "," || "title" || "," || "sip" || out:nl() ), for $i in 1 to 20 (: some http:send-request stuff :) return( prof:sleep(250), if ($request[1]//@status/data() = 201) then file:append( "/tmp/bridger-test.csv", "book: " $i || "," || "title: " $i || "," || "" ) else "something went wrong with the request; the response was: " || $request[1]//@status/data() I have something that kinda works, but I'd like it to be better XQuery, avoid the two step create-file-then-append, and leverage more of built-in awesomeness in the language. I'm sure that I need to have a better understanding of the Update syntax, and/or %updating annotation, but would someone be willing to give some hints and/or suggestions for a better way to handle situations like this? Thanks in advance for your time and trouble. Best, Bridger

2 1

Command Files - string syntax
by Jonathan Robie 29 Jun '21

29 Jun '21

I decided to try creating some command files to simplify administration: https://docs.basex.org/wiki/Commands#String_Syntax I was surprised that this did not work: % cat foo.bsx CREATE DB test ADD TO embedded.xml <root>embedded</root> # run query XQUERY <hits>{ count(//text()) }</hits> CLOSE % basex foo.bsx Stopped at /.../foo.bsx, 1/9: [XPST0003] Unexpected end of query: 'DB test...'. This works fine: % basex <foo.bsx BaseX 9.1 [Standalone] Try 'help' to get more information. > CREATE DB test Database 'test' created in 196.04 ms. > ADD TO embedded.xml <root>embedded</root> Resource(s) added in 101.36 ms. > # run query > XQUERY <hits>{ count(//text()) }</hits> <hits>1</hits> Query executed in 393.09 ms. > CLOSE Database 'test' was closed. What am I missing? Jonathan

4 7

Sed-like CLI tool
by Iwan Briquemont 23 Jun '21

23 Jun '21

Hi, BaseX works very well to do some quick queries or updates over files (rather than using it as a DB). I usually use the GUI for that but I think it could be quicker to have a simple CLI tool for the terminal like sed to do things like: Have a fancy XPath tool: (-e is -q in basex) echo "<root><ele>1</ele><ele>2</ele></root>" | basexsed -e 'root/ele' 1 2 Work over multiple documents quickly (somewhat equivalent to "for $path in $args let $doc := doc($path) return xquery:eval($expr, map { '': $doc}))) basexsed -e 'some/xquery' xmldocuments/*.xml # Output the result Allow multiple documents on stdin too, so you can chain expressions: echo "<doc1/><doc2/><doc1/>' | basexsed -e 'doc1' | basexsed -e 'doc1/x' Do updating expressions, by default outputting the changes: basexsed -e 'delete node //n' xmldocuments/*.xml # Outputs the docs without the n nodes (-i is -u in basex) basexsed -i -e 'delete node //n' xmldocuments/*.xml # Changes the files in place Automatically handle other formats like json, csv, html: basexsed -e 'map:keys(.)' jsondocuments/*.json # Outputs as json as well AFAIK everything is already there in the basex command, so the main change would be to support multiple files/documents in a streaming fashion, and then a few smaller things like using stdin as the context- item if no file is given and allowing the same arguments as in standard tools like sed. To make it nicer you could also add syntax highlighting to the console, some shortcuts for longer XQuery parameters like serialization indentation. This would provide an alternative to tools like Xidel ( https://www.videlibri.de/xidel.html ) or jq ( https://stedolan.github.io/jq/ ). Is this something you would be interested in implementing? Or have some people already done something similar? Regards, Iwan

3 4

Re: [basex-talk] Package for system integration
by Christian Grün 23 Jun '21

23 Jun '21

Hi Karlo, > Where does basex set the location of the configuration? If you point me > to the file, I might try changing it. By default, BaseX creates and updates the configuration files in the project’s "Home Directory". This directory is determined at startup time as described in the Wiki [1]. If you download and start the JAR file on a completely fresh system, the home directory will be the same as the user home directory. Hope this helps, Christian [1] https://docs.basex.org/wiki/Configuration#Home_Directory On Tue, Jun 22, 2021 at 6:37 AM <karlo(a)fastmail.com> wrote: > > Liam R. E. Quin writes: > > The things that are binary-only, if they have a compatible licence, > > could either be included or added in a separate package. > > Indeed, when registering it with a package repository, we would replace > those binary libaries with dependent packages from the repository. > > Liam R. E. Quin writes: > > Yes, it should look in $HOME for a conf file. > > Even better, ${XDG_CONFIG_HOME:-${HOME}/.config} > > Where does basex set the location of the configuration? If you point me > to the file, I might try changing it.

1 0

Re: [basex-talk] Package for system integration
by Christian Grün 22 Jun '21

22 Jun '21

Dear Kralj, That’s a great contribution; I really appreciate your efforts. A little while ago, Bridger asked users on the list to share their experiences installing and running BaseX. It would be great if those of you who are using Linux and UNIX-based systems could have a look at Kralj’s package and tell us about your observations! Thanks all, best regards, Christian On Sun, Jun 20, 2021 at 8:36 AM <karlo(a)fastmail.com> wrote: > > Dear Christian, > > With your help I have prepared a new version with a similar user > interface but that also builds BaseX. The released binaries run > perfectly of course, so the role of the source build package is > to demonstrate that the binaries are built from a free source. > > Because the package follows certain conventions, we can quickly prepare > BaseX for many different operating systems and maintain BaseX packages > for several package repositories. > > I explain the method in more detail below. For those who are just > joining the discussion, I include some background before continuing > the discussion about this package. > > I have the honor to be Your Majesty's humble and obedient servant. > > Kralj Karlo > > > > INTRODUCTION > > BaseX has a bespoke procedure initiating builds. If the build could > instead be initiated through execution of "make" in a directory > with the conventional inputs, it would be fast to port to the many > different operating systems whose packaging systems follow this > convention. > > I propose a procedure to create BaseX source packages. The resulting > BaseX source packages contain a Makefile with instructions for building > and installing from source. > > I create the source package as a directory containing the basex source code, > the basex-dist source code, a version-specific Makefile, and other files > to support the build. > > I distribute the source package, and someone else can build it. A build > of the source package produces a tree that can be installed by unpacking > to the specified PREFIX, /usr/local by default. > > To build it within the framework of a package manager, you would > configure the package manager to retrieve the source package and, > possibly, set a different PREFIX, set the dependencies, and copy > other package metadata (for example, homepage and description). > The package manager will likely already have tooling around the "make" > and "make install" commands, so you should not really need to understand > the build process. > > > > PROCEDURE > > Copy the program for creating the source package. > https://klarinet.kos.ovh/fossil/basex-make/tarball/basex-make.tar.gz > > You can alternatively use fossil, for example: > > fossil clone https://klarinet.kos.ovh/fossil/basex-make ../basex-make.fossil > fossil open ../basex-make.fossil > > I run "make update prune all tgz" from the basex-make directory > to do the following. > > * Download the latest sources and copy them to ./basex-src > * Generate the file ./basex-src/Makefile for building the sources. > * Archive the directory basex-src as basex-src.tar.gz > > Now I can send the basex-src.tar.gz to someone else. Here it is. > https://klarinet.kos.ovh/fossil/basex-make/uv/basex-src.tar.gz > > Then you can extract basex-src.tar.gz and run "make && make install" > to build and install. That does the following. > > * Build BaseX by the conventional process. > * Unpack the zip file created by the conventional BaseX build. > * Replace the conventional executibles. The new executables > handle the PREFIX and are written in sh rather than bash. > > > > BUGS > > * With this installation method BaseX looks for the configuration file > $PREFIX/lib/BaseX$VERSION/.basex and creates it if it does not exist. > It would be good to change that to look for $HOME/.basex instead. > * Certain vendor libraries are still distributed as binaries > in basex/basex-core/lib. I plan to remove these so we can truly call > it a "source" package. > * basex-src.tar.gz should instead be named BaseX$VERSION.tar.gz, and > it should contain a directory named BaseX$VERSION instead of basex-src. > > NOT BUGS > > * The minor complexity of this process is unfortunate. It would be nice > to modify release.pl so there are fewer layers, but this could break other > things that depend on the present build process. > > APPLICATION > > I propose that a tarball like that created by the present procedure be > distributed on the BaseX download webpage (https://basex.org/download/) > and that it be addressed as the "Source Package" with support for all > Unix-like operating systems. I believe it will in fact work on Windows > with Cygwin, but I don't care to support that. > > Once we have begin releasing source packages, we proceed to add BaseX > to various package repositories. I think we would start with Debian apt, > because we have already done one of those, and OpenBSD ports, because > that's what I use.

3 2

ft:normalize: potential bug?
by Alexander Witzigmann 21 Jun '21

21 Jun '21

hi, if you call ft:normalize using a string containing "*" ft:normalize("a*",map { 'stemming': 'true', 'language':'de' }) this results in Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.5.2 Java: Oracle Corporation, 13.0.2 OS: Windows 10, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 8 at org.basex.util.TokenBuilder.get(TokenBuilder.java:191) at org.basex.util.ft.GermanStemmer.resub(GermanStemmer.java:150) at org.basex.util.ft.GermanStemmer.stem(GermanStemmer.java:42) at org.basex.util.ft.Stemmer.nextToken(Stemmer.java:99) at org.basex.util.ft.FTLexer.nextToken(FTLexer.java:156) at org.basex.query.func.ft.FtTokenize.tokens(FtTokenize.java:49) at org.basex.query.func.ft.FtNormalize.item(FtNormalize.java:23) .... anything I missed? Alex

2 2

Error message, can this be fixed?
by Roland Haarbrink 21 Jun '21

21 Jun '21

Hi, Getting this message: org.basex.core.BaseXException: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.8.0_161 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException When using this validator: https://validatie.geostandaarden.nl/etf-webapp/testruns/start-direct Can this be fixed? Thanks URL: http://localhost:8080/etf-webapp/testruns/start-direct Exception id: EXID-06e97066-fe8f-43cd-adc5-df641b843f75 de.interactive_instruments.etf.dal.basex.dao.BasexDataSource.create(BasexDat aSource.java:397) de.interactive_instruments.etf.dal.basex.dao.BasexDataSource.create(BasexDat aSource.java:381) de.interactive_instruments.etf.dal.basex.dao.BasexTestObjectDao.doCreate(Bas exTestObjectDao.java:134) de.interactive_instruments.etf.dal.dao.AbstractTestObjectDao.create(Abstract TestObjectDao.java:89) de.interactive_instruments.etf.webapp.controller.TestObjectController.addFil eTestData(TestObjectController.java:366) de.interactive_instruments.etf.webapp.controller.TestRunController.startDire ct(TestRunController.java:238) sun.reflect.GeneratedMethodAccessor245.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl .java:43) java.lang.reflect.Method.invoke(Method.java:498) org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(Invoc ableHandlerMethod.java:221) org.springframework.web.method.support.InvocableHandlerMethod.invokeForReque st(InvocableHandlerMethod.java:136) org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandle rMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerA dapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:817) org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerA dapter.handleInternal(RequestMappingHandlerAdapter.java:731) org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.hand le(AbstractHandlerMethodAdapter.java:85) org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServl et.java:959) org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServle t.java:893) org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSer vlet.java:968) org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.jav a:870) javax.servlet.http.HttpServlet.service(HttpServlet.java:648) org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.ja va:844) javax.servlet.http.HttpServlet.service(HttpServlet.java:729) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:292) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:207) org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application FilterChain.java:240) org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh ain.java:207) org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja va:212) org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja va:106) org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase .java:502) org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141 ) org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogVa lve.java:616) org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java :88) org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Proce ssor.java:1095) org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Abstrac tProtocol.java:672) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.jav a:1500) org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java: 1456) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 49) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 24) org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.ja va:61) java.lang.Thread.run(Thread.java:748)

2 1

Package for system integration
by karlo＠fastmail.com 20 Jun '21

20 Jun '21

I have packaged basex in a way that I believe will be easier for integration with package managers. https://klarinet.kos.ovh/fossil/basex-root-install/uv/basex-9.4.3.tar.gz >From the root of that archive, you can build and install like this. make make install # as root The above will install to the prefix /usr/local. You can set a different prefix. Here is an example. make clean make PREFIX=/usr make PREFIX=/usr install # as root A secondary benefit of this package is the removal of the bash dependency. Could a package like this be distributed officially? If that is agreeable, I am happy to try incorporating this into the present build system. Here is a utility for preparing the package. It depends on sh and other Unix-like utilities. https://klarinet.kos.ovh/fossil/basex-root-install/dir?ci=tip With distinguished salutations, Kralj Karlo

3 6

getting request's headers inside a Web Application
by dudzikp＠yahoo.com 16 Jun '21

16 Jun '21

Dear all, I do not see an option to obtain the headers from the HTTP request from inside webapp function. Just like in the example: declare %rest:GET %rest:path('tests-html') ???%rest:headers('User-Agent')??? %output:method('html') function page-main:start-empty( ???$user-agent as xs:string??? ) as element(html) { ... } Is this possible? BR

3 3

http:sent-request with username, password and auth-method results in improper use msg
by Rob Stapper 15 Jun '21

15 Jun '21

Hi Cristian, When I add the username, password,auth-method to a http-request [1] I get an improper use message from BaseX[2]. The message suggest a bug. I hope this helps solving it. Kind regards, Rob [1] http:send-request( <http:request method='post' username='admin' password='admin' auth-method='Basic' > <http:body media-type='text/plain'/> </http:request> , 'http://LAPTOP-F17D4BIM:8984/test' , "test" ) [2] Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.5.3 beta Java: Oracle Corporation, 1.8.0_271 OS: Windows 10, amd64 Stack Trace: java.lang.IllegalStateException: Already connected at sun.net.www.protocol.http.HttpURLConnection.setRequestProperty(Unknown Source) at org.basex.util.http.HttpClient.setContentType(HttpClient.java:258) at org.basex.util.http.HttpClient.sendRequest(HttpClient.java:82) at org.basex.query.func.http.HttpSendRequest.value(HttpSendRequest.java:34) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) at org.basex.query.scope.MainModule.iter(MainModule.java:97) at org.basex.query.QueryContext.iter(QueryContext.java:326) at org.basex.query.QueryContext.cache(QueryContext.java:607) at org.basex.query.QueryProcessor.cache(QueryProcessor.java:113) at org.basex.core.cmd.AQuery.query(AQuery.java:99) at org.basex.core.cmd.XQuery.run(XQuery.java:22) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.gui.GUI.exec(GUI.java:416) at org.basex.gui.GUI.lambda$4(GUI.java:359) at java.lang.Thread.run(Unknown Source) Sent from Mail for Windows 10 -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

2 2

differencing the string value of documents
by Graydon Saunders 12 Jun '21

12 Jun '21

Hello -- So I have XML documents A and B. For this purpose, the markup (which is known to be different) is not relevant; the thing I need to do is prove that all of the text in the string value of A is present in B in the same order in which it exists in A. Document B can have additional text and still be correct, so it is not sufficient to check if string(A) = string(B). (Ideally, B's extra text can be extracted and then compared against the sources of extra text in the transformation between A and B so as to confirm that it's at least all plausible new text.) I've been poking at this for a while and am getting that "reinventing the wheel by gnawing on anvils" feeling. Does a facility to do this exist in BaseX? I've had a look at the documentation and I observe some mention of diff.xqm but that appears to be a node-based diff. thanks! Graydon

5 9

Permissions
by SW-Service 12 Jun '21

12 Jun '21

Hello, should perm force authentication and only the user "admin" would be authorised? Best regards Frank module namespace page = 'http://basex.org/test'; declare %rest:path("basex-rest/test/check-rest-perm") %perm:allow("admin") %rest:GET %output:method("xml") function page:test-check-rest-perm() as element(response) { let $response := element response { 'Response' } return $response }; <response>Response</response>

2 1

HTTP Client DELETE with body
by Marco Lettere 12 Jun '21

12 Jun '21

Dear all, I have to call an API which model an HTTP DELETE operation based on the body of the request. It looks like currently http:send-request is not allowing body to be present ("Body not expected for method DELETE."). I feel that this is bit too restrictive interpretation of the spec which states (as far as I recall) that there is no defined semantic for the operation but not that it is forbidden. Is there any possibility to relax this behaviour and allow for bodies in DELETE calls? [1] Thank you very much. Marco. [1] https://github.com/BaseXdb/basex/blob/382c8097afb1ec5bac1e9511a2287686c33f6…

3 6

Re: [basex-talk] Error with full-text and fuzzy query
by Christian Grün 07 Jun '21

07 Jun '21

cc to the list (2)… Christian Grün <christian.gruen(a)gmail.com> schrieb am Mo., 7. Juni 2021, 15:47: > …and another snapshot for you: modifier letters [1] will now be > regarded as diacritical characters, and queries such as… > > tokenize('faraʼid faraid Birkan')[. contains text 'faraʼid' using fuzzy] > > …will only regard the first two terms as similar. As a consequence, > fuzzy queries for terms with modifiers should get a lot faster. > > You’ll need to recreate your full-text index to take full advantage of the > fix. > > [1] https://de.wikipedia.org/wiki/Unicodeblock_Spacing_Modifier_Letters > > > > On Mon, Jun 7, 2021 at 2:00 PM Christian Grün <christian.gruen(a)gmail.com> > wrote: > > > > > Out of curiosity, is there a way to access those individual term > similarity statistics via XQuery? > > > > I assume there is no straightforward way, but you can check the first > > results of a full-text query: > > > > ft:search('your-db', 'faraʼid', map { "fuzzy": true() })[position() < > 10] > > > > And I noticed that due to the modifier letter (ʼ), many non-similar > > results are returned as well. I’ve raised another issue to track this > > down [1]. > > > > [1] https://github.com/BaseXdb/basex/issues/2015 >

1 0

Re: [basex-talk] Error with full-text and fuzzy query
by Christian Grün 07 Jun '21

07 Jun '21

cc to the list… Christian Grün <christian.gruen(a)gmail.com> schrieb am Mo., 7. Juni 2021, 13:50: > Thanks, Tim, that was helpful. It turned out that the index contains > an unusually high number of similar tokens for one search terms in > your query: > > faraʼid : 13709 similar terms > muhammad : 131 similar terms > siraj : 50 similar terms > ... > > A new snapshot with a bug fix is available, in which similar terms are > collected non-recursively [1]. I’d have wished that your search was > evaluated faster (it takes around 6 seconds on machine) but at least > it succeeds. I’ll have some more thoughts. > > Cheers, > Christian > > [1] https://files.basex.org/releases/latest/ > > > > On Mon, Jun 7, 2021 at 1:17 PM Tim Thompson <timathom(a)gmail.com> wrote: > > > > Thank you, Christian. If it's still helpful, here is a link to download > the document I am using as my full-text index. > > > > All best, > > Tim > > > > > > -- > > Tim A. Thompson > > Metadata Librarian > > Yale University Library > > > > On Mon, Jun 7, 2021 at 6:19 AM Christian Grün <christian.gruen(a)gmail.com> > wrote: > >> > >> I think I managed to build up a bug scenario that should resemble > >> yours [1]: If thousands of similar terms are found, the results cannot > >> be recursively joined anymore. > >> > >> [1] https://github.com/BaseXdb/basex/issues/2014 > >> > >> > >> > >> On Mon, Jun 7, 2021 at 10:28 AM Christian Grün > >> <christian.gruen(a)gmail.com> wrote: > >> > > >> > Hi Tim, > >> > > >> > I’m still trying to get this reproduced. > >> > > >> > The error is caused by excessive recursive functions call. Out of > >> > interest, could you try to increase the size of the Java Stack Trace > >> > and see if the error persists? This can be achieved by assigning a > >> > large value to the JVM via the Xss flag [1]. You could e.g. add > >> > -Xss64m to the BASEX_JVM variable in the BaseX start scripts [2]. > >> > > >> > Internal notes… If yes, we might be confronted with an infinite loop. > >> > If no, we should try to rewrite a recursive index lookup to an > >> > iterative one. > >> > > >> > If your data is not confidential, feel free to provide me with a > download link. > >> > > >> > Cheers, > >> > Christian > >> > > >> > [1] > https://stackoverflow.com/questions/3700459/how-to-increase-the-java-stack-… > >> > [2] https://docs.basex.org/wiki/Start_Scripts > >> > > >> > > >> > > >> > On Fri, Jun 4, 2021 at 11:03 PM Tim Thompson <timathom(a)gmail.com> > wrote: > >> > > > >> > > Thanks, Christian. The rest of the stack trace was just many lines > of "at > org.basex.index.query.FTIndexIterator$2.pos(FTIndexIterator.java:74)" until > the end: > >> > > > >> > > [qtp1546693040-47] WARN > org.eclipse.jetty.server.handler.ErrorHandler - Error page too large: 500 > java.lang.StackOverflowError Request(POST // > 10.5.157.229:10214/dba/query-eval)@7dcadc62 > >> > > [qtp1546693040-47] INFO > org.eclipse.jetty.server.handler.ErrorHandler - Disabling showsStacks for > ErrorPageErrorHandler@53032c30{STARTED} > >> > > > >> > > Best regards, > >> > > Tim > >> > > > >> > > > >> > > -- > >> > > Tim A. Thompson > >> > > Metadata Librarian > >> > > Yale University Library > >> > > > >> > > On Fri, Jun 4, 2021 at 1:30 PM Christian Grün < > christian.gruen(a)gmail.com> wrote: > >> > >> > >> > >> Hi Tim, > >> > >> > >> > >> > [qtp1546693040-47] WARN org.eclipse.jetty.server.HttpChannel - > /dba/query-eval > >> > >> > java.lang.StackOverflowError > >> > >> > at > org.basex.index.query.FTIndexIterator$2.pos(FTIndexIterator.java:73) > >> > >> > at > org.basex.index.query.FTIndexIterator$2.pos(FTIndexIterator.java:74) > >> > >> > >> > >> Could you share some more lines of the stack trace with us? > >> > >> > >> > >> Thanks in advance > >> > >> Christian >

1 0

Error with full-text and fuzzy query
by Tim Thompson 07 Jun '21

07 Jun '21

Hello, I have a full-text index defined on a database with 21,936,670 entries. I'm hitting a stack overflow error when I try to query the following (normalized) string with a fuzzy query: "sajawandi, siraj al-din muhammad ibn muhammad, active 12th century. faraʼid al-sirajiyah" The index entries look like this: <d a="http://id.loc.gov/authorities/names/n2008037847"> <n>Sajāwandī, Sirāj al-Dīn Muḥammad ibn Muḥammad, active 12th century. Farāʼiḍ al-Sirājīyah</n> <d>12th cent.</d> </d> If I create a new DB ("ftindex-test") with only that one entry and try the following lookup, it works: ft:search( "ftindex-test", "sajawandi, siraj al-din muhammad ibn muhammad, active 12th century. faraʼid al-sirajiyah" , map {"mode": "phrase", "content": "entire", "fuzzy": true()} )/.. However, if I try the same query against the full index (with ~22 million entries), it fails with a stack overflow error: [qtp1546693040-47] WARN org.eclipse.jetty.server.HttpChannel - /dba/query-eval java.lang.StackOverflowError at org.basex.index.query.FTIndexIterator$2.pos(FTIndexIterator.java:73) at org.basex.index.query.FTIndexIterator$2.pos(FTIndexIterator.java:74) ... If I remove the "fuzzy" parameter, the query does work against the full index. Is this a bug, a known limitation here, or something that I'm missing? Thanks in advance, Tim -- Tim A. Thompson Metadata Librarian Yale University Library

2 4

speaking effectively to the optimizer
by Graydon Saunders 04 Jun '21

04 Jun '21

hello -- So I want to subtract one list of sentences from another list of sentences. (As a way of checking "what did processing do to this document?"; we know it added some words, and if we added words to the sentence it was at the start of the sentence, but none of the original words should be changed or missing or reordered relative to the document.) I have tested that I have the sequences of strings which are the list of sentences before I use them to call: declare function local:subtractSentences($before as xs:string*,$after as xs:string*) as xs:string* { let $thisBefore as xs:string? := head($before) let $thisAfter as xs:string? := head($after) return if (not(exists($thisBefore)) or not(exists($thisAfter))) then $after (: we're done because we're out of strings, but if we have any after left we want it :) else if (ends-with($thisAfter,$thisBefore)) then (replace($thisAfter,$thisBefore,''),local:subtractSentences(tail($before),tail($after))) else ($thisAfter,local:subtractSentences($before,tail($after))) }; In ~150 ms, I get Error: Stack Overflow: Try tail recursion? so I have written the function wrong. What's the more useful pattern for this case? Thanks! Graydon

4 3

thesaurus URILiteral vs node
by Liam R. E. Quin 01 Jun '21

01 Jun '21

What would be involved in extending BaseX's full text to allow a thesaurus to be supplied as a node instead of a URI? I want to construct one on the fly, and using file:write() seems excessive. Liam -- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org

2 6

insert MS SQL server bigint from BaseX via sql:execute-prepared
by Martin Honnen 31 May '21

31 May '21

I am trying to use BaseX to enter data from its XML db into an MS SQL server database, according to https://docs.microsoft.com/en-us/sql/connect/jdbc/using-basic-data-types?vi… the MS SQL server column type bigint is mapped to long in Java. However, when I try <sql:parameter type="long">{$id}</sql:parameter> with BaseX and sql:execute-prepared, I get an error saying [sql:error] An SQL exception occurred: unsupported type: long The BaseX documentation https://docs.basex.org/wiki/SQL_Module#sql:execute-prepared indeed only lists attribute type { "int" | "string" | "boolean" | "date" | "double" | "float" | "short" | "time" | "timestamp" | "sqlxml" } as the possible types. Am I out of luck trying to use BaseX that way for that datatype? Using <sql:parameter type="int">{$id}</sql:parameter> the sql doesn't succeed.

2 2

BaseX with Apache
by Tim Thompson 27 May '21

27 May '21

With apologies for the not-strictly-BaseX question--but I'm running BaseX through a reverse proxy with Apache (and Let's Encrypt), and I can't figure out how to disable directory listing for the BaseX webapp/static directory. I've tried editing the apache2 conf files in the standard ways. Hoping someone here has a quick solution to share. Thanks in advance, Tim -- Tim A. Thompson Metadata Librarian Yale University Library

3 3

BaseX 9.5.3
by Christian Grün 27 May '21

27 May '21

Dear all, A new BaseX patch release is available. It comes with minor bug fixes and little enhancements: • RESTXQ responses now come with a Server-Timing HTTP header (motivated by Reece) • A context file can be specified via the Bindings Dialog of the XQuery editor (motivated by Hans-Jürgen) • The current HTTP context will now be propagated if jobs:eval is called • The DBA CodeMirror and Jetty libraries have been updated Find the new version on our web site: https://basex.org/ Have fun, Christian

1 0

Running XQuery Update via apply Ant task
by Zimmel, Daniel 26 May '21

26 May '21

Hi, is there anyone using BaseX in an Apache Ant scenario? I have the following target for XQuery Updating some files (no database access), but I am too dumb to make an apply task that reads every file in a fileset. Usually you place the <srcfile/> where you need it to be in your external command, but unfortunately the BaseX command-line syntax won't allow this to be easily placed: the filename is supposed to follow directly suffixed to "-i" where I cannot use an element, of course. <target name="xquery_update_default"> <echo level="info" message="XQuery Update (Default): ${param.xquery-update-uri}"></echo> <apply executable="java" > <arg value="-cp"/> <arg value="BaseX95.jar"/> <arg value="org.basex.BaseX"/> <arg line="-u"/> <arg line="-w"/> <arg line="-c'set exporter omit-xml-declaration=no,indent=no'"/> <arg line="-b'path=${param.folder}'"/> <arg line="-i${dummy.xml}"/> <srcfile/> <fileset dir="../${dir.xquery}" includes="${param.xquery-update-uri}"/> </apply> </target> As a workaround, I set a dummy.xml for input and bind the current folder as a parameter and in my Query I can use file:list() which then is updating all files in the directory, but this makes the Query dependent on the parameter, and updates are obviously not written back unless all files have been processed in memory. I would prefer to have a standalone query operating only on the current document-uri (so I can better reuse it elsewhere). Can someone give me a hint if it is possible in plain Ant to apply a CCL-BaseX to any file in a given fileset? Other than that, BaseX performs well as a file processor (if you ignore the time related to file handling). Thanks, Daniel

1 1

BUG node comparison
by ydyxeb＠post.cz 26 May '21

26 May '21

Dear BaseX, according to github instructions I’m sending a bug report via email: Using BaseX 9.5.1, cmd-line “basex.bat -i datafile.xml queryfile.xq” leads to no result at all. I’m convinced there should be result equivalent to the one of the online xquery tester (https://www.videlibri.de/cgi-bin/xidelcgi) For a modified query using string comparison based on fn:path(), BaseX gives the expected result. %%% query begin %%% let $rootElement := /child::* for $x in //Section[parent::* != $rootElement] return fn:path($x) %%% query end %%% %%% data begin %%% <root> <Section id="should not be reported 1" /> <Section id="should not be reported 2"> <Section id="should be reported 1"/> <a> <Section id="should be reported 2"/> </a> </Section> <a> <Section id="should be reported 3"/> </a> </root> %%% data end %%% %%% modified query begin %%% let $rootElementPath := fn:path(/child::*) for $x in //Section[fn:path(parent::*) != $rootElementPath] return fn:path($x) %%% query end %%% Kind regards Jan Červák

3 5

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk