BaseX-Talk July 2019

basex-talk@mailman.uni-konstanz.de

22 participants
30 discussions

Options for ft:search
by Sebastian Zimmer 15 Mar '21

15 Mar '21

Hi, I'd like to request a feature concerning the ft:search method: An `lserror` option that works exactly like the global option `LSERROR`. It would be nice if we could set the maximum Levenshtein distance specifically for each fuzzy search without having to adjust the global option. Another question on ft:search: Is there a reason why it doesn't have a `case` option just like ft:contains has one? Best regards, Sebastian

2 3

Feature Request: Save untitled, open scripts when exiting GUI
by Andreas Mixich 30 Jul '20

30 Jul '20

SublimeText has this nifty feature, that it saves all open tabs, even if the have no representation on the filesystem, so one can continue exactly, where one has left off. I find myself often to have several, unnamed, XQueries open, to test this concept, to compose that function, etc. It would be nice, if one could simply shut off and have them reopened next time without naming a file and saving each item in question. -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 6

9.2.1 not found in maven central
by andreas.hengsbach＠nexoma.de 07 May '20

07 May '20

Hi, is there a reason, that I can't find the current Version (9.2.1) in maven central? Thanks and best regards, Andreas

3 3

Database corrupted on updating with RestXQ
by Marco Lettere 26 Sep '19

26 Sep '19

Hi all, starting with 9.2.1 we experience a strange error with a RESTXQ API of ours that we have been using for years. The typical pattern is lookup a document update it and store it back. We have done this milions of time and also all the tests work neatly. But when using it from inside the web application, at around the tenth operation we get the DB corrupted with the stack trace [1]. This seems to happen only when the database is nearly empty and, for sure, it doesn't appear on 8.x series. It feels like some concurrency flaw is introduced somewhere but it's hard to say because the API is eight years in age and it never changed significantly. I know it's hard without an SSCE... but while working to insulate this byzantine behaviour (I've been trying for days now) maybe someone in the list has a clue or is able to interpret the stack trace? Thanks and have a nice weekend, Marco. [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.3 beta Java: Ubuntu, 13 OS: Linux, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1 at org.basex.io.random.TableDiskAccess.fpre(TableDiskAccess.java:507) at org.basex.io.random.TableDiskAccess.cursor(TableDiskAccess.java:468) at org.basex.io.random.TableDiskAccess.read1(TableDiskAccess.java:156) at org.basex.data.Data.kind(Data.java:304) at org.basex.query.value.node.DBNode.<init>(DBNode.java:50) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:139) at org.basex.query.value.seq.DBNodeSeq.get(DBNodeSeq.java:164) at org.basex.query.func.db.DbOpen.value(DbOpen.java:19) at org.basex.query.func.StandardFunc.optimize(StandardFunc.java:82) at org.basex.query.expr.Arr.inline(Arr.java:69) at org.basex.query.expr.path.Path.inline(Path.java:962) at org.basex.query.expr.gflwor.ForLet.inline(ForLet.java:66) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:804) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:785) at org.basex.query.expr.Try.inline(Try.java:106) at org.basex.query.expr.gflwor.GFLWOR.inline(GFLWOR.java:816) at org.basex.query.expr.gflwor.GFLWOR.inlineLets(GFLWOR.java:339) at org.basex.query.expr.gflwor.GFLWOR.optimize(GFLWOR.java:106) at org.basex.query.func.StaticFunc.inline(StaticFunc.java:294) at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:67) at org.basex.query.scope.MainModule.comp(MainModule.java:81) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) at org.basex.query.QueryContext.compile(QueryContext.java:312) at org.basex.query.QueryContext.iter(QueryContext.java:331) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:63) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:53) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) at java.base/java.lang.Thread.run(Thread.java:835)

2 11

ß = ss: not equal in all cases
by Alexander Witzigmann 31 Jul '19

31 Jul '19

Hi All, the following query declare default collation 'http://basex.org/collation?lang=de;strength=secondary'; "anschließend" contains text "anschliessend" returns *false* but the following query declare default collation 'http://basex.org/collation?lang=de;strength=secondary'; "anschließend" = "anschliessend" returns *true*. and the following query declare default collation 'http://basex.org/collation?lang=de;strength=secondary'; "anschließend" eq "anschliessend" returns *true*. I wonder why 'contains text' does not return true as well? All full-text related functions doesn't support ß = ss for german collation. Alex

2 1

Full-Text Search long s
by Günter Dunz-Wolff 26 Jul '19

26 Jul '19

Hi all, I’m working since some years on a digital edition of the works of a former german author. In my transcription of those works are lots of gothic characters like the old german long s (Unicode: LATIN SMALL LETTER LONG S). For example: Büchſe (exactly Buͤchſe). In my Full-Text-Search my goal is, that the user asks for „Büchse“ and gets „Büchse“ AND „Büchſe“ (with long s). In best case, she should get „Büchse“ AND „Büchſe“ AND „Buͤchſe“. How can I achieve, that //text[. contains text { } treats s and ſ and ü and uͤ as the same character? Thanks a lot for any help. Best regards, Guenter

3 2

Managing external entities
by Majewski, Steven Dennis (sdm7g) 26 Jul '19

26 Jul '19

I have a corpus of TEI files with figures and page images encoded as external entities. It appears that even when choosing “Parse DTDs and entities” this info is lost when parsing files into database, And in any case, unparsed-entity-uri() is an XSLT only function. It would appear that I first need to transform the files first and replace @entity attributes with @url attributes while these unparsed entity values are available, before creating the database, or else generate another database to map entity names to values later. Are there any better ways to handle this case ? Is there any way to do these transforms on the fly before parsing the files into the database ? The only thing that comes to mind is to set up a local SaxonServlet to do the transforms, and load from URLs instead of file paths. ( I’ve been doing something similar for a different case, and running into memory errors that I don’t see when loading from a directory when creating a database. Increasing memory didn’t help much, but inserting ‘flush’ command between even ‘add’ commands seemed to work. ) — Steve Majewski

2 2

BaseX 9.2.1: Maintenance Release
by Christian Grün 26 Jul '19

26 Jul '19

Dear all, A new BaseX maintenance release is available: - GUI: Parsing of project files has become faster and smarter. - RESTXQ: Custom reponses are now sent as streams. - Performance tweaks (static typing: typeswitch, instance of). - Minor bugs have been fixed. Looking forward to your feedback, Christian BaseX Team

1 3

File-io:error. File lock by another process (windows-only)
by France Baril 24 Jul '19

24 Jul '19

Hi, We have this issue only on Windows system since 9.2 that we know of... could be from 902. It is still happening after upgrade to 923. I am not able to provide a sample to reproduce systematically as it never happens at the same .xqm location. It does however always seem that the 1st error message occurs on a line with a file:write or file;append. I am giving you everything I know, in case that it rings a bell. 1. Start server with basexhttp 2. Run a process that has many functions and many file:write and file:append. 3. This generally works properly. The process ends with success. 4. Run the same process again, there is a good chance that we'll get an error that says something like: [file:io-error] Resource "debug-list-imtes.xml' locked by another process. 5. Run the same process again May get file-oi:error at the same location or a different one. 6. Restarting the server brings us back to 1... but sometimes we need to manually delete the files written with function from the file module before we can go back to 1 and get 1 good run of the process. Note: Although the process runs correctly and seems to finish running (to the same point that releases all locks on Linux). The person on the Windows system also often gets: [db:open] Resource "C:\basex\data\staging-en-us\txt.basex locked by another process and then the related database also remains locked and we may get messages: Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.2.3 Java: Oracle Corporation, 1.8.0_211 OS: Windows 10, amd64 Stack Trace: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source) at sun.nio.ch.SharedFileLockTable.add(Unknown Source) at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source) at org.basex.io.random.TableDiskAccess.lock(TableDiskAccess.java:147) at org.basex.io.random.TableDiskAccess.(TableDiskAccess.java:79) at org.basex.data.DiskData.init(DiskData.java:125) at org.basex.data.DiskData.(DiskData.java:84) at org.basex.core.cmd.Open.open(Open.java:99) at org.basex.query.QueryResources.database(QueryResources.java:179) at org.basex.query.func.StandardFunc.checkData(StandardFunc.java:429) at org.basex.query.func.db.DbOpen.value(DbOpen.java:17) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:45) at org.basex.query.expr.path.IterPath$1.init(IterPath.java:79) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:46) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:1) at org.basex.query.iter.Iter.value(Iter.java:65) at org.basex.query.expr.path.IterPath.nodes(IterPath.java:87) at org.basex.query.expr.path.AxisPath.value(AxisPath.java:39) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:173) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:80) at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:172) at org.basex.query.func.FuncCall.invoke(FuncCall.java:117) at org.basex.query.func.FuncCall.value(FuncCall.java:56) at org.basex.query.scope.MainModule.value(MainModule.java:135) at org.basex.query.QueryContext.update(QueryContext.java:673) at org.basex.query.QueryContext.iter(QueryContext.java:332) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:56) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:54) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) at java.lang.Thread.run(Unknown Source)undefined -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 10

Re: [basex-talk] Transaction Management
by Christian Grün 24 Jul '19

24 Jul '19

Hi Radim (cc to the list), > it takes so long even when it was being stored from a file on local hard drive. The xml file has about 15 to 20 MB. Wow. How large is your database? How did you add the documents? > Could you give me the XQuery or the BaseX command to move the file from the TempDB to the TargetDB the quickest possible way? Sure. There are various ways to do so, here’s just one (based on the database module [1]): 1. upload your document to database 'db-temp' 2. insert documents into final database: db:replace('db', db:list('db-temp'), db:open('db-temp')) 3. remove all your documents from the temporary database: db:delete('db-temp', '/') You can also drop the temporary database completely via db:drop. Hope this helps, Christian [1] http://docs.basex.org/wiki/Database_Module And how about database cache settings? Is something like that in BaseX? > > Thank you in advance for your response. > Best regards, > > Radim Havlicek > > > ---------- Původní e-mail ---------- > Od: Christian Grün <christian.gruen(a)gmail.com> > Komu: Radim Havlicek <radim-havlicek(a)post.cz> > Datum: 23. 7. 2019 15:12:04 > Předmět: Re: [basex-talk] Transaction Management > > Hi Radim, > > Do you have a clue how the long update time can be explained? Does it > take 20 minutes to upload your document to the BaseX server, or does > it take the same time if you update your database directly on the > server? If it’s the upload time, you could e.g. insert your document > in another temporary database and copy it to the final database in a > second step. > > Hope this helps, > Christian > > > > On Tue, Jul 23, 2019 at 10:34 AM <radim-havlicek(a)post.cz> wrote: > > > > Hello. > > > > We use client/server architecture, especially the C# client. I read that: > > > > Read transactions are executed in parallel. > > If an updating transaction comes in, it will be queued and executed after all previous read transaction have been executed. > > > > We have a web Front-End built on BaseX. We read XML documents stored in the database, and show charts with statistics calculated from those documents. Few times per day, we copy a new document to the database (via REPLACE command to make sure, the document is not stored in the database already). > > > > The problem we are facing is that while big document is being stored to the database (10 to 20 minutes), we cannot read documents from the database and display charts for users on the web interface. The web interface is waiting until the document is stored, and then read the the needed documents, and display the charts. It is very inconvenient for users to wait so long for a chart that is normally displayed in few seconds. > > > > Is there a way we could read the xml documents from the database while another document is being stored to the database? It is very, very needed. > > > > Best regards, > > > > Radim Havlicek

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk July 2019