BaseX-Talk

basex-talk@mailman.uni-konstanz.de

1 participants
5252 discussions

Full-Text Search long s
by Günter Dunz-Wolff 26 Jul '19

26 Jul '19

Hi all, I’m working since some years on a digital edition of the works of a former german author. In my transcription of those works are lots of gothic characters like the old german long s (Unicode: LATIN SMALL LETTER LONG S). For example: Büchſe (exactly Buͤchſe). In my Full-Text-Search my goal is, that the user asks for „Büchse“ and gets „Büchse“ AND „Büchſe“ (with long s). In best case, she should get „Büchse“ AND „Büchſe“ AND „Buͤchſe“. How can I achieve, that //text[. contains text { } treats s and ſ and ü and uͤ as the same character? Thanks a lot for any help. Best regards, Guenter

3 2

Managing external entities
by Majewski, Steven Dennis (sdm7g) 26 Jul '19

26 Jul '19

I have a corpus of TEI files with figures and page images encoded as external entities. It appears that even when choosing “Parse DTDs and entities” this info is lost when parsing files into database, And in any case, unparsed-entity-uri() is an XSLT only function. It would appear that I first need to transform the files first and replace @entity attributes with @url attributes while these unparsed entity values are available, before creating the database, or else generate another database to map entity names to values later. Are there any better ways to handle this case ? Is there any way to do these transforms on the fly before parsing the files into the database ? The only thing that comes to mind is to set up a local SaxonServlet to do the transforms, and load from URLs instead of file paths. ( I’ve been doing something similar for a different case, and running into memory errors that I don’t see when loading from a directory when creating a database. Increasing memory didn’t help much, but inserting ‘flush’ command between even ‘add’ commands seemed to work. ) — Steve Majewski

2 2

BaseX 9.2.1: Maintenance Release
by Christian Grün 26 Jul '19

26 Jul '19

Dear all, A new BaseX maintenance release is available: - GUI: Parsing of project files has become faster and smarter. - RESTXQ: Custom reponses are now sent as streams. - Performance tweaks (static typing: typeswitch, instance of). - Minor bugs have been fixed. Looking forward to your feedback, Christian BaseX Team

1 3

File-io:error. File lock by another process (windows-only)
by France Baril 24 Jul '19

24 Jul '19

Hi, We have this issue only on Windows system since 9.2 that we know of... could be from 902. It is still happening after upgrade to 923. I am not able to provide a sample to reproduce systematically as it never happens at the same .xqm location. It does however always seem that the 1st error message occurs on a line with a file:write or file;append. I am giving you everything I know, in case that it rings a bell. 1. Start server with basexhttp 2. Run a process that has many functions and many file:write and file:append. 3. This generally works properly. The process ends with success. 4. Run the same process again, there is a good chance that we'll get an error that says something like: [file:io-error] Resource "debug-list-imtes.xml' locked by another process. 5. Run the same process again May get file-oi:error at the same location or a different one. 6. Restarting the server brings us back to 1... but sometimes we need to manually delete the files written with function from the file module before we can go back to 1 and get 1 good run of the process. Note: Although the process runs correctly and seems to finish running (to the same point that releases all locks on Linux). The person on the Windows system also often gets: [db:open] Resource "C:\basex\data\staging-en-us\txt.basex locked by another process and then the related database also remains locked and we may get messages: Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.2.3 Java: Oracle Corporation, 1.8.0_211 OS: Windows 10, amd64 Stack Trace: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source) at sun.nio.ch.SharedFileLockTable.add(Unknown Source) at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source) at org.basex.io.random.TableDiskAccess.lock(TableDiskAccess.java:147) at org.basex.io.random.TableDiskAccess.(TableDiskAccess.java:79) at org.basex.data.DiskData.init(DiskData.java:125) at org.basex.data.DiskData.(DiskData.java:84) at org.basex.core.cmd.Open.open(Open.java:99) at org.basex.query.QueryResources.database(QueryResources.java:179) at org.basex.query.func.StandardFunc.checkData(StandardFunc.java:429) at org.basex.query.func.db.DbOpen.value(DbOpen.java:17) at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:45) at org.basex.query.expr.path.IterPath$1.init(IterPath.java:79) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:46) at org.basex.query.expr.path.IterPath$1.next(IterPath.java:1) at org.basex.query.iter.Iter.value(Iter.java:65) at org.basex.query.expr.path.IterPath.nodes(IterPath.java:87) at org.basex.query.expr.path.AxisPath.value(AxisPath.java:39) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:173) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:80) at org.basex.query.func.StaticFunc.invValue(StaticFunc.java:172) at org.basex.query.func.FuncCall.invoke(FuncCall.java:117) at org.basex.query.func.FuncCall.value(FuncCall.java:56) at org.basex.query.scope.MainModule.value(MainModule.java:135) at org.basex.query.QueryContext.update(QueryContext.java:673) at org.basex.query.QueryContext.iter(QueryContext.java:332) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:73) at org.basex.http.web.WebResponse.create(WebResponse.java:56) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:54) at org.basex.http.BaseXServlet.service(BaseXServlet.java:64) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:542) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) at java.lang.Thread.run(Unknown Source)undefined -- France Baril Architecte documentaire / Documentation architect france.baril(a)architextus.com

2 10

Re: [basex-talk] Transaction Management
by Christian Grün 24 Jul '19

24 Jul '19

Hi Radim (cc to the list), > it takes so long even when it was being stored from a file on local hard drive. The xml file has about 15 to 20 MB. Wow. How large is your database? How did you add the documents? > Could you give me the XQuery or the BaseX command to move the file from the TempDB to the TargetDB the quickest possible way? Sure. There are various ways to do so, here’s just one (based on the database module [1]): 1. upload your document to database 'db-temp' 2. insert documents into final database: db:replace('db', db:list('db-temp'), db:open('db-temp')) 3. remove all your documents from the temporary database: db:delete('db-temp', '/') You can also drop the temporary database completely via db:drop. Hope this helps, Christian [1] http://docs.basex.org/wiki/Database_Module And how about database cache settings? Is something like that in BaseX? > > Thank you in advance for your response. > Best regards, > > Radim Havlicek > > > ---------- Původní e-mail ---------- > Od: Christian Grün <christian.gruen(a)gmail.com> > Komu: Radim Havlicek <radim-havlicek(a)post.cz> > Datum: 23. 7. 2019 15:12:04 > Předmět: Re: [basex-talk] Transaction Management > > Hi Radim, > > Do you have a clue how the long update time can be explained? Does it > take 20 minutes to upload your document to the BaseX server, or does > it take the same time if you update your database directly on the > server? If it’s the upload time, you could e.g. insert your document > in another temporary database and copy it to the final database in a > second step. > > Hope this helps, > Christian > > > > On Tue, Jul 23, 2019 at 10:34 AM <radim-havlicek(a)post.cz> wrote: > > > > Hello. > > > > We use client/server architecture, especially the C# client. I read that: > > > > Read transactions are executed in parallel. > > If an updating transaction comes in, it will be queued and executed after all previous read transaction have been executed. > > > > We have a web Front-End built on BaseX. We read XML documents stored in the database, and show charts with statistics calculated from those documents. Few times per day, we copy a new document to the database (via REPLACE command to make sure, the document is not stored in the database already). > > > > The problem we are facing is that while big document is being stored to the database (10 to 20 minutes), we cannot read documents from the database and display charts for users on the web interface. The web interface is waiting until the document is stored, and then read the the needed documents, and display the charts. It is very inconvenient for users to wait so long for a chart that is normally displayed in few seconds. > > > > Is there a way we could read the xml documents from the database while another document is being stored to the database? It is very, very needed. > > > > Best regards, > > > > Radim Havlicek

1 0

Transaction Management
by radim-havlicek＠post.cz 23 Jul '19

23 Jul '19

Hello. We use client/server architecture, especially the C# client. I read that: * Read transactions are executed in parallel. * If an updating transaction comes in, it will be queued and executed after all previous read transaction have been executed. We have a web Front-End built on BaseX. We read XML documents stored in the database, and show charts with statistics calculated from those documents. Few times per day, we copy a new document to the database (via REPLACE command to make sure, the document is not stored in the database already). The problem we are facing is that while big document is being stored to the database (10 to 20 minutes), we cannot read documents from the database and display charts for users on the web interface. The web interface is waiting until the document is stored, and then read the the needed documents, and display the charts. It is very inconvenient for users to wait so long for a chart that is normally displayed in few seconds. Is there a way we could read the xml documents from the database while another document is being stored to the database? It is very, very needed. Best regards, Radim Havlicek

2 1

Quickest way to parse a DomSource
by Marco Lettere 22 Jul '19

22 Jul '19

Hi all, I'm currently working on the usage of BaseX Xquery scripts inside a BPMN engine. I have to access portions of the XML representation of the BPMN file by binding the engine's API. The closest that I can get to a standard XML representation is currently a javax.xml.transform.dom.DOMSource. What is the shortest way to transform (in XQuery) this DomSource to a BaseX representation of the Document in order to query it? Thanks, Marco.

2 2

PATCH verb in RestXQ implementation
by Schopper, Daniel 19 Jul '19

19 Jul '19

Dear list, I just came across the fact that BaseX does not recognize PATCH as a known HTTP verb (at least in my admittedly not bleading-edge version). Is this a feature or a bug? And, in order to work around that: is there a way of passing payload to a custom %rest:method (without having an additional PUT or POST annotation on the same XQuery function)? Thanks in advance! Best Daniel

3 4

Update operation is spawning new databases with underscore
by Zimmel, Daniel 18 Jul '19

18 Jul '19

Hi, sometimes in my update operations BaseX deliberately is spawning several databases like the following, with a name of the database plus underscore and number: DBNAME_2618234251 caused by db:replace($dbname, $dbpath|| '/' || $fname, $file-to-import, map { 'intparse': true(), 'chop': false(), 'addcache': false() }) I cannot find anything about this behavior in the docs. It seems to happen randomly, and the main database does seem ok. How can I avoid these extra databases? They will vanish only by force. BaseX 9.2.1 Thanks, Daniel

2 4

Bug in 9.2.3??
by Buddy Kresge 17 Jul '19

17 Jul '19

The below code works fine in 9.2.2 (and any version before), but when we execute the below in 9.2.3 we get the following error. Thanks in advance for your assistance. Cannot convert element() to xs:string?: $status := <EmploymentStatusCode>Active</Employment.... declare function local:employmentStatus($employee as element()?) as xs:string? { $employee/*:Employment/*:EmploymentStatus/*:EmploymentStatusCode }; declare function local:isActive($employee as element()?) as xs:boolean { let $status := local:employmentStatus($employee) return if (string-length($status) > 0) then contains($status, 'Active') else false() }; let $ee := <Subscriber><Employment><EmploymentStatus><EmploymentStatusCode>Active</EmploymentStatusCode></EmploymentStatus></Employment></Subscriber> return local:isActive($ee)

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk