BaseX-Talk April 2020

basex-talk@mailman.uni-konstanz.de

23 participants
24 discussions

RuntimeException: Key should not exist
by Braunschweig, Björn 28 Apr '20

28 Apr '20

Hey there, I use basex in production for a couple of years now without problems. However on Saturday I got the error messages down below [1]. My storage architecture is that documents are saved throughout the day in a single database. For the next day a new database is created. When I got these errors and created a new database documents could be saved again. I think it has something to do with the index structures. The error messages were all similar but with different values after "key should not exist". Deployment method: docker BaseX 9.3.1 [Standalone] Try 'help' to get more information. > info General Information: Version: 9.3.1 Used Memory: 10 MB Global options: AUTHMETHOD: Basic CACHETIMEOUT: 3600 DBPATH: /srv/basex/data DEBUG: false FAIRLOCK: false GZIP: false HOST: localhost HTTPLOCAL: false IGNORECERT: false IGNOREHOSTNAME: false KEEPALIVE: 600 LANG: English LANGKEYS: false LOG: true LOGMSGMAXLEN: 1000 LOGPATH: .logs NONPROXYHOSTS: PARALLEL: 8 PARSERESTXQ: 3 PASSWORD: PORT: 1984 PROXYHOST: PROXYPORT: 0 REPOPATH: /srv/basex/repo RESTPATH: RESTXQERRORS: true RESTXQPATH: SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 30 USER: WEBPATH: /srv/basex/webapp Local options ADDARCHIVES: true ADDCACHE: false ADDRAW: false ARCHIVENAME: false ATTRINCLUDE: ATTRINDEX: true AUTOFLUSH: true AUTOOPTIMIZE: false BINDINGS: CASESENS: false CATFILE: CHECKSTRINGS: true CHOP: true COMPPLAN: true COPYNODE: true CREATEFILTER: *.xml CREATEONLY: false CSVPARSER: DEFAULTDB: false DIACRITICS: false DTD: false ENFORCEINDEX: false EXPORTER: FORCECREATE: false FTINCLUDE: FTINDEX: false FULLPLAN: false HTMLPARSER: INLINELIMIT: 100 INTPARSE: false JSONPARSER: LANGUAGE: en LSERROR: 0 MAINMEM: false MAXCATS: 100 MAXLEN: 96 MAXSTAT: 30 MIXUPDATES: false PARSER: xml QUERYINFO: false RUNQUERY: true RUNS: 1 SERIALIZE: true SERIALIZER: SKIPCORRUPT: false SPLITSIZE: 0 STEMMING: false STOPWORDS: STRIPNS: false TAILCALLS: 256 TEXTINCLUDE: TEXTINDEX: true TEXTPARSER: TOKENINCLUDE: TOKENINDEX: false UPDINDEX: false WITHDB: true WRITEBACK: false XINCLUDE: true XMLPLAN: false [1] Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.3.1 Java: IcedTea, 1.8.0_212 OS: Linux, amd64 Stack Trace: java.lang.RuntimeException: Key should not exist: 'urn:diasid:fty:kopal:0200507050000000000047' at org.basex.util.Util.notExpected(Util.java:61) at org.basex.index.value.UpdatableDiskValues.add(UpdatableDiskValues.java:77) at org.basex.data.Data.indexAdd(Data.java:1090) at org.basex.data.Data.insert(Data.java:828) at org.basex.query.up.atomic.Insert.apply(Insert.java:44) at org.basex.query.up.atomic.AtomicUpdateCache.applyUpdates(AtomicUpdateCache.java:294) at org.basex.query.up.atomic.AtomicUpdateCache.execute(AtomicUpdateCache.java:278) at org.basex.core.cmd.Replace.replace(Replace.java:106) at org.basex.core.cmd.Replace.access$000(Replace.java:20) at org.basex.core.cmd.Replace$1.run(Replace.java:58) at org.basex.core.cmd.ACreate.update(ACreate.java:90) at org.basex.core.cmd.Replace.run(Replace.java:55) at org.basex.core.Command.run(Command.java:257) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:105) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:93) at org.basex.http.rest.RESTExec.run0(RESTExec.java:31) at org.basex.http.rest.RESTCmd.run(RESTCmd.java:70) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.core.Command.execute(Command.java:116) at org.basex.http.rest.RESTServlet.run(RESTServlet.java:31) at org.basex.http.BaseXServlet.service(BaseXServlet.java:65) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1391) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:760) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang.Thread.run(Thread.java:748) Regards, Björn

2 3

Doc-available() function, behavior?
by Arjan Loeffen 26 Apr '20

26 Apr '20

Dear Basex People, I have this fragment: let $xml-doc := 'MyDatabase/bogus-entry.xml' let $list := *if (doc-available($xml-doc)) then doc($xml-doc)/** else () return $list I'd expect BaseX to return the empty sequence when the document is not in the collection. But Basex returns an error: [basex:doc] Database path 'MyDatabase/bogus-entry.xml' yields no documents. What am I doing wrong? I thought I was following the example in https://docs.basex.org/wiki/Databases#XML_Documents nicely... Thanks in advance!! -- *Arjan Loeffen* Armatiek BV & Armatiek Solutions BV 06-12918997

3 4

Grouping words in phrasal matches with full-text indexes
by Graydon Saunders 26 Apr '20

26 Apr '20

Hello -- So my overall goal is to take a bunch of XML, mark all the (generally phrasal) terms of art, take that modified content and mark all the (possibly phrasal) glossary terms, and then go through and remove all the glossary markers that happen to be inside terms of art and then remove all the term-of-art markers. (There's an intermediate step between "found all the possible glossary terms" and "have applied the glossary terms" where the list of candidate terms gets sent off for semantic approval, so the "find a term" steps and "change the documents in which the terms are found" steps have to be distinct.) My initial problem was marking phrasal terms; the full-text index is very fast and solves the "this rapidly becomes a nightmare with regular expressions, especially regular expressions with no "whole words only" switch, problem, but it marks every word in the phrasal term individually. I think I have figured out a way to connect the adjacent marked words in the phrasal term into a single mark element. I cannot convince myself that this is the right way; is there a better approach than tumbling windows? (: db:create("DB", <para id="GUID-12354" >Diverse and various words, some of which are going to be tagged for review as glossary terms.</para>, 'test.xml', map { 'ftindex': true() }) :) (: example phrasal term :) let $term as xs:string := 'Diverse and various' for $ft in (db:open('DB')//*[text() contains text { $term } phrase using case sensitive]) return <changed>{ let $contents as node()+ := ft:mark($ft[text() contains text { $term } phrase using case sensitive],'mark') return element {name($contents)} { $contents/@*, (: has to handle hyphens as well as spaces :) for tumbling window $w in $contents/node() start $s when true() end $e previous $eprev next $enext when ( $enext[not(self::mark)] and ($enext[normalize-space()][not(matches(.,'^-$'))]) ) or ($enext[self::mark] and $e[normalize-space()][not(matches(.,'^-$'))]) return if ($w[self::mark]) then <mark>{string-join($w,'')}</mark> else $w } }</changed> thanks! Graydon

4 4

analyze-string
by Graydon Saunders 23 Apr '20

23 Apr '20

So if I create a query in the BaseX 9.3.2 GUI, thus: declare function local:processMatch($in as text(),$matchPattern as xs:string) as item()+ { let $search as element() := analyze-string(normalize-space($in),$matchPattern) return $search }; let $example := <para>These are a bunch of words, and some of the words are <em>special</em> words.</para> for $text in $example//text() return local:processMatch($text,'bunch|special') I get: <fn:analyze-string-result xmlns:fn="http://www.w3.org/2005/xpath-functions"> <fn:non-match>These are a </fn:non-match> <fn:match>bunch</fn:match> <fn:non-match> of words, and some of the words *are*</fn:non-match> </fn:analyze-string-result> <fn:analyze-string-result xmlns:fn="http://www.w3.org/2005/xpath-functions"> <fn:match>special</fn:match> </fn:analyze-string-result> <fn:analyze-string-result xmlns:fn="http://www.w3.org/2005/xpath-functions"> <fn:non-match>words.</fn:non-match> </fn:analyze-string-result> The trailing space after the *are* in the second fn:non-match element is gone. Should that happen? Can I make it stop happening by invoking the function differently? Thanks! Graydon

2 1

A question about RESTXQ rest:form-param()
by Zimmel, Daniel 23 Apr '20

23 Apr '20

Hi, in an HTML form with a RESTXQ POST action, do I really have to declare %rest:form-param() for every param, or is there a way to pass the full query component in one parameter? Best, Daniel

2 7

Create a document in a created database
by Ramzi Hammouda 20 Apr '20

20 Apr '20

Hello BaseX team, This may sound simple, but didn't find how to do it, what is the xquery for creating an empty xml document in a chosen database. Thanks. P Respectons ensemble l'environnement. N'imprimez ce message que si n?cessaire. Let's respect the environment together. Only print this message if necessary.

3 2

Error in the text for the server protocol?
by Ben Engbers 16 Apr '20

16 Apr '20

Hi Michael, I only noticed today that in the server procol (https://docs.basex.org/wiki/Server_Protocol) there are 2 different instructions for adding a new resource. On page 3: ADD \09 {name} {path} {input} On page 6: void add(String path, InputStream input) I has been pure luck that I mixed both instructions. I have implemented ADD as \09 {path} {input} and that works perfectly. Session$Add("Test.xml", "<Line_1 line='1'>Content 1</Line_1>") adds resource Test.xml with the xml as content. Meanwhile it is very easy for me to change the instructions that are send to the server so I tried to add {name} to the ADD command. That results in errors. My guess is that you should use either {name} or {path} and that ADD only works on the database in use. Is that correct? Cheers, Ben

1 0

Binding a variable to a sequence
by Ben Engbers 14 Apr '20

14 Apr '20

Hi, I am still trying to improve my R-package and at the moment I am working (again) on the 'Binding' command. I have added this XML to a database: <book title="XQuery" author="Walmsley"/> <book title="Advanced R" author="Wickham"/> This query: for $t in collection("TestDB/Books")/book where $t/@author = "Walmsley" return $t/@title/string() returns: XQuery My command: Bind(Query_3, "$name", list("Walmsley", "Wickham")) sends the following byte-sequence to the server: 57 61 6c 6d 73 6c 65 79 01 57 69 63 6b 68 61 6d 00 00 And this sequence is accepted by the server. In Query_3, I try to bind $name to the sequence: declare variable $name external; for $t in collection('TestDB/Books')/book where $t/@author = $name return $t/@title/string() The query is executed but no results are given. (I had expected that it would return the sequence {"XQuery", "Advanced R"} How should I correct the query-statement? Cheers, Ben

2 1

Unescaping revritten element name
by Mauro Mugnaini 11 Apr '20

11 Apr '20

Hi all, After parsing strings in XML with direct representation (e.g. with CSV and JSON module parsing) the specific entries are represented via elements and their names are rewritten. In particular (as per documentation): "underscores are rewritten to two underscores (__), and characters that are not valid in element names are rewritten to an underscore and the character’s four-digit Unicode.” It is possible to convert back the element’s node name with an XQuery function present in some module? Regards, Mauro

2 2

Write output using proc:execute
by Tim Thompson 09 Apr '20

09 Apr '20

Hello, Is it possible to redirect output from a system command within the proc:execute function? I would like to do something like: proc:execute("echo", ("hello!", "> hello.txt")) but the ">" does not seem to be recognized. Thanks in advance, Tim -- Tim A. Thompson Discovery Metadata Librarian Yale University Library

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk April 2020