BaseX-Talk

basex-talk@mailman.uni-konstanz.de

1 participants
5252 discussions

REPLACE vs STORE
by radim-havlicek＠post.cz 14 Nov '19

14 Nov '19

Hello, I would like to store new reports into existing database folder (with other existing files already) after opening. Help: REPLACE Syntax REPLACE [path] [input] XML Syntax <replace path='...'>[input]</replace> STORE Syntax STORE (TO [path]) [input] XML Syntax <store (path='...')>[input]</store> But actually I encountered this problem: > open mydb Database 'mydb' was opened in 3.0 ms. > replace /Report/ c:/Share/REPORTS/ 903 resource(s) replaced in 11077.73 ms. > store to /Report/ c:/Share/REPORTS/ Resource "C:/Share/REPORTS/" not found. What am I doing wrong? Is it possible to use store to instead of replace to just add files to existing ones? Thanks in advance for your answer, Radim Havlicek

2 1

Feature request: leveraging db:open-id and db:node-id
by France Baril 14 Nov '19

14 Nov '19

Hi, any chance we could make the db indexed ids match the xml doc ids? - db:open-id() could match the id at the root of then documents (when one is provided) - db:node-id() could match other nodes match internal ids (when they are provided). It would make indexing and querying a lot more efficient for any type of navigation/linking purposes. Options: - I'd suggest that this could be an option that can be turned on/off for each doc level ids and node level ids - I'd suggest that node id has the option to be unique within the db or within a document when the value for the entire db would then have to become something like doc-id#node-id to stay unique within the db Then I can easily imagine also expending to db:save-id to reduce the workload of having to pass through base-uri() once the doc exists in the db under its path let $db-name := 'en-us' let $doc := db:open-id($db-name, $doc-id) let $new-doc := copy, modify, return copy db:replace-id('en-us', $doc-id, $new-doc) instead of let $db-name := 'en-us' let $doc := db:open($db-name)/*[@id=$doc-id] let $new-doc := copy, modify, return copy let $path := substring-after($doc/base-uri(), $db-name) return db:replace($db-name, $path, $new-doc) Is that something that can be considered?

2 4

I am looking for the fastest way to sort 2.4 Mio tags by two attribute ascending and descending
by Omar Siam 14 Nov '19

14 Nov '19

Hi, I have a custom index that looks like this (one db, different files): <_:dryed xmlns:_="https://www.oeaw.ac.at/acdh/tools/vle/util" db_name="z881_qdb-TEI-02n" order="none"> <_:d pre="15627" db_name="z881_qdb-TEI-02n" xml:id="z881_qdbn-d16e2" vutlsk="tsįttr Ziter [Subst]" vutlsk-archiv="HK 881, z8810118.sch#1"/> <_:d pre="15673" db_name="z881_qdb-TEI-02n" xml:id="z881_qdbn-d16e21" vutlsk="tsįttr Ziter [Subst]" vutlsk-archiv="HK 881, z8810118.sch#1"/> ... </_:dryed> <_:dryed xmlns:_="https://www.oeaw.ac.at/acdh/tools/vle/util" db_name="f227_qdb-TEI-02n" order="none"> <_:d pre="467" db_name="f227_qdb-TEI-02n" xml:id="f237_qdb-d1e29398" vutlsk="(aus)faren [Verb]" vutlsk-archiv="HK 327, f227#944.1 = fare0126.eck#1.1"/> <_:d pre="591" db_name="f227_qdb-TEI-02n" xml:id="f237_qdb-d1e29438" vutlsk="(aus)faren [Verb]" vutlsk-archiv="HK 327, f227#945.1 = fare0126.eck#2.1"/> ... </_:dryed> There are about 2.4 Mio _:d tags in this db. I need to sort them by the @vutlsk* attributes alphabetically in ascending and descending order. With the code I have now: declare namespace _ = "https://www.oeaw.ac.at/acdh/tools/vle/util"; let $sorted-ascending := subsequence(for $d in collection('_qdb-TEI-02__cache')//*[@order="none"]/_:d order by $d/@vutlsk ascending return $d/(@ID, @xml:id)/data(), 1, 10000) let $sorted-descending := subsequence(for $d in collection('_qdb-TEI-02__cache')//*[@order="none"]/_:d order by $d/@vutlsk descending return $d/(@ID, @xml:id)/data(), 1, 10000) let $sorted-ascending-archiv := subsequence(for $d in collection('_qdb-TEI-02__cache')//*[@order="none"]/_:d order by $d/@vutlsk-archiv ascending return $d/(@ID, @xml:id)/data(), 1, 10000) let $sorted-descending-archiv := subsequence(for $d in collection('_qdb-TEI-02__cache')//*[@order="none"]/_:d order by $d/@vutlsk-archiv descending return $d/(@ID, @xml:id)/data(), 1, 10000) return (db:replace("_qdb-TEI-02__cache", 'ascending_cache.xml', <_:dryed order="ascending" ids="{string-join($sorted-ascending, ' ')}"/>), db:replace("_qdb-TEI-02__cache", 'descending_cache.xml', <_:dryed order="descending" ids="{string-join($sorted-descending, ' ')}"/>), db:replace("_qdb-TEI-02__cache", 'ascending-archiv_cache.xml', <_:dryed order="ascending" label="archiv" ids="{string-join($sorted-ascending-archiv, ' ')}"/>), db:replace("_qdb-TEI-02__cache", 'descending-archiv_cache.xml', <_:dryed order="descending" label="archiv" ids="{string-join($sorted-descending-archiv, ' ')}"/>)) This takes 30 s to about a minute depending on the subsequence I choose. I did experiments with doing multithreading and not. Multiple jobs or fork-join make it worse. Worst case I need to do it every time I save a change to the original DBs for which I maintain that index. Any ideas how to speed this up? Best regards Omar Siam

2 5

Re: [basex-talk] http:send-request - problem with response
by Christian Grün 13 Nov '19

13 Nov '19

Hi Bogdan (cc to the list), Thanks for digging deeper. I noticed that the standard Java function that we used returned only one value per header field, and dropped the others [1]. I managed to fix this in the latest stable snapshot; could you give it a try [2]? BaseX 9.3 will be released end of November. Cheers, Christian [1] http://files.basex.org/releases/latest/ [2] https://github.com/BaseXdb/basex/issues/1751 On Wed, Nov 13, 2019 at 4:26 PM Bogdan Bogucki <bbogucki(a)hussar.pl> wrote: > > Hi Christian, > Lack of many redirection is not a big problem. I can handle it manually. > Problem is not complete response header form first request. > > Browser make 3 redirection I am talking about first. > > Please take a look. > > > Response header from browser (first request) - contains three fields Set > Cookies: > > set-cookie __cfduid=daf54110b2a87d66c2a53…; domain=.pracuj.pl; HttpOnly > > set-cookie _yaic=13; expires=Fri, 31-Dec-9999 23:59:59 GMT; path=/ > > set-cookie _urnadiam=A; domain=.pracuj.pl…3:00:00 GMT; path=/; HttpOnly > > but response header form http:send-request contains only last one: > > <http:header name="Set-Cookie" value="_urnadiam=B; domain=.pracuj.pl; > expires=Tue, 31-Dec-2019 23:00:00 GMT; path=/; HttpOnly"/> > > I need information from previous set-cookie fields to make manual > redirection passing information about sessions to next requests. > > Regards > > Bogdan Bogucki > > W dniu 29.10.2019 o 03:06, Christian Grün pisze: > > Hi Bogdan, > > > > The current http:send-request implementation is based on the default > > Java HttpURLConnection, which does not resolve redirects that use > > different protocols [1]. This is the reason why your request will not > > be fully processed (as it e.g. happens when you use the browser). > > > > It seems that your initial request to the https protocol returns a 302 > > redirect to a (now unsafe) http URL, which returns another redirect to > > https. I don’t know who maintains the discussed web site, but it could > > be worth contacting the admins and asking them if they could update > > and simplify their redirect policy. > > > > Hope this helps, > > Christian > > > > [1] https://stackoverflow.com/a/1884427/1018790 > > > > > > > > > > On Fri, Oct 25, 2019 at 1:52 PM Bogdan Bogucki <bbogucki(a)hussar.pl> wrote: > >> > >> Hello, > >> > >> I encounter problem with http:send-request function. I need handle multiple forward requests (3) with cookies. > >> > >> First request returns cookies information which are required with next requests. Output from http:send-request dosen't contains all fields which are returned by server. > >> > >> Response from browser is flowing: > >> cache-control > >> private > >> cf-cache-status > >> DYNAMIC > >> cf-ray > >> 52b3cedc5b9bcc9f-WAW > >> content-type > >> text/html; charset=utf-8 > >> date > >> Fri, 25 Oct 2019 11:21:37 GMT > >> expect-ct > >> max-age=604800, report-uri="ht….com/cdn-cgi/beacon/expect-ct" > >> location > >> http://www.pracuj.pl/praca/jav…eloper-warszawa,oferta,7171988 > >> server > >> cloudflare > >> set-cookie > >> __cfduid=daf54110b2a87d66c2a53…; domain=.pracuj.pl; HttpOnly > >> set-cookie > >> _yaic=13; expires=Fri, 31-Dec-9999 23:59:59 GMT; path=/ > >> set-cookie > >> _urnadiam=A; domain=.pracuj.pl…3:00:00 GMT; path=/; HttpOnly > >> x-aspnet-version > >> 4.0.30319 > >> x-aspnetmvc-version > >> 5.2 > >> X-Firefox-Spdy > >> h2 > >> x-powered-by > >> ASP.NET > >> x-ua-compatible > >> IE=edge > >> > >> > >> Request: > >> > >> http:send-request( > >> <http:request method='get' follow-redirect='false'> > >> <http:header name="User-Agent" value="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0"/> > >> <http:header name="Accept" value="text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"/> > >> <http:header name="Accept-Languag" value="pl,en-US;q=0.7,en;q=0.3"/> > >> <http:header name="DNT" value="1"/> > >> <http:header name="Connection" value="keep-alive"/> > >> <http:header name="Upgrade-Insecure-Requests" value="1"/> > >> <http:body media-type='application/x-www-form-urlencoded; charset=UTF-8'/> > >> </http:request>, 'https://www.pracuj.pl/praca/java-developer-warszawa,oferta,7171988') > >> > >> Response form http:send-request is: > >> > >> <http:response xmlns:http="http://expath.org/ns/http-client" status="302" message="Found"> > >> <http:header name="Transfer-Encoding" value="chunked"/> > >> <http:header name="CF-RAY" value="52b3d0de097fb839-WAW"/> > >> <http:header name="Server" value="cloudflare"/> > >> <http:header name="Connection" value="keep-alive"/> > >> <http:header name="Date" value="Fri, 25 Oct 2019 11:23:00 GMT"/> > >> <http:header name="X-AspNetMvc-Version" value="5.2"/> > >> <http:header name="X-UA-Compatible" value="IE=edge"/> > >> <http:header name="CF-Cache-Status" value="DYNAMIC"/> > >> <http:header name="Cache-Control" value="private"/> > >> <http:header name="X-AspNet-Version" value="4.0.30319"/> > >> <http:header name="Set-Cookie" value="_urnadiam=B; domain=.pracuj.pl; expires=Tue, 31-Dec-2019 23:00:00 GMT; path=/; HttpOnly"/> > >> <http:header name="Expect-CT" value="max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct…"/> > >> <http:header name="X-Powered-By" value="ASP.NET"/> > >> <http:header name="Content-Type" value="text/html; charset=utf-8"/> > >> <http:body media-type="text/html"/> > >> </http:response> > >> > >> Two set-cookie fields is missing. > >> > >> How should I invoke http:send-request to receive proper result ? > >> > >> Regards > >> > >> Bogdan

2 1

proc:system
by Giuseppe G. A. Celano 13 Nov '19

13 Nov '19

Hi, When I try to run proc:system("/usr/local/bin/tesseract") I get the error [proc:code0001] Similarly: proc:system("tesseract") returns [proc:error] Cannot run program "tesseract": error=2, No such file or directory Similarly: proc:system("tesseract", (), map {"dir" : "/usr/local/bin/"}) returns [proc:error] Cannot run program "tesseract" (in directory "/usr/local/bin"): error=2, No such file or directory The command "tesseract" works at the command line. I suspect there may be a problem with permissions: is there a way to overcome this error? Thanks. Best, Giuseppe

2 5

Docker query
by Joseph Szili 13 Nov '19

13 Nov '19

Hello all, I'm trying to use the docker image for 9.x but when I surf to http://localhost:8984/dba/ per the documentation ... I get the following response in the browser No function found that matches the request. Using this command in linux (Ubuntu 19.04) ›$ docker run -d \ --name basexhttp \ --publish 1984:1984 \ --publish 8984:8984 \ --volume "$HOME/Projects/basex-dev/data":/srv/basex/data \ basex/basexhttp:latest container log >>>>>>>>>> /srv/basex/.basex: writing new configuration file. BaseX 9.3 beta [HTTP Server] [main] INFO org.eclipse.jetty.util.log - Logging initialized @248ms to org.eclipse.jetty.util.log.Slf4jLog [main] INFO org.eclipse.jetty.server.Server - jetty-9.4.21.v20190926; built: 2019-09-26T16:41:09.154Z; git: 72970db61a2904371e1218a95a3bef5d79788c33; jvm 1.8.0_212-b04 [main] INFO org.eclipse.jetty.util.TypeUtil - JVM Runtime does not support Modules [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor - NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet [main] INFO org.eclipse.jetty.server.session - DefaultSessionIdManager workerName=node0 [main] INFO org.eclipse.jetty.server.session - No SessionScavenger set, using defaults [main] INFO org.eclipse.jetty.server.session - node0 Scavenging every 600000ms Server was started (port: 1984). java.io.FileNotFoundException: /srv/basex/data/.logs/2019-11-08.log (No such file or directory) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.<init>(FileOutputStream.java:213) at org.basex.server.LogFile.create(LogFile.java:31) at org.basex.server.Log.write(Log.java:128) at org.basex.server.Log.writeServer(Log.java:70) at org.basex.BaseXServer.<init>(BaseXServer.java:122) at org.basex.http.HTTPContext.init(HTTPContext.java:101) at org.basex.http.BaseXServlet.init(BaseXServlet.java:37) at org.eclipse.jetty.servlet.ServletHolder$WrapperServlet.init(ServletHolder.java:1287) at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:599) at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:425) at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:751) at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743) at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:361) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1443) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1407) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:821) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:276) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:407) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:106) at org.eclipse.jetty.server.Server.doStart(Server.java:371) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.basex.BaseXHTTP.<init>(BaseXHTTP.java:120) at org.basex.BaseXHTTP.main(BaseXHTTP.java:52) [main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@545997b1{BaseX: The XML Database and XQuery Processor,/,file:///srv/basex/webapp/,AVAILABLE}{/srv/basex/webapp} [main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@3c756e4d{HTTP/1.1,[http/1.1]}{0.0.0.0:8984} [main] INFO org.eclipse.jetty.server.Server - Started @678ms HTTP STOP Server was started (port: 8985). java.io.FileNotFoundException: /srv/basex/data/.logs/2019-11-08.log (No such file or directory) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.<init>(FileOutputStream.java:213) at org.basex.server.LogFile.create(LogFile.java:31) at org.basex.server.Log.write(Log.java:128) at org.basex.server.Log.writeServer(Log.java:70) at org.basex.BaseXHTTP.<init>(BaseXHTTP.java:150) at org.basex.BaseXHTTP.main(BaseXHTTP.java:52) HTTP Server was started (port: 8984).

3 2

file:list and symbolic links
by Marco Lettere 09 Nov '19

09 Nov '19

Dear all, what is the rationale behind the fact that file:list has been rewritten to not following symbolic links [1]? We had some jobs related code that relied on symlinks and stopped working after moving to new version of BaseX. Is there any work around? By using children or the new descendants functions maybe? Thanks for your support. Marco. [1] https://github.com/BaseXdb/basex/blob/bbb62d1ec3e10b4fa1cc48bd4e16911fa73e4…

2 3

Backups
by Christoph Gaukel 09 Nov '19

09 Nov '19

Hi, the following handling of backups is surprising: * db:alter changes the name of the database but does not alter the names of the corresponding backups * db:drop deletes both, the database and the corresponding backups I would prefer, if there was an option in db:drop to keep the backups, which would allow to restore a deleted database from their backups. Is it possible to add options to db:drop, which allow that? For me, the behavior of db:alter looks like a bug. There is another minor handling, i would like to have changed: * db:info($db)/resourceproperties/inputdate is a UTC dateTime with a time zone offset * db:backups($db)/@date is a local dateTime without a time zone offset If I compare these two, it works correctly if the local time zone offset is the same as for the creation of the backup E.g.: if ( not(db:backups($db)) or max(db:backups($db)/@date/(. cast as xs:dateTime)) < db:info($db)/resourceproperties/inputdate/. cast as xs:dateTime ) then db:create-backup($db) else () Adding a time zone offset +hh:mm or -hh:mm to db:backups($db)/@date would solve this. Thanks, Christoph

2 2

"Out of Memory" when inserting data from one DB to another
by BIRKNER Michael 30 Oct '19

30 Oct '19

Hi to all, I get an "Out of Memory" error (using the BaseX GUI on Ubuntu Linux) when I try to insert quite a lot of data into a BaseX database. The use case: I have a database (size is about 2600 MB, 13718400 nodes) with information in <record> elements that should be added to <record> elements in another database. The <record>s have a 1 to 1 connection identified by an ID that is available in both databases. An example (simplified) of the DB with the information I want to add to the other DB: <collection> <record> <id>1</id> <data>Some data</data> <data>More data</data> <data>More data</data> ... </record> <record> <id>2</id> <data>Some data</data> <data>More data</data> <data>More data</data> ... </record> <record> <id>3</id> <data>Some data</data> <data>More data</data> <data>More data</data> ... </record> ... many many more <record>s </collection> Here an example (simplified) of the DB to which the above <data> elements should be added: <collection> <record> <id>1</id> <mainData>Main data</mainData> <mainData>More main data</mainData> <mainData>More main data</mainData> ...  </record> <record> <id>2</id> <mainData>Main data</mainData> <mainData>More main data</mainData> <mainData>More main data</mainData> ...  </record> <record> <id>3</id> <mainData>Main data</mainData> <mainData>More main data</mainData> <mainData>More main data</mainData> ...  </record> ... many many more <record>s </collection> This is the XQuery I use to insert the given <data> elements from the <record> in one database to the corresponding <record> in the other database. It results in an "Out of Memory" error: let $infoRecs := db:open('db-with-data')/collection/record let $mainRecs := db:open('db-to-insert-data')/collection/record for $infoRec in $infoRecs let $id := data($infoRec/id) let $mainRec := $mainRecs[id=$id] let $dataToInsert := $infoRec/*[not(name()='id')] return insert node ($dataToInsert) into $mainRec I assume that the error is a result of the large amount of data that is processed. My question is if a strategy exists to work with such an amount of data without getting an "Out of Memory" error? Thanks very much to everyone in advance for any hint and advice. If you need more information about DB setup or options just let me know. Best regrads, Michael Dieses Mail ist ausschließlich für die Verwendung durch die/den darin genannten AdressatInnen bestimmt und kann vertrauliche bzw rechtlich geschützte Informationen enthalten, deren Verwendung ohne Genehmigung durch den/ die AbsenderIn rechtswidrig sein kann. Falls Sie dieses Mail irrtümlich erhalten haben, informieren Sie uns bitte und löschen Sie die Nachricht. UID: ATU 16209706 I https://wien.arbeiterkammer.at/datenschutz

2 2

http:request parsing question
by Bridger Dyson-Smith 29 Oct '19

29 Oct '19

Hi all - I was hoping that someone on the list could help with my thinking about parsing the http:response values from a request; I've tried the following, but I'm not thinking about this correctly. declare namespace oai = "http://www.openarchives.org/OAI/2.0/"; declare namespace h = "http://expath.org/ns/http-client"; http:send-request(<http:request method='get' href=' http://export.arxiv.org/oai2?verb=Identify'/>)//@status/data() returns '200', but trying http:send-request(<http:request method='get' href=' http://export.arxiv.org/oai2?verb=Identify'/>)/h:response/@status fails. I'm guessing (but am not sure) that the combination of the <http:response> with the body of the response is causing my problem, but I'm not sure how to wiggle my way around it. Note: this is not in a RESTXQ context (yet), but for my simple trials is there a way to get at the http -namespace elements? Thank you for your time and trouble. Best, Bridger

3 5

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk