BaseX-Talk

basex-talk@mailman.uni-konstanz.de

4 participants
5268 discussions

Optimizer and JSON/XML response
by Marco Lettere 17 May '19

17 May '19

Hello everyone, we came accross an inconsistency which we are not able to find an explanation for and giving up after having spent on it the whole morning... From a restxq we call an external REST service (InfluxDB) like shown in [1]. This usually works as expected returning a response forged as in [2]. When the same RESTXQ is deployed in a much more complex application (with several RESTXQ modules and REPO modules) the first call (which takes a lot of time because it incurs the initial compiling delay) returns [3] instead of [2]. From the second call onwards the result stabilizes correctly to [2]. If I call any other RESTXQ function as a sort of warm-up (spending there all the initial delay) [1] behaves correctly returning always [2]. Do you please have a clue on why this happens? We noticed it for the first time because we started to use service jobs which cause this access pattern ... Thanks for any help. [1] module namespace test = 'test'; declare %rest:path("test") %rest:GET function test:test() { let $url := "http://localhost:8086/query?pretty=false&chunked=false&db=nestway&a…" let $results := http:send-request(<http:request method="get"/>, $url) return $results }; [2] <http:response xmlns:http="http://expath.org/ns/http-client" status="200" message="OK"> <http:header name="Transfer-Encoding" value="chunked"/> <http:header name="Connection" value="close"/> <http:header name="X-Influxdb-Version" value="1.1.1"/> <http:header name="Date" value="Thu, 16 May 2019 11:05:59 GMT"/> <http:header name="Request-Id" value="95b729c7-77ca-11e9-8065-000000000000"/> <http:header name="Content-Type" value="application/json"/> <http:body media-type="application/json"/> </http:response> <json type="object"> <results type="array"> <_ type="object"> <series type="array"> <_ type="object"> <name>frontoffice</name> <columns type="array"> <_>time</_> <_>consumable</_> <_>customer</_> <_>guests</_> <_>type</_> <_>user</_> </columns> <values type="array"> <_ type="array"> <_>2019-05-10T21:55:00Z</_> <_>access</_> <_>c9a83ce3-c938-4677-a0df-72bc96354509</_> <_ type="number">0</_> <_>starttimer</_> <_>d50de945-23ed-4b33-84e8-8ef533a96fb7</_> </_> </values> </_> </series> </_> </results> </json> [3] <http:response xmlns:http="http://expath.org/ns/http-client" status="200" message="OK"> <http:header name="Transfer-Encoding" value="chunked"/> <http:header name="Connection" value="close"/> <http:header name="X-Influxdb-Version" value="1.1.1"/> <http:header name="Date" value="Thu, 16 May 2019 11:19:30 GMT"/> <http:header name="Request-Id" value="7957030a-77cc-11e9-8066-000000000000"/> <http:header name="Content-Type" value="application/json"/> <http:body media-type="application/json"/> </http:response> map { "results": [map { "series": [map { "values": [["2019-05-10T21:55:00Z", "access", "c9a83ce3-c938-4677-a0df-72bc96354509", 0.0e0, "starttimer", "d50de945-23ed-4b33-84e8-8ef533a96fb7"]], "name": "frontoffice", "columns": ["time", "consumable", "customer", "guests", "type", "user"] }] }] }

2 3

Database lock during RESTXQ operation
by Martin Iggulden 17 May '19

17 May '19

We just encountered a stuck lock on a BaseX database. It was during repeated calls to a RESTXQ operation that modifies the database. The operations were working OK, then one returned an error: [db:lock] Database 'd7c552cc-1f5e-47dc-ba91-37054695aabf' is currently opened by another process. This error then appeared for any further attempt to update the database, and persisted for several days (although it seems to have cleared now). It's possible that two updates were being processed concurrently, but they would have been processed by the same Java VM because the RESTXQ server is the only active interface to the database. We have seen this error before, but it's difficult to reproduce. Do you know how this might have happened? And is there a procedure for clearing down an erroneous database lock if it happens? We are running BaseX version 9.1.2. Thanks, Martin Iggulden Screen Systems Martin Iggulden | Software Engineer | SCREEN Main Line : +44 1473 831700 | Direct Dial: +44 1473 831700 Fax : +44 1473 830078 Martin.Iggulden(a)screensystems.tv<mailto:Martin.Iggulden@screensystems.tv> Visit us at Broadcast Asia 18-20th June 2019, Suntec Singapore, stand 6J4-02 [http://www.subtitling.com]<http://www.subtitling.com/> [https://www.linkedin.com/company/screen-subtitling-systems-ltd] <https://www.linkedin.com/company/screen-subtitling-systems-ltd> [https://www.youtube.com/channel/Screen Subtitling Systems] <https://www.youtube.com/channel/UCBp2nyUeNbIFz9cD66Ym3AQ> [https://twitter.com/ScreenSystems] <https://twitter.com/ScreenSystems> [cid:SS2979-sasquatch-final-logos_landscape-logo-w236-h56_0717b6a0-e87e-466e-ba39-e63542a11eee.jpg]<https://mysasquatch.info> P Before printing, think about the environment This message may contain confidential and/or privileged information. If you are not the intended recipient you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. Screen Subtitling Systems Ltd. Registered in England No. 2596832. Registered Office: The Old Rectory, Claydon Church Lane, Claydon, Ipswich, Suffolk, IP6 0EQ

2 4

Extracting function signatures from documentation
by Andreas Mixich 16 May '19

16 May '19

Hi, I am doing some tooling for XQuery and need to extract function signatures from text, like from the BaseX module library documentation. But with my regex skills I just hit the wall. Next step would be to write some complex program, that does the parsing, maybe in combination with regexes, but before I start into that, I wanted to ask, whether the work has already been done. Is there a marked up version of the BaseX function docs? Or, even better (since I could apply it to other docs, then) does anyone know of a regular expression or an XQuery routine, that analyzes function signatures? Thanks a lot! -- Minden jót, all the best, Alles Gute, Andreas Mixich

4 7

Re: [basex-talk] Extracting function signatures from documentation
by Christian Grün 16 May '19

16 May '19

Hi Loren, So far I haven’t. Where do I find it? Does it also contains the BaseX functions, or the standard W3 function set? Thanks, Christian Loren Cahlander <loren.cahlander(a)gmail.com> schrieb am Do., 16. Mai 2019, 21:02: > Hello Christian, > > Have you taken a look at the xqDoc module generated from ANTLR 4? > > Loren > > > On May 16, 2019, at 2:59 PM, Christian Grün <christian.gruen(a)gmail.com> > wrote: > > Exactly; thanks, Bridger! In the etc directory of our zip distribution, > there is an xqdoc.zip file, which contains the generated library modules > for all BaseX-specific functions. > > > > > > Bridger Dyson-Smith <bdysonsmith(a)gmail.com> schrieb am Do., 16. Mai 2019, > 20:47: > >> Hi Andreas - >> >> (someone on the BaseX team will surely correct where I get this wrong, >> so...) as far as I know, many/most/all of the BaseX builtin modules are >> written in Java, not XQuery, so I don't know that there are modules to >> inspect, so to speak. However, late last year there was some discussion on >> the list along these same lines and Christian wrote up an XQuery script to >> generate xqDoc output from the wiki [1]. >> >> Maybe that would be helpful for your project? >> Best, >> Bridger >> >> [1] https://github.com/BaseXdb/basex-dist/blob/master/wiki2xqdoc.xq >> >> >> >> On Thu, May 16, 2019 at 2:34 PM Andreas Mixich <mixich.andreas(a)gmail.com> >> wrote: >> >>> On Thu, May 16, 2019 at 5:54 PM Loren Cahlander < >>> loren.cahlander(a)gmail.com> wrote: >>> >>>> Please take a look at xqDoc for XQuery that is like JavaDoc for Java. >>>> >>>> https://github.com/lcahlander/xqdoc >>>> >>>> >>> Thanks for the hint! I did not yet think about looking at xqdoc. >>> Definitely will do this. See also my reply to Christian. >>> >>> -- >>> Minden jót, all the best, Alles Gute, >>> Andreas Mixich >>> >> >

1 0

inspect:functions#1 on XPath funcs errors out with [XPST0003] and returns an XHTML file
by Andreas Mixich 16 May '19

16 May '19

Hi, opening a new sheet in the GUI I just executed: inspect:functions("http://www.w3.org/2005/xpath-functions") and this is what I got: Error: Stopped at http://www.w3.org/2005/xpath-functions, 1/6: [XPST0003] Processing instruction has illegal name: xml. Optimized Query: inspect:functions("http://www.w3.org/2005/xpath-functions") The Query itself is, as returned by BaseX, interestingly, an XHTML file, which seems to have been produced via Saxon, here are the first lines: <?xml version="1.0" encoding="UTF-8"?>  <html xmlns="http://www.w3.org/1999/xhtml" lang="EN"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> <title>XQuery, XPath, and XSLT Functions and Operators Namespace Document </title> <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"/> <style type="text/css"> </style> <link rel="stylesheet" type="text/css" href=" http://www.w3.org/StyleSheets/TR/base.css"/> </head> <body> <div class="head"> <p> <a href="http://www.w3.org/"> <img src="http://www.w3.org/Icons/w3c_home" alt="W3C" height="48" width="72"/> </a> </p> <h1><a name="title" id="title"/>XQuery, XPath, and XSLT Functions and Operators Namespace Document </h1> <h2>21 March 2017</h2> </div> <nav id="toc"> <h2><a id="contents"/>Table of Contents </h2> <ol class="toc"> <li> <a href="#intro"> <span class="secno">1 </span> <span class="content">Introduction</span> </a> <ol class="toc"/> </li> And here is the QueryPlan: Query Plan: <QueryPlan compiled="true" updating="false"> <InspectFunctions name="inspect:functions" type="(function(*))*"> <Str type="xs:string" size="1">http://www.w3.org/2005/xpath-functions </Str> </InspectFunctions> </QueryPlan> -- Minden jót, all the best, Alles Gute, Andreas Mixich

2 2

Change data directory for basex & BaseXGUI
by Majewski, Steven Dennis (sdm7g) 16 May '19

16 May '19

I’ve been learning and testing basex with the default MacOS home-brew installation. For production, I’m planning on using the war file served using tomcat. I would like to use the basex command line client and BaseXGUI to do some initial setup and testing on the tomcat web app databases. Is there a java property setting or other way to set the data directory location for these programs ? Or do I have to set these paths in ~/basex/.basex file . The latter is not ideal, as I may need to run update scripts from different user cron jobs, so I’ld rather not depend on home directory. — Steve Majewski

2 3

Adventures with XQJ
by Gardner, Arthur 16 May '19

16 May '19

Hi folks. First a disclaimer: I'm omitting a lot of information, because there is hardly any limit to what I could include. Instead, I'll mention what seems most important, then dialog if anyone shows an interest. The objective: convert an existing private java library from ddxq to BaseX. I am using these libraries, all coming to me via BaseX (I think): [cid:image002.jpg@01D50A5D.04478550] My intention is to use the local option ... this library is already running in the server (or in Eclipse). import net.xqj.basex.local.BaseXXQDataSource; Here's my connection code (actually a constructor method): public XQTransformer (String xqueryName) throws Exception { String queryPath = String.format(filePath, xqueryName); URI queryUri = new URI(queryPath); FileReader xqueryReader = new FileReader(new File(queryUri)); xqds = new BaseXXQDataSource(); //xqds.setOptions("serialize=indent=yes"); //xqds.setBaseUri(queryPath); xqconnection = xqds.getConnection(); XQStaticContext context = xqconnection.getStaticContext(); context.setBindingMode(XQConstants.BINDING_MODE_IMMEDIATE); xqconnection.setStaticContext(context); xqpe = xqconnection.prepareExpression(xqueryReader); } setBaseUri is for references from the xQuery to helper files in the same directory. Later I compensate by overriding the current user directory. But I am throwing an Exception here: while (!rs.isClosed() && rs.next()) { java.lang.NullPointerException at org.basex.query.QueryContext.next(QueryContext.java:353) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:64) at org.basex.query.QueryContext.next(QueryContext.java:353) at org.basex.query.scope.MainModule$1.next(MainModule.java:101) at net.xqj.basex.bin.h.b(Unknown Source) at net.xqj.basex.bin.r.a(Unknown Source) at net.xqj.basex.bin.bL.next(Unknown Source) at net.xqj.basex.bin.bR.next(Unknown Source) at com.emc.metlife.util.xql.XqueryTransformation.xmlFormatted(XqueryTransformation.java:89) at com.emc.metlife.util.xql.XqueryTransformation.getTransformedData(XqueryTransformation.java:194) at com.emc.metlife.util.xql.XqueryPooledTester.test(XqueryPooledTester.java:58) at com.emc.metlife.util.xql.XqueryPooledTester.main(XqueryPooledTester.java:43) The particular xquery file is working for me in the BaseX GUI. Thanks! Arthur Gardner | Infosys Ltd MetLife | Clarks Summit, PA Office 570-587-6898 | Mobile 262-442-7472 The information contained in this message may be CONFIDENTIAL and is for the intended addressee only. Any unauthorized use, dissemination of the information, or copying of this message is prohibited. If you are not the intended addressee, please notify the sender immediately and delete this message.

2 5

Re: [basex-talk] New user questions
by Christian Grün 15 May '19

15 May '19

Hallo Joshua (cc to the list). If you want to process data that is bound to environment variables or system properties, you can use fn:environment-variable or proc:property [1]. In the prolog of your XQuery module, you can specify external variables. If you call BaseX on command line, values can be bound to these variables via the -b option: query.xq: declare variable $v external; $v * $v command-line call: basex -b v=123 query.xq 15129 XQuery itself is a very powerful language, and the built-in BaseX XQuery modules allow you to do many things that you are used from other programming languages (accessing files, sending HTTP requests, processing archives, JSON, HTML, etc) [3]. If you should require features that are not part of XQuery of the provided languages, you can also resort to Java code [4] or integrate additional modules [5]. Hope this helps, Christian [1] http://docs.basex.org/wiki/Process_Module#proc:property [2] http://docs.basex.org/wiki/Command-Line_Options#Standalone [3] http://docs.basex.org/wiki/Module_Library [4] http://docs.basex.org/wiki/Java_Bindings [5] http://docs.basex.org/wiki/Repository On Wed, May 15, 2019 at 6:30 PM Joshua Kordani <joshua.kordani(a)speedcastwireless.com> wrote: > > Christian, > > Thank you for your reply! I feel confident in my ability to construct the system in question, but I'm wondering how much of it will live in XQuery and how much will remain outside. As I've read through the documentation in more detail since my initial email, I do not see how the current offerings of XQuery modules will provide this functionality (I.e. user input either from environment variables or command line options fed to XQuery scripts) nor do I see this in the various tools command line options although I have not exhaustively searched. If I am wrong, I would appreciate a pointer to such, otherwise I suppose the answer will have to come from using the BaseX libraries or APIs. Do I have this right? > > Danke, > Joshua Kordani > > -----Original Message----- > From: Christian Grün <christian.gruen(a)gmail.com> > Sent: Wednesday, May 15, 2019 5:36 AM > To: Joshua Kordani <joshua.kordani(a)speedcastwireless.com> > Cc: basex-talk(a)mailman.uni-konstanz.de > Subject: Re: [basex-talk] New user questions > > Dear Joshua, > > I believe there are many ways to build a system you are envisioning; but I am not sure how to answer your question(s). If you are looking for someone who could build an individual software solution for you, based on BaseX, we invite you to write to info(a)basexgmbh.de and ask for our commercial services. > > Hope this helps, > Christian > > > On Tue, May 14, 2019 at 5:22 AM Joshua Kordani <joshua.kordani(a)speedcastwireless.com> wrote: > > > > Greetings all. > > > > I am attempting to setup an environment for processing junit xml output files, to be handed off to the end user. > > > > In general, batches of junit files will be imported over time, and as they get imported I wish to add timestamp tags to certain elements. Right now the only way I see to programmatically do this is take details from the filename, derive a datetime, and apply it to the elements in question that are associated with a given file. I am trying to make this process easy on the user, such that they specify a new file or set of files to import, then all pertinent records in each file get timestamped. I don't quite see how to access enough "external environment" to allow for a script to grab a user specified filename and do this work and I could use some pointers. I undertand that datetimes are not indexable fields, or else i'm not sure exactly what the documentation has to say about indexes and datetimes, but if there is anything I can do to allow a user to filter by them that would be great. > > > > The user is fine with what ultimately becomes a command line interface to import these files and generates reports. I can see how to build xquery commands to isolate all of the elements in question and pass statically baked data to variables such as date time to add to existing elements in the database, but I still am not sure how to use this to build a system to allow a user to add a file or set of files at a time and have them processed in this way. > > > > Thank you! > > Joshua Kordani > > DISCLAIMER: This email communication and any attached files are Speedcast Wireless proprietary and may be legally privileged. Export-controlled information shall not be disseminated without proper authorization and proper export-control markings, per Speedcast Wireless export policy. If you have received this transmission in error please notify the sender immediately and then delete this email and all its attachments. If you are not the addressee, any disclosure, reproduction, copying, distribution, or any other dissemination or use of this communication is strictly prohibited. Thank you.

2 1

New user questions
by Joshua Kordani 15 May '19

15 May '19

Greetings all. I am attempting to setup an environment for processing junit xml output files, to be handed off to the end user. In general, batches of junit files will be imported over time, and as they get imported I wish to add timestamp tags to certain elements. Right now the only way I see to programmatically do this is take details from the filename, derive a datetime, and apply it to the elements in question that are associated with a given file. I am trying to make this process easy on the user, such that they specify a new file or set of files to import, then all pertinent records in each file get timestamped. I don't quite see how to access enough "external environment" to allow for a script to grab a user specified filename and do this work and I could use some pointers. I undertand that datetimes are not indexable fields, or else i'm not sure exactly what the documentation has to say about indexes and datetimes, but if there is anything I can do to allow a user to filter by them that would be great. The user is fine with what ultimately becomes a command line interface to import these files and generates reports. I can see how to build xquery commands to isolate all of the elements in question and pass statically baked data to variables such as date time to add to existing elements in the database, but I still am not sure how to use this to build a system to allow a user to add a file or set of files at a time and have them processed in this way. Thank you! Joshua Kordani DISCLAIMER: This email communication and any attached files are Speedcast Wireless proprietary and may be legally privileged. Export-controlled information shall not be disseminated without proper authorization and proper export-control markings, per Speedcast Wireless export policy. If you have received this transmission in error please notify the sender immediately and then delete this email and all its attachments. If you are not the addressee, any disclosure, reproduction, copying, distribution, or any other dissemination or use of this communication is strictly prohibited. Thank you.

2 1

XQTest Exception
by Markus Schmid 15 May '19

15 May '19

Hallo ich bekomme folgende Exception bei dem auf der Website docs.basex.org/wiki/Unit_Module angegebenen Beispiel. Die Exception wird scheinbar durch Tests ausgelöst, die fehlschlagen. Werden die fehlgeschlagenen Tests rausgenommen, läuft das Modul durch. > TEST tests.xqm Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 9.2 Java: Oracle Corporation, 1.8.0_211 OS: Windows 10, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.unit.Unit.addError(Unit.java:246) at org.basex.query.func.unit.Unit.test(Unit.java:158) at org.basex.query.func.unit.Suite.test(Suite.java:54) at org.basex.core.cmd.Test.run(Test.java:37) at org.basex.core.Command.run(Command.java:257) at org.basex.core.Command.execute(Command.java:93) at org.basex.api.client.LocalSession.execute(LocalSession.java:132) at org.basex.api.client.Session.execute(Session.java:36) at org.basex.core.CLI.execute(CLI.java:92) at org.basex.core.CLI.execute(CLI.java:76) at org.basex.BaseX.console(BaseX.java:176) at org.basex.BaseX.<init>(BaseX.java:151) at org.basex.BaseX.main(BaseX.java:42)

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk