- BaseX-Talk - mailman.uni-konstanz.de

Single quotes with json::parse
by Thomas Daly 10 Aug '17

10 Aug '17

Regarding the JSON module json::parse function, how do you cope with single quotes within the JSON data? I have tried escaping them with a backslash, but to no avail. In this example I think it's the single quote in "St. John's College" that is causing the problem: XQUERY let $database := 'lk' let $options := map { 'format': 'direct', 'escape': 'yes' } let $j := json:parse('{"roles":["poet","cleric"],"institutions":["St John's College","Cambridge University"],"characterId":"595051204300","homes":["Cheapside","London","Dea n Prior","Devon"],"image":"robert_herrick.jpg","DOB":"1591-08-24","attributes" :["People educated at Merchant Taylors' School","Northwood","Alumni of St John's College","Cambridge","Alumni of Trinity Hall","Cambridge","English poets","People from the City of London","People educated at Westminster School","London","English male poets"],"DOD":"1674-10-08","nationality":"British","givenName":"Robert","fam ilyName":"Herrick"}', $options) return db:add($database, $j, 'characters/c_595051204300_Robert_Herrick.xml') Stopped at ., 3/78: [XPST0003] Expecting closing bracket: json:parse. at /usr/local/lib/basex-api/src/main/perl/BaseXClient.pm line 58. Many thanks, Thomas

3 3

How could I reuse sub queries?
by donaldjohn 07 Aug '17

07 Aug '17

Hi, everyone: Here is my query that finds the distinct value and counts them, you can see that I used the same sub query twice, first for finding the distinct values, second for count the distinct values, is there a way that I can cache the sub query result and reuse it somewhere else? I think it will run faster in that way. import module namespace fx = "http://www.functx.com" at "functx-1.0-doc-2007-01.xq"; declare namespace business="http://www.sipo.gov.cn/XMLSchema/business"; declare namespace base="http://www.sipo.gov.cn/XMLSchema/base"; (:Iterate through the distributed values:) for $dis_val in ( ( (:open multi database, then query the doc status field:) for $result in doc("50PatentDividedCreatingClause.xml")/results/result return db:open($result/dbName)/business:PatentDocumentAndRelated/@status ) =>distinct-values() ) return "value : " || $dis_val || ", count: "|| count( for $status in ( for $result in doc("50PatentDividedCreatingClause.xml")/results/result return db:open($result/dbName)/business:PatentDocumentAndRelated/@status ) where $status = $dis_val return $status||"")

2 2

How could I reuse sub queries?
by donaldjohn 07 Aug '17

07 Aug '17

Hi, everyone: Here is my query that finds the distinct value and counts them, you can see that I used the same sub query twice, first for finding the distinct values, second for count the distinct values, is there a way that I can cache the sub query result and reuse it somewhere else? I think it will run faster in that way.

1 0

jobs:result deadlock
by Kendall Shaw 07 Aug '17

07 Aug '17

Can someone elaborate on this sentence in the documentation about jobs:result: “Please note that this query can easily cause a deadlock if the asynchronously executed query will be queued. “ Referring to this example: let $query := jobs:eval('(1 to 10000000)[. = 1]', map{}, map{ 'cache': true() }) return ( jobs:wait($query), jobs:result($query) ) And this about jobs:wait “If the function is called with the id of a queued job, or repeatedly executed job, it may stall and never terminate.” Can jobs:eval above be queued before jobs:wait is evaluated within the let expression? Kendall

2 1

Web application port
by Thomas Daly 07 Aug '17

07 Aug '17

Regarding http://docs.basex.org/wiki/Options#HTTP_Services , I see there's an option STOPPORT to change the server stop port. I think it should be possible to configure the server listen port (default 8984) there as well. I realise it's possible to do this through command line options, but it would be nice to be able to do it in the .basex file too. Thanks, Thomas

2 1

GUI
by Thomas Daly 06 Aug '17

06 Aug '17

a) It is not possible to log into a BaseX database running on a remote server from a GUI running on a different machine? b) Do I need any particular Linux distribution / GUI version to run the BaseX Linux GUI? Many thanks, Thomas

3 3

Shouldn't CHOP = false make xml:space="preserve" the default behavior?
by Ottid Mes 05 Aug '17

05 Aug '17

>From the documentation about the CHOP option I assumed that since xml:space="preserve" sets CHOP = false for that part of the document, that if I set CHOP = false in my configuration file, that the behavior you get when you use xml:space="preserve" would be applied to the whole database (I created the database after setting the option). However the only way I have ever been able to get this behavior, has been to set xml:space="preserve" at the root element. Am I missing something, or is this a bug? How could I get this behavior by default in my databases? I thought this would not warrent a thorough example given the clear conditions which cause the above situation, but I was asked for it anyway, so here it goes: A little bit of context, although it should not matter, I have had this issue for years (at least 4) under Arch Linux. At the time I just assumed I did something wrong and went with xml:space="preserve" workaround everywhere. > uname -a Linux phoenix 4.9.38 #1-NixOS SMP Sat Jul 15 10:17:55 UTC 2017 x86_64 GNU/Linux The default situation: No config file (.basex) > basex BaseX 8.6.4 [Standalone] > create db chop-test Database 'chop-test' created in 123.01 ms. > open chop-test Database 'chop-test' was opened in 1.19 ms. > replace /a <root><a>foo</a> bar</root> 0 resource(s) replaced in 103.68 ms. > xquery /root <root> <a> foo </a>bar</root> Query executed in 106.05 ms. I never use the REPL other than to create and drop databases, so I was a bit suprised that this did not work: > replace /a <root xml:space="preserve"><a>foo</a> bar</root> "a.xml" (Line 1): Open quote is expected for attribute "xml:space" associated with an element type "root". While this does: > replace /a <root xml:space='preserve'><a>foo</a> bar</root> 1 resource(s) replaced in 4.06 ms. > xquery /root <root xml:space="preserve"><a>foo</a> bar</root> Query executed in 0.95 ms. > quit Have fun. The resource with xml:space="preserve" is the behavior I want to have within my database, because all my documents are mixed content. On the wiki (http://docs.basex.org/wiki/Options#CHOP) this is also mentioned: It explicitly states that in my use case I should set CHOP to false: "The flag should be turned off if a document contains mixed content." It also states that setting the xml:space="preserve" attribute is the same as having CHOP = false: "If the xml:space="preserve" attribute is attached to an element, chopping will be turned off for all descendant text nodes." So lets do that: Let us first confirm that the config file is correctly read: > echo 'FOO = 0' > /some/path/.basex > BASEX_JVM='-Dorg.basex.path=/some/path' basex /some/path/.basex: Unknown option 'FOO'. /some/path/.basex: writing new configuration file. Now we set the option CHOP = false in our config: > echo 'CHOP = false' >> /some/path/.basex So lets see what this changes in the basex REPL: > BASEX_JVM='-Dorg.basex.path=/some/path' basex BaseX 8.6.4 [Standalone] > drop db chop-test Database 'chop-test' was dropped. > create db chop-test Database 'chop-test' created in 106.42 ms. > open chop-test Database 'chop-test' was opened in 0.05 ms. > replace /a <root><a>foo</a> bar</root> 0 resource(s) replaced in 39.24 ms. > xquery /root <root> <a> foo </a> bar</root> Query executed in 97.09 ms. > quit Have fun. This is not what I expect, it should have been: > xquery /root <root><a>foo</a> bar</root> And hence my question: Shouldn't CHOP = false make xml:space="preserve" the default behavior? I even tried this: > BASEX_JVM='-Dorg.basex.path=/some/path' basex BaseX 8.6.4 [Standalone] Try 'help' to get more information. > open chop-test Database 'chop-test' was opened in 90.65 ms. > set chop off CHOP: false > replace /a <root><a>foo</a> bar</root> 1 resource(s) replaced in 45.58 ms. > xquery /root <root> <a> foo </a> bar</root> Query executed in 96.78 ms. > quit See you. Am I making some mistake in the above? Is the wiki simply outdated and should this be configured differently? Is having having mixed content in basex so rare that this bug has gone unnoticed for years?

1 0

Performance analysis in web DBA
by Erik Peterson 03 Aug '17

03 Aug '17

Is there a way to get the query plan and stats in the web dba...like the native client has with the info view?

2 1

Artifact in Maven Central?
by Kirsten, Dirk 02 Aug '17

02 Aug '17

Hi guys, awesome database you got there! Imagine you want to use this database in a big enterprisey environment, where they block all kinds of ridiculous stuff, because of reasons... So for example you can't simply add the BaseX repository to your pom.xml, because actually all access is routed through a central mirror repository. So it would be super handy if BaseX was available in Maven central, because this stuff will be available. Unfortunately, the most current BaseX release on Maven central is 7.3.1, which is rather outdated. Are there any plans to release to Maven Central once again? I think the easiest way would be to sign up for http://central.sonatype.org/pages/ossrh-guide.html, which syncs to maven central. There seems to be some maven automation available, so once it is setup I would guess that you simply have to add it to Christians' magic build scripts. My other alternative would be to add the BaseX repository to this proxy repository, which most likely involves me getting a Passagierschein A38. Please don't let me do that :( All the best Dirk Dirk Kirsten ______________________________ Senacor Technologies AG D?sseldorfer Stra?e 13 65760 Eschborn M +49 (151) 4435 1967 Dirk.Kirsten(a)senacor.com www.senacor.com Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht Frankfurt am Main - Reg.-Nr.: HRB 105546 Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: Daniel Gr?zinger Diese E-Mail inklusive Anlagen enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten, informieren Sie bitte den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. This e-mail including any attachments may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the materials in this e-mail is strictly forbidden.

2 3

Connect a BaseX web application to an already running BaseX server
by Fabrice ETANCHAUD 02 Aug '17

02 Aug '17

Dear all, I would like to connect a BaseX web application to an already running BaseX server, But I cannot figure out how to do it without using the Client module. I tried to set the -n (HOST) option at webapp startup but it did not help. Best regards, [cerfranceimage] Fabrice ETANCHAUD Analyste Developpeur Les Rocs, Chavagné CS 40070 79260 LA CRECHE Tél. 05 49 76 45 45- Fax 05 49 25 83 87 Email : fetanchaud(a)pch.cerfrance.fr [cerfrancedevise] poitoucharentes.cerfrance.fr<http://www.poitoucharentes.cerfrance.fr> [facebook]<https://www.facebook.com/Cerfrance-Poitou-Charentes-686810324682939> [twitter]<https://twitter.com/CERFRANCE_PCH> [youtube]<https://www.youtube.com/user/CERFRANCEnational> [linkedin]<https://fr.linkedin.com/company/cerfrance-national-> [viadeo]<http://fr.viadeo.com/fr/company/cerfrance>

2 2

State of replication and clustering
by Andreas Jung 02 Aug '17

02 Aug '17

Hi there, what is the state of replication and clustering of BaseX? I found an XML Prague 2013 presentation but almost no documentation on these topics on the website. In our case we need to scale out horizontally with a growing number of reads (no writes involved). Andreas

4 5

Deployment of BaseX on Windows in Service Fabric?
by Andreas Jung 01 Aug '17

01 Aug '17

Hi there, has anyone experiences with deploying BaseX in Azure Service Fabric on a Windows node either as Guest Executable or Windows Container? Andreas

1 0

Atomic replacement of documents with a collection
by Andreas Jung 01 Aug '17

01 Aug '17

Hi there, I am finally heading for the first time to use BaseX in a customer project. We need to import XML based product catalogs in different languages. The catalog files should be updated each night without service interruption. What is the best strategy to update and replace an existing document without race condition? Should I uploaded the new version of a catalog under a different filename first and then use REPLACE? Is REPLACE atomic in the sense that an ongoing request will see the „old“ state of the document and new requests will see the new state of the document without any undefined state in between? Andreas

2 2

Bug Report creating instance of BXDatabase
by Ignacio Hernandez-Ros 31 Jul '17

31 Jul '17

Hi, Bug Report! When running this code in a class on my own package I receive an IllegalAccessException Class<?> c = Class.forName("org.basex.api.xmldb.BXDatabase"); Database db = (Database) c.newInstance(); java.lang.IllegalAccessException: Class my.package can not access a member of class org.basex.api.xmldb.BXDatabase with modifiers “" I think the problem is that the class is not public. Best regards Ignacio

2 1

Re: [basex-talk] Timezone and job start time
by Kendall Shaw 29 Jul '17

29 Jul '17

I think I can assume it is UTC. org/basex/util/DateTime.java contains: static { FULL.setTimeZone(TimeZone.getTimeZone("UTC")); } If there is a way to use the default timezone that would be better. Meanwhile, I made a module so that I can use source from JDK 1.8.x: package somewhere; import java.time.LocalDateTime; import java.time.ZonedDateTime; import java.time.ZoneId; import java.time.format.DateTimeFormatter; import org.basex.query.QueryException; import org.basex.query.QueryModule; public class Time extends QueryModule { public String zonedTime(String pattern) throws QueryException { return ZonedDateTime.from(LocalDateTime.now() .withHour(20) .withMinute(0) .withSecond(0) .atZone(ZoneId.systemDefault())) .withZoneSameInstant(ZoneId.of("UTC")) .format(DateTimeFormatter.ofPattern(pattern)); } } and, then in XQuery: jobs:eval(‘trace(“WENT”)’, map {}, map {‘id’: ‘somejob’, ‘start’: t:zoned-time(‘HH:mm:ss’)}) Kendall From: <basex-talk-bounces(a)mailman.uni-konstanz.de> on behalf of Kendall Shaw <kendall.shaw(a)workday.com> Date: Friday, July 28, 2017 at 2:37 PM To: BaseX <basex-talk(a)mailman.uni-konstanz.de> Subject: [basex-talk] Timezone and job start time declare namespace tz = "java:java.time.ZonedDateTime"; tz:now() returns 2017-07-28T14:23:08.334-07:00[America/Los_Angeles] Logs in dba seem to use the local time zone to display the date. The timestamp on resources is in UTC time, which is good. modified-date=”2017-07-28T21:17:53.347Z” But, do I have to use UTC time for scheduling jobs? For example: job:eval(‘…’, map {}, map {‘start’: ‘20:00:00’}) schedules the job for 5AM instead of 10PM, I think. If I can assume the time is UTC then I can figure out how to schedule the correct time. But, can I assume that it is UTC, or can I specify a Zoned time without a date? Kendall

2 4

Timezone and job start time
by Kendall Shaw 28 Jul '17

28 Jul '17

declare namespace tz = "java:java.time.ZonedDateTime"; tz:now() returns 2017-07-28T14:23:08.334-07:00[America/Los_Angeles] Logs in dba seem to use the local time zone to display the date. The timestamp on resources is in UTC time, which is good. modified-date=”2017-07-28T21:17:53.347Z” But, do I have to use UTC time for scheduling jobs? For example: job:eval(‘…’, map {}, map {‘start’: ‘20:00:00’}) schedules the job for 5AM instead of 10PM, I think. If I can assume the time is UTC then I can figure out how to schedule the correct time. But, can I assume that it is UTC, or can I specify a Zoned time without a date? Kendall

1 0

BaseX 8.6.5
by Christian Grün 28 Jul '17

28 Jul '17

Hi all, We have just released another version of our XML DBMS. The XQuery compiler of BaseX 8.6.5 provides some performance tweaks to improve typing of pre-evaluation expressions and, as usual, minor bug fixes. Visit basex.org to get the latest update. Your feedback is welcome. Have fun and a nice weekend, Christian

1 0

org.basex.http source code and client-server mode
by Ignacio Hernandez-Ros 27 Jul '17

27 Jul '17

Hello, As this is my first post on the list I’ll introduce myself first. My name is Ignacio Hernandez-Ros (if you google my name there is only me and my son with that name…) and I’m from Madrid - Spain. About 15 years ago I learned about XQuery using “tamino” form Software-AG and now I’ve discovered baseX that looks amazing to me. I’ve been trying to configure the web administration portal to be used with a remote server with no success. It looks like it is not possible to take advantage of the client-server architecture from the Web Application http://docs.basex.org/wiki/Web_Application <http://docs.basex.org/wiki/Web_Application> I would like to know more about how the org.basex.http.restxq.RestXqServlet works and looking at the source code on GitHub I’m missing the org.basex.http package so I cannot see how can I improve it to allow it to connect to an existing server. I will appreciate any hint on how can I continue, either about where the source code is and how to contribute to it or how to set the WebApplication to run under client-server mode Best regards Ignacio

3 5

Re: [basex-talk] Join operation and the database
by Christian Grün 27 Jul '17

27 Jul '17

Hi Giuseppe, > If I compare values just using one indexed string (the one in @v), this is the fastest way (about one second on my machine). It depends on the value you are looking for. If it only occurs once in your database, the lookup will be very fast (in database terms, this is called a “high selectivity”). > If I compare against two distinct indexed values, their order matters, in that -if I understand correctly- the database uses the index only(?) for the first values. Exactly. You can enforce index access by directly using db:text(). > I see that [p = $t/@o and f = $t] is much slower than [f = $t and p = $t/@o]. I calculated that on average f contains about 8 characters while p always contains 9. However, (Ancient Greek) characters in f are heavier ( 2 or 3 bytes each) than the (Latin) ones in p (1 byte each). Can this be the reason why [f = $t and p = $t/@o] is evaluated faster? This doesn’t matter (as long as the string length does not exceed MAXLEN [1]). The critical question is how many index results you will get for a single lookup. See the following example: declare variable $txts := doc("tlg0001.tlg001.perseus-grc2.xml"); for $t in ($txts//t)[position() = 1 to 10] return ( "* " || $t/@o || ": " || count(db:text("splitted-db", $t/@o)), "* " || $t || ": " || count(db:text("splitted-db", $t)) ) The first lookup will return much more hits than the second one. You can call the following function to get a complete list of all index entries: index:texts('splitted-db') Cheers, Christian [1] http://docs.basex.org/wiki/Options#MAXLEN

1 0

Join operation and the database
by Giuseppe Celano 27 Jul '17

27 Jul '17

Hi, I performed join operations between many files and a dictionary. The files contain tokenized texts, where one finds word forms + fine-grained POS tags. Look at the following file: https://raw.githubusercontent.com/gcelano/POStaggedAncientGreekXML/master/t… <https://raw.githubusercontent.com/gcelano/POStaggedAncientGreekXML/master/t…> The dictionary, which contains word forms + fine-grained POS tags + lemmas, can be found here: https://github.com/gcelano/LemmatizedAncientGreekXML/tree/master/uniqueToke… <https://github.com/gcelano/LemmatizedAncientGreekXML/tree/master/uniqueToke…> I created a database for the dictionary and wrote a query (here simplified) like the following: for $t in $s/t (: t are the tokens in the file containing the tokens :) let $match := $lemm//d[./p = $t/@o and ./f = $t/text()] (: $lemm//d are the single entries in the dictionary :) return $match I see that if I use this query, it is slow, as if the processor cannot use the database indexes (./p and ./f). The situation does not seem to improve with ./p/text() and ./f/text(), which I would assume to be equivalent to the former because of atomization. On the contrary, if the same information contained in ./p and ./f are merged together and put in an attribute (see @v in the dictionary files) and this is compared against the values in the text (after concatenating them properly), the join operation is super fast (i.e., the index for the values in the attributes are used by BaseX). Does anyone know why? I have been able to get my results via the above (slow) comparison, but I would like to know what the cause of the problem was, if possible. Thanks. Best, Giuseppe Universität Leipzig Institute of Computer Science, Digital Humanities Augustusplatz 10 04109 Leipzig Deutschland E-mail: celano(a)informatik.uni-leipzig.de E-mail: giuseppegacelano(a)gmail.com Web site 1: http://www.dh.uni-leipzig.de/wo/team/ Web site 2: https://sites.google.com/site/giuseppegacelano/

2 4

Access output:media-type value from QueryProcessor
by Andrew Tsui 26 Jul '17

26 Jul '17

Hi, Is it possible to get the value of "output:media-type" through QueryProcessor? I am currently trying to use BaseX as a view renderer for a Spring MVC application. The view is rendering fine using embedded BaseX. However, I can't figure out how to get the "output:media-type" value from the XQuery so that I can set the response header during runtime. I can set the content-type through the controller in Spring. But I would rather not have to do it in two separate places. Below is the relevant code snippet from the view renderer. try(QueryProcessor proc = new QueryProcessor(query, context)) { Iter iter = proc.iter(); //response is HTTPServletResponse response.setContentType(????); try(Serializer ser = proc.getSerializer(response.getOutputStream())) { for(Item item; (item = iter.next()) != null;) { ser.serialize(item); } } } Any suggestion? Thanks Andrew

2 2

JSON to map
by Florian Wagner 19 Jul '17

19 Jul '17

Hey there, I'm probably missing something but please bear with my as I'm new to XML databases and XQuery. So what I've done is put JSON documents into BaseX using the REST interface and PUT requests. Now I want to use them in a RESTXQ handler. Loading the document using "db:open" will return me a document-node() but I'd like a map instead for simpler use and automatic types (I'll be outputing JSON again). The only thing I could come up with and that works is using "Content-Type: application/json;format=basic" when uploading and then: parse-json(xml-to-json(db:open(...))) But that seems very complicated and wasteful. Is there no other way? Regards Florian Wagner --- Universitaetsbibliothek Tuebingen Wilhelmstr. 32, 72074 Tuebingen IT-Abteilung, Raum B109 +49(0)7071/29-77860 http://www.ub.uni-tuebingen.de

2 1

Local permissions / REST / write permission needed
by Florian Wagner 19 Jul '17

19 Jul '17

Hi everyone! Trying out BaseX I'm having problems getting a very simple setup to work correctly. > list Name Resources Size Input Path ------------------------------------- test 2 33470 ~webdav 1 6004 2 database(s). > show users Username Permission -------------------- admin admin test none 2 user(s). > show users on test Username Permission -------------------- test write 1 user(s). $ curl -u test -i -X PUT -T file.xml http://host:8080/basex/rest/test/file.xml Enter host password for user 'test': HTTP/1.1 100 Continue HTTP/1.1 400 Bad Request Content-Type: text/plain;charset=utf-8 Content-Length: 24 Server: Jetty(9.3.9.v20160517) write permission needed. Using the "admin" user instead, works! Giving the test user global writer permissions makes it also work... So what am I doing wrong here? Regards Florian Wagner --- Universitaetsbibliothek Tuebingen Wilhelmstr. 32, 72074 Tuebingen IT-Abteilung, Raum B109 +49(0)7071/29-77860 http://www.ub.uni-tuebingen.de

2 4

Embedded BaseX
by George Sofianos 14 Jul '17

14 Jul '17

Hello, I'm building a web service that along with some other things (XML Validation, Saxon XQuery, etc) includes a BaseX processor for running XQuery scripts. I'm wondering what is the best way to use an embedded BaseX processor. Can some objects be shared (e.g Context, QueryProcessor), or are they very lightweight and there is no need to share / reuse ? Can I get the number of the jobs running and the job status like in the client / server mode? My intention is to make sure I can limit the amount of the jobs running in parallel, and have more control over the execution (maybe cancel jobs if necessary) Thanks, George

3 4

Re: [basex-talk] How could I get the current database name in xquery?
by Christian Grün 14 Jul '17

14 Jul '17

Hi Donald, Did you try db:name()? See our documentation for more information [1]. Christian [1] http://docs.basex.org/wiki/Database_Module Am 14.07.2017 8:12 vorm. schrieb "donaldjohn" <donaldjohn(a)qq.com>: > Hi, > How could I get the current database name in xquery? I need extract the > database input path and then concate it with the relative XML path, I want > to get the current opened database name rather than specify it. How could I > archive this? >

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk