BaseX-Talk January 2016

basex-talk@mailman.uni-konstanz.de

39 participants
68 discussions

Re: [basex-talk] Connecting database in C#
by Christian Grün 11 Feb '16

11 Feb '16

Hi Raj (cc to the list), > Yes, I did. I have created a Session class (provided online) but unable to connect it to the existing database. > // create session > Session session = new Session("localhost", 1984, "admin", "admin"); > > It doesn't find the localhost on my machine. I guess, I am missing something. Do I need to install SQL server first? BaseX is the database, so there’s no need for SQL server. Did you start a BaseX server instance? Our documentation might give you more insight, see the "Getting Started" Section.

2 3

BaseX, Zorba, e-mail module
by Tim Thompson 04 Feb '16

04 Feb '16

Hello, I would like to be able to send e-mail from a Web app I am developing in BaseX and was wondering whether anyone else had implemented this functionality (or whether a BaseX e-mail module might be in the works?). I see that Zorba has an e-mail module with support for SMTP and IMAP (not to mention many other useful modules), which leads me to wonder whether anyone has worked on a BaseX connector for Zorba. In the meantime, I decided to try using the BaseX process module to call Zorba, like so: proc:system( 'zorba', ( '-q', convert:binary-to-string( db:retrieve( 'out', 'email.xq' ) ) ) ) I stored the XQuery code for Zorba as a raw file in the BaseX database, since storing it in the webapp directory caused an error, due to the unrecognized Zorba module declarations. This approach seems to work adequately, but does anyone have any suggestions for improvement, or a different approach? Thanks in advance, Tim -- Tim A. Thompson Metadata Librarian (Spanish/Portuguese Specialty) Princeton University Library

3 4

file:write and include-content-type
by James Ball 31 Jan '16

31 Jan '16

Hello, I’m getting an unexpected behaviour with the serialization option include-content-type and file:write with HTML. If I write a document without a <head> element then one doesn’t get added and nothing happens. file:write("test.htm",<html><body/></html>,map {"method":"html", "html-version":"4.0","include-content-type":"yes”}) returns: <html> <body></body> </html> Is this correct? If there is a head then the <meta> gets added with no problems - unless there is already a meta element, in which case it gets replaced. file:write("/Users/jb8748/test.htm",<html><head><meta http-equiv="refresh" content="60" /></head><body/></html>,map {"method":"html", "html-version":"4.0","include-content-type":"yes”}) returns: <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body></body> </html> Is this correct? All running Running Beta 8.4 f5341cc Many thanks, James

2 11

basexhttp jetty 413 (Request Entity Too Large: head) / consecutive queries
by Günter Dunz-Wolff 31 Jan '16

31 Jan '16

Hi everyone, searching the basex rest-interface with a VERY long query ends in the following error-message: In Terminal basexhttp: [qtp1513712028-21] WARN org.eclipse.jetty.http.HttpParser - HttpParser Full for SCEP@5be8c98d{l(/0:0:0:0:0:0:0:1:49641)<->r(/0:0:0:0:0:0:0:1:8984),s=1,open=true,ishut=false,oshut=false,rb=false,wb=false,w=true,i=0r}-{AsyncHttpConnection@5d928144,g=HttpGenerator{s=0,h=-1,b=-1,c=-1},p=HttpParser{s=-10,l=0,c=-3},r=0} In the Browser: 413 (Request Entity Too Large: head) Is there any way, to enhance the memory for the HttpParser or is the only way to limit the http-request params? Here is the request, I know, it's a little bit crazy, but it's searching for synonyms and their wordforms for the german word "zahlen": http://localhost/rest?run=xqs/search_new.xq&$suche=zahlen&collect=all_kleis… Alternativ perhaps I could use consecutive queries in xquery, but I can't figure out the right approach. Where and how to store the temporary sub-sequences? Thanks for any help! Best regards, Günter

2 1

Re: [basex-talk] Rép: Re: Slow XQueries on a quite small database but great number of docs
by Christian Grün 31 Jan '16

31 Jan '16

Hi Jean Marc (cc to the list), What happens if you run the query in the GUI (how long does it take)? How much time is spent if you write the client output to a file (basexclient -o output.xml)? Which client do you use? On Sun, Jan 31, 2016 at 5:41 PM, <ansel.jm(a)free.fr> wrote: > Many many thanks for your quickness ! > Yes I optimized the database (OPTIMIZE ALL). I don't undestand where the pb is. > The query I ran is the one I have transmitted to you in the last mail. So there is a pb with the server I'm afraid. May be with java ? > When I try to execute the query from basexclient, I'have a immediate answer but a very very very slow display : lines are displayed in blocks of one or two screen terminals... There is a problem with the data exchange between the daemon and the client I think...

1 0

BaseX fn:collection() questions
by Bridger Dyson-Smith 31 Jan '16

31 Jan '16

Hi all, I'm using BaseX 8.3.1 and am trying to access a directory structure like so: declare namespace t = "http://www.tei-c.org/ns/1.0"; for $ready in fn:collection("/usr/home/bridger/Documents/foo/bar/?select=*.xml;recurse=yes")/t:TEI let $title := fn:data($ready/t:teiHeader/t:fileDesc/t:titleStmt/t:title) return $title The structure under bar/ looks something like bar/ 001/ 001.xml 002/ 002.xml 003/ 003.xml ... ... 009/ 009.xml Does BaseX's fn:collection() allow for recursion into a directory structure like this with the /?select syntax options or should I use a combination of functions from the File Module to move through these directories? Note: I'm not pulling these files into a database, rather I'm just using BaseX as a query processor. Thanks for your time. Best, Bridger

2 2

tumbling window and db:add in same expression
by Hondros, Constantine (ELS-AMS) 29 Jan '16

29 Jan '16

Hello all, BaseX doesn't seem to like the query below. Any particular reason - it makes sense to me. I'm trying to add items to a DB incrementally to (attempt to) avoid running into out of memory exceptions when cutting a slice of a large DB into a smaller one. for tumbling window $w in db:open(myDB')//ani:ref-authors/ani:author start at $s when $s mod 50000 = 1 return db:add( 'sliceDB', <index>{$w}</index>, 'somepath.xml' ) Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 8.3 Java: Oracle Corporation, 1.8.0_40 OS: Windows 7, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.expr.ParseExpr.checkAllUp(ParseExpr.java:219) at org.basex.query.expr.ParseExpr.checkNoneUp(ParseExpr.java:203) at org.basex.query.expr.gflwor.Window.checkUp(Window.java:341) at org.basex.query.expr.gflwor.GFLWOR.checkUp(GFLWOR.java:791) at org.basex.query.QueryContext.check(QueryContext.java:265) at org.basex.query.QueryParser.finish(QueryParser.java:279) at org.basex.query.QueryParser.parseMain(QueryParser.java:176) at org.basex.query.QueryContext.parseMain(QueryContext.java:221) at org.basex.query.QueryProcessor.parse(QueryProcessor.java:56) at org.basex.core.cmd.AQuery.parse(AQuery.java:144) at org.basex.core.cmd.AQuery.updating(AQuery.java:158) at org.basex.core.cmd.AQuery.updating(AQuery.java:288) at org.basex.gui.GUI.exec(GUI.java:458) at org.basex.gui.GUI.access$400(GUI.java:43) at org.basex.gui.GUI$7.run(GUI.java:412) ________________________________ Elsevier B.V. Registered Office: Radarweg 29, 1043 NX Amsterdam, The Netherlands, Registration No. 33156677, Registered in The Netherlands.

2 1

Unrecognized Options Running Basex in Docker Container
by Eliot Kimber 29 Jan '16

29 Jan '16

I'm seeing a difference in how the .basex file is processed between running a server under OS X directly and running it in a Docker container. In particular, my local settings are being rejected as unrecognized. Here's the startup messages from within the basex container: basex@611404b26b04:~$ basexhttp /home/basex/.basex: Unknown option 'CATFILE'. /home/basex/.basex: Unknown option 'DTD'. /home/basex/.basex: Unknown option 'SKIPCORRUPT'. /home/basex/.basex: Unknown option 'CHOP'. /home/basex/.basex: writing new configuration file. [main] INFO org.eclipse.jetty.server.Server - jetty-8.1.17.v20150415 [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor - NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet BaseX 8.3 [Server] Server was started (port: 1984). [main] INFO org.eclipse.jetty.server.AbstractConnector - Started SelectChannelConnector@0.0.0.0:8984 HTTP Server was started (port: 8984). I verified that with the same .basex file running 8.3 under OS X the options are recognized and set as expected. What would cause this difference in behavior? Here is the .basex file in the /home/basex directory when the server starts: USER = admin PASSWORD = admin DEBUG = false DBPATH = /home/basex/basex/data REPOPATH = /home/basex/basex/repo LANG = English LANGKEYS = false GLOBALLOCK = false # Client/Server Architecture HOST = localhost PORT = 1984 SERVERPORT = 1984 SERVERHOST = PROXYHOST = PROXYPORT = 0 NONPROXYHOSTS = IGNORECERT = false TIMEOUT = 30 KEEPALIVE = 600 PARALLEL = 8 LOG = true LOGMSGMAXLEN = 1000 # HTTP Services WEBPATH = /home/basex/basex/webapp RESTPATH = RESTXQPATH = CACHERESTXQ = false HTTPLOCAL = false STOPPORT = 8985 AUTHMETHOD = Basic # Local options CATFILE = /opt/dita-ot/DITA-OT/catalog-dita.xml DTD = true SKIPCORRUPT = true CHOP = false -- (this is the end of the file) -- After the server starts the offending options are omitted from the rewritten .basex file. The only difference I can think of is the Java version. The container uses the OpenJDK while I have Oracle Java running in OS X: basex@611404b26b04:~$ java -version openjdk version "1.8.0_66-internal" OpenJDK Runtime Environment (build 1.8.0_66-internal-b17) OpenJDK 64-Bit Server VM (build 25.66-b17, mixed mode) basex@611404b26b04:~$ But otherwise it's exactly the same code running in both environments and the same config file. Thanks, Eliot ---- Eliot Kimber, Owner Contrext, LLC http://contrext.com

2 3

Java client and binary files
by E. Wray Johnson 29 Jan '16

29 Jan '16

When using the sample Java client to store/retrieve binary/raw files I have to execute separate open/close commands. It is my understanding that this locks the database. In a multi-user scenario, such locking is undesirable as binary/raw files can be quite large. And since there is no alteration of the indexes etc during such operations, is there a better way? Also, why isn't there a corresponding retrieve method on the Java client, similar to the store method? My code to store: BaseXClient session = new BaseXClient(host, port, username, password); session.execute("open " + database); session.store(path, inputStream); session.execute("close"); My code to retrieve: BaseXClient session = new BaseXClient(host, port, username, password); session.execute("open " + database); session.execute("retrieve " + path, outputStream); session.execute("close"); Thanks in advance!

3 9

Fwd: Re: Status of BaseX Docker Container?
by Christian Grün 29 Jan '16

29 Jan '16

---------- Weitergeleitete Nachricht ---------- Von: "Eliot Kimber" <ekimber(a)contrext.com> Datum: 28.01.2016 11:57 nachm. Betreff: Re: [basex-talk] Status of BaseX Docker Container? An: "Christian Grün" <christian.gruen(a)gmail.com> Cc: > Refined the dfst/basex container definition and figured out what I was > doing wrong on OS X. > > I updated the container to expose as volumes the repo, data, and webapp > directories. This allows two things: > > 1. You can mound a host directory as any of these directories in place of > the default one. This lets you have local persistent storage or swap in a > different set of modules or webapp modules for the same container. > > 2. Other containers can access these volumes from within the same Docker > network using the --volumes-from parameter of the docker run command (not > sure why you'd want to do this but there might be some reason. > > On OS X I realized that when you start a Docker-specific shell window > (e.g., using the Kitematic tools "docker cli" button) it sets environment > variables that will not, by default, be set in any other shell window. So > you have to remember to do all your Docker stuff in the cli shell or > figure out how to set the environment in your .bashrc or whatever. > > In any case, assuming you have Docker otherwise running you can run the > container like this: > > docker run -d -p 8984:8984 -p 1984:1984 --name basex dfst/basex > > > This will run the server as a background task (-d) and expose the default > server ports on the docker machine. > > If you want to swap in your own data directory you use the -v parameter > like so: > > docker run -d -p 8984:8984 -p 1984:1984 -v > /Users/someuser/data:/opt/basex/data --name basex dfst/basex > > However, I don't yet know what the right permissions settings are to allow > basex to write to a mounted data volume. In my experiments BaseX could > create a database directory but then failed with a permissions error when > it tried to create its files with in it. Something silly on my part I'm > sure. > > > My understanding is that Docker on OS X only shares your home directory > through the VirtualBox VM, so you can't mount directories outside your > home directory, at least by default. You also have to make sure the > directory permissions allow the basex user to write to it. > > > You then use the IP address of the Docker machine as the base IP address, > which you can determine using the command "docker-machine ip default: > > bash-3.2$ docker-machine ip default > 192.168.99.100 > > > With that you can connect to the running Web server using > http://192.168.99.100:8984 and away you go. > > The image is "dfst/basex" on the Docker hub server. I also created an > image with the tag "8.3" (dfst/basex:8.3). > > Cheers, > > Eliot > > ---- > Eliot Kimber, Owner > Contrext, LLC > http://contrext.com > > > > > On 1/27/16, 4:06 AM, "Christian Grün" <christian.gruen(a)gmail.com> wrote: > > >…thanks for the hints on the Windows solution! I’ll try it in a while. > > > > > >On Tue, Jan 26, 2016 at 11:13 PM, Eliot Kimber <ekimber(a)contrext.com> > >wrote: > >> Forgot to add the Windows part: > >> > >> Under windows have to explicitly publish the ports when running the > >> container: > >> > >> docker run --name=basex -p 8984:8984 -p 1984:1984 dfst/basex > >> > >> then you can use the IP address of the docker-engine VM to connect to > >>the > >> server: > >> > >> http://192.168.99.100:8984/ > >> > >> you can get the ip address using the "docker-machine ip" command: > >> c:\Program Files\Docker Toolbox>docker-machine ls > >> NAME ACTIVE URL STATE URL > >> SWARM DOCKER ERRORS > >> default - virtualbox Running tcp://192.168.99.100:2376 > >> v1.9.1 > >> c:\Program Files\Docker Toolbox>docker-machine ip default > >> 192.168.99.100 > >> > >> the docker-machine command replaces the older boot2docker command > >> > >> So now I have BaseX server running and accessible as Docker containers > >> under Windows and CentOS. > >> > >> OSX is still a problem: there's some issue with the docker daemon and > >> getting the docker client to connect to it. Haven't been able to find a > >> good solution in my searching. Very annoying. > >> > >> Cheers, > >> > >> E. > >> > >> > >> ---- > >> Eliot Kimber, Owner > >> Contrext, LLC > >> http://contrext.com > >> > >> > >> > >> > >> On 1/26/16, 3:50 PM, "Eliot Kimber" > >> <basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of > >> ekimber(a)contrext.com> wrote: > >> > >>>For what it's worth, I've pushed a generic BaseX HTTP container to > >>>Docker > >>>Hub as "dfst/basex". It exposes the default ports for the base and HTTP > >>>servers (1984 and 8984). > >>> > >> >From a Linux system you connect by using the IP address of the Docker > >>>network, e.g.: > >>> > >>>http://172.17.0.2:8984/ > >>> > >>>Under Windows: > >>> > >>>The DockerFile is in GitHub here: > >>> > >>>https://github.com/dita-for-small-teams/dfst-docker > >>> > >>>My next task will be to create a container based on this one that > >>>includes > >>>specific configuration and the DFST Web application modules. Should be > >>>easy once I get to it. > >>> > >>>Cheers, > >>> > >>>E. > >>> > >>>---- > >>>Eliot Kimber, Owner > >>>Contrext, LLC > >>>http://contrext.com > >>> > >>> > >>> > >>> > >>>On 1/26/16, 9:48 AM, "Eliot Kimber" > >>><basex-talk-bounces(a)mailman.uni-konstanz.de on behalf of > >>>ekimber(a)contrext.com> wrote: > >>> > >>>>OK, I'm going to proceed with making a generic basex Docker container > >>>>in > >>>>the DFST Docker project on GitHub. I'm using Andreas' as a starting > >>>>point > >>>>but it's pretty trivial. > >>>> > >>>>I'm working/testing on OS X, Windows, and CentOS so I should be able to > >>>>get something going. > >>>> > >>>>Cheers, > >>>> > >>>>E. > >>>>---- > >>>>Eliot Kimber, Owner > >>>>Contrext, LLC > >>>>http://contrext.com > >>>> > >>>> > >>>> > >>>> > >>>>On 1/26/16, 2:02 AM, "Christian Grün" <christian.gruen(a)gmail.com> > >>>>wrote: > >>>> > >>>>>Hi Eliot, > >>>>> > >>>>>Personally, I had a hard time making Docker work on Windows machines, > >>>>>but we have various Docker aficionados in our team and around, so I > >>>>>hope they¹ll give you some feedback soon. > >>>>> > >>>>>Cheers, > >>>>>Christian > >>>>> > >>>>> > >>>>>On Sun, Jan 24, 2016 at 8:22 PM, Eliot Kimber <ekimber(a)contrext.com> > >>>>>wrote: > >>>>>> I'm working toward using BaseX in a Docker container as part of the > >>>>>>DITA > >>>>>> for Small Teams project (we're trying to set up a system of Docker > >>>>>> containers with all the DFST parts integrated out of the box). > >>>>>> > >>>>>> I notice that Andreas Jung has create a simple container here: > >>>>>> > >>>>>> https://github.com/zopyx/docker-basex > >>>>>> > >>>>>> And pushed it to the Docker Hub in the basex namespace. > >>>>>> > >>>>>> I'm wondering what the relationship of this is, if any, to any > >>>>>>official > >>>>>> BaseX Docker support? > >>>>>> > >>>>>> Thanks, > >>>>>> > >>>>>> Eliot > >>>>>> ---- > >>>>>> Eliot Kimber, Owner > >>>>>> Contrext, LLC > >>>>>> http://contrext.com > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >>>> > >>>> > >>> > >>> > >>> > >> > >> > > > >

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk January 2016