- BaseX-Talk - mailman.uni-konstanz.de

Accessing basex from rest and from java api
by Henrique Oliveira 04 Jun '12

04 Jun '12

Hi All, Is possible to access BaseX from REST and Java API at the same time, same dbpath? I deployed BaseX.war in Jetty server, it is running on port 8080, I tried to connect using Java API without success. it seems is running on 1984 port too. INFO: General Information Version: 7.2.1 Used Memory: 2883 KB Main Options DBPATH: /private/var/folders/lb/_mgf_38n5nz05z_jbs3zkvbc0000gn/T/jetty-0.0.0.0-8080-BaseX721.war-_BaseX721-any-/webapp/data DEBUG: false EVENTPORT: 1985 HOST: localhost HTTPPATH: /private/var/folders/lb/_mgf_38n5nz05z_jbs3zkvbc0000gn/T/jetty-0.0.0.0-8080-BaseX721.war-_BaseX721-any-/webapp HTTPPORT: 8984 KEEPALIVE: 0 LANG: English LANGKEYS: false NONPROXYHOSTS: PARALLEL: 8 PORT: 1984 PROXYHOST: PROXYPORT: 80 REPOPATH: /private/var/folders/lb/_mgf_38n5nz05z_jbs3zkvbc0000gn/T/jetty-0.0.0.0-8080-BaseX721.war-_BaseX721-any-/webapp/repo SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 0 Options ADDARCHIVES: true ADDRAW: false ATTRINDEX: true AUTOFLUSH: true BINDINGS: CACHEQUERY: false CASESENS: false CATFILE: CHOP: true COMPPLAN: true CREATEFILTER: *.xml DIACRITICS: false DOTCOMPACT: false DOTDISPLAY: true DOTPLAN: false DOTTY: dotty DTD: false EXPORTER: FORCECREATE: false FTINDEX: false HTMLOPT: html=false,omit-xml-declaration=false,method=xml,nons=false,nobogons=false,nodefaults=false,nocolons=false,norestart=false,ignorable=false,emptybogons=false,any=false,norootbogons=false,nocdata=false,lexical=false,encoding=utf-8 INTPARSE: true LANGUAGE: en LSERROR: 0 MAINMEM: false MAXCATS: 100 MAXHITS: -1 MAXLEN: 96 MAXSTAT: 30 PARSER: xml PARSEROPT: encoding=UTF-8,flat=false,format=verbose,header=false,jsonml=false,lines=true,separator=comma PATHINDEX: true QUERYINFO: false QUERYPATH: RUNS: 1 SCORING: 0 SERIALIZE: true SERIALIZER: SKIPCORRUPT: false STEMMING: false STOPWORDS: TAILCALLS: 42 TEXTINDEX: true UPDINDEX: false WILDCARDS: false WRITEBACK: false XMLPLAN: false XQUERY3: true -- Henrique Oliveira

2 3

Re: [basex-talk] Deleting one of many resources at the same path (sorry for hoging the email distribution list)
by Anupam Bakshi 01 Jun '12

01 Jun '12

Hi Dimitar, Since collections are sequences, can I use [1] or [last()] to specify which one to delete? As per my requirement, I need to be able to delete older resources after some processing (data consolidation). Thanks for your help. Best regards, Anupam

4 4

How to retrieve same name resource
by Anupam Bakshi 01 Jun '12

01 Jun '12

Hello all, I have two resources at the same path in a database. XQUERY db:list("project1") Returns : resultset1.xml resultset1.xml The two resources have different contents. However, when I do the following, I get the XML content from the first file. XQUERY db:open("project1", "resultset1.xml") What is the way to get the second XML file? Thanks for your help. Anupam

2 4

Attribute indexing
by Alex Markin 01 Jun '12

01 Jun '12

Hello, BaseX team. I have a question about attribute indexing in basex. For example i have an xml file (in attachement). I execute the following sequence of commands: $ ./basex -Vc"SET ATTRINDEX true;SET UPDINDEX true;CREATE DB test" ATTRINDEX: ON UPDINDEX: ON Database 'test' created in 90.22 ms. $ ./basex -Vc"OPEN test;CREATE INDEX ATTRIBUTE;OPTIMIZE" Database 'test' was opened in 70.03 ms. Index 'ATTRIBUTE' created in 39.48 ms. Database 'test' was optimized in 31.14 ms. $ ./basex -Vq"index:attributes(\"test\")" <value count="70"> 0)(286, 1)[_ 4¦°䐪J'(*¤#(1, 1)</value> <value count="3">86, 1)[_ 4¦°䐪J'(*¤#(1, 1)(1, 12)-fi</value> <value count="1">/home/alex/tmp/erts/ntbuild.E</value> ... Out of Main Memory. There are no indices, given in first lines and some strange "Out of memory". What's wrong with this way of getting indices and how to fix it? Markin Alex

2 2

List of all resources in XQuery?
by Anupam Bakshi 31 May '12

31 May '12

Hello all, In XQuery I can use doc('resource_path') to open a resource from the baseX database but is there a way to list all available resources in the database? In other words, what is the equivalent of "LIST database_name" in XQuery? Thanks for your help. Anupam

2 2

Ask about the Context object!
by Truong An Nguyen 31 May '12

31 May '12

Hi, Can I use a Context object to create more than one database? For example: new CreateDB("db1","c:\db1").execute(this.Context); new CreateDB("db2","c:\db2").execute(this.Context); Thanks. Cheers, An

2 3

Problem with copy-namespaces declaration
by Hans-Juergen Rennau 30 May '12

30 May '12

Hello, I observed a little problem concerning the interpretation of a copy-namespaces declaration. More specifically, the value no-preserve is correctly observed if the source is constructed within the query, but not if the source is read from a file. Example query: declare copy-namespaces no-preserve, inherit; let $in := <data xmlns:r="http://example.com"><elem/></data> return <result>{$in/elem, doc('problem-doc.xml')//elem}</result> where "problem-doc.xml" contains exactly the same document as the query (<data xmlns:r="http://example.com"><elem/></data>) Output: <result> <elem/> <elem xmlns:r="http://example.com"/> </result> Kind regards, Hans-Juergen

2 1

Question about Servlet accessing BaseX
by Anupam Bakshi 29 May '12

29 May '12

Hello, I'm creating my first BaseX based app and have a few questions about the best possible way to architect it around BaseX. I need to use BaseX as a service in a Tomcat container on the same server machine. I have the following doubts: 1. I need to validate the XML coming into the database and do XSLT transformation. Should this be done in a servlet that connects to BaseX? 2. If so, should the servlet use REST too, or is there a better way for direct connection? 3. Since BaseX is running in Tomcat context, is it possible to access it using a local client on the server machine- for debug etc on the server machine? Thank you for your help. Best regards, Anupam

2 3

Who is using...?
by Christian Grün 28 May '12

28 May '12

Hi all, we are interested in your opinion! As one of our main focus is to keep BaseX small and cute (..sad but true: the BaseX core library already exceeds 2mb.. ;), we're currently contemplating the removal of two existing BaseX features, namely.. 1) the BaseXHTTP client mode (flag -c). Currently, we offer three operation modes: default, local, and client [1]. In client mode, all requests are supposed to be passed on to yet another BaseX server instance. Currently, however, WebDAV is the only service which is 100% compatible with this mode, while RESTXQ isn't tailored at all to handle remote database requests. Instead, we are contemplating on adding a light-weight XQuery Client API Module, which can then be used to explicitly address remote BaseX instances from within XQuery whenever databases are not stored locally anyway. 2) the current ZIP Module. The functionality won't disappear, though: we will replace it with a new, more flexible proposal specified by Matthias Branter (28msec) and me [2]. In a nutshell: who of you is using either 1) or 2) and would run into big troubles if those features were ousted and replaced with the approaches we believe are more promising for future development? Thanks as usual for your feedback, either to the list (preferred) or in private, Christian [1] http://docs.basex.org/wiki/Startup_Options#BaseX_HTTP_Server [2] http://spex.basex.org/index.php?title=ZIP_Module

6 8

bindings command in GUI not functioning as expected
by Brown, David (SSC-A) 28 May '12

28 May '12

Running 7.2.2 beta The standalone interface takes the following input: "set bindings $date=20120507, $envir=PRD_Win" and returns "bindings: $date=20120507, $envir=PRD_Win" The GUI interface on the command bar takes the following input: "set bindings $date=20120507" and returns "bindings: $date=20120507" "set bindings $date=20120507, $envir=PRD_Win" and returns "Error: Syntax: SET [option] ([value]) Set global options." This is a problem if you have more than one binding required. v/r, David Brown Fully Qualified Navy Certification Agent LSS Certified Blackbelt, CISSP, NSA-IAM/IEM, MSCIS SPAWARSYSCEN ATLANTIC ISS LOB SSC Lead Email: <mailto:david.k.brown1@navy.mil> david.k.brown1(a)navy.mil Phone: 843.218.6227 DSN: 588.6227 Fax: 843.218.5461 "Let us not look back in anger or forward in fear, but around in awareness." James Thurber

2 1

Connecting to BaseX running as a Tomcat servlet
by Anupam Bakshi 28 May '12

28 May '12

Hello, I'm using "net.xqj.basex.BaseXXQDataSource" to connect to a BaseX servlet running in Tomcat but it doesn't connect. I can run the following command : http://localhost:8080/BaseX721/rest?command=info Main Options DBPATH: C:\Apache Tomcat 6.0.20\webapps\BaseX721\\data DEBUG: false EVENTPORT: 1985 HOST: localhost HTTPPATH: C:\Apache Tomcat 6.0.20\webapps\BaseX721\ HTTPPORT: 8984 KEEPALIVE: 0 LANG: English LANGKEYS: false NONPROXYHOSTS: PARALLEL: 8 PORT: 1984 PROXYHOST: PROXYPORT: 80 REPOPATH: C:\Apache Tomcat 6.0.20\webapps\BaseX721\\repo SERVERHOST: SERVERPORT: 1984 STOPPORT: 8985 TIMEOUT: 0 But when I run : private static final String DRIVER = "net.xqj.basex.BaseXXQDataSource"; XQDataSource xqds = (XQDataSource) Class.forName(DRIVER).newInstance(); xqds.setProperty("serverName", "localhost"); xqds.setProperty("port", "1985"); I get : Exception in thread "main" javax.xml.xquery.XQException: XQJNC001 - Connection refused: connect Root Cause: java.net.ConnectException: Connection refused: connect Is this even possible? Any pointers will be greatly appreciated. Thanks, Anupam

2 1

error message
by Brown, David (SSC-A) 28 May '12

28 May '12

The first 8 characters of the root is a folder name. I want to group by folder name. I simplified the xquery to eliminate the collateral interactions, but still receive this error. p.s. I was able to get bindings working using quotes: open C4;set QUERYPATH c:\dev\xq;set bindings "$date=20120507, $envir=PRD_Win" Source scheme: <20120507/PRD_Win.nessus> <NessusClientData_v2> <Policy> <Report> <ReportHost> < ReportItem> ------------------- Query: declare variable $date external; <results> { for $x in //Report/ReportHost/ReportItem let $src := fn:root($x) return <summary> <source>substring(fn:name({$src}),1,8)</source> </summary> }</results> Query plan: <QueryPlan> <VarStack> <Var name="$date" id="0"> <Atm value="20120507" type="xs:untypedAtomic"/> </Var> <Var name="$envir" id="1"> <Atm value="PRD_Win" type="xs:untypedAtomic"/> </Var> </VarStack> <CElem> <QNm value="results" type="xs:QName"/> <FLWR> <For var="$x"> <AxisPath> <DBNodeSeq size="28"> <DBNode name="C4" pre="0"/> <DBNode name="C4" pre="21934"/> <DBNode name="C4" pre="74441"/> <DBNode name="C4" pre="121967"/> <DBNode name="C4" pre="145433"/> </DBNodeSeq> <IterStep axis="descendant" test="Report"/> <IterStep axis="child" test="ReportHost"/> <IterStep axis="child" test="ReportItem"/> </AxisPath> </For> <Let var="$src"> <FNNode name="root([node])"> <VarRef> <Var name="$x" id="3"/> </VarRef> </FNNode> </Let> <Return> <CElem> <QNm value="summary" type="xs:QName"/> <CElem> <QNm value="source" type="xs:QName"/> <Str value="substring(fn:name(" type="xs:string"/> <VarRef> <Var name="$src" id="4"/> </VarRef> <Str value="),1,8)" type="xs:string"/> </CElem> </CElem> </Return> </FLWR> </CElem> </QueryPlan> Error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 7.2.2 beta Java: Oracle Corporation, 1.7.0 OS: Windows 7, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException v/r, David Brown Fully Qualified Navy Certification Agent LSS Certified Blackbelt, CISSP, NSA-IAM/IEM, MSCIS SPAWARSYSCEN ATLANTIC ISS LOB SSC Lead Email: <mailto:david.k.brown1@navy.mil> david.k.brown1(a)navy.mil Phone: 843.218.6227 DSN: 588.6227 Fax: 843.218.5461 "Let us not look back in anger or forward in fear, but around in awareness." James Thurber

2 1

Output xquery to xml file
by Brown, David (SSC-A) 27 May '12

27 May '12

Sorry to ask such a basic question, but: I am using your product to read through a large set of xml files and output a simplified xml file. It works well manually, but would like to automate this process. I want all the query files to resolve to an output, each their own patterned file name. I have configured the query file to accept external variables, but as is stated in my previous message the bind function is not working for me just now. given that bind works I can use the variable values to create the output file name value. if I could get the query to create an xml file instead of output to console. v/r, David Brown Fully Qualified Navy Certification Agent LSS Certified Blackbelt, CISSP, NSA-IAM/IEM, MSCIS SPAWARSYSCEN ATLANTIC ISS LOB SSC Lead Email: <mailto:david.k.brown1@navy.mil> david.k.brown1(a)navy.mil Phone: 843.218.6227 DSN: 588.6227 Fax: 843.218.5461 "Let us not look back in anger or forward in fear, but around in awareness." James Thurber

2 1

XQuery Client API. WAS: whitespace handling
by Dimitar Popov 27 May '12

27 May '12

Am Freitag, 25. Mai 2012, 17:37:03 schrieb Christian Grün: > ..fertig ;) Zumindest mit Version 1; Tests are welcome: > > http://docs.basex.org/wiki/Client_Module > > > und ein paar weitere Punkte die man vlt. später umsetzen könnte: > > 1. Connection Registry: die Idee ist die Einzelheiten (Adresse, Username, > > Passwort) für eine Verbindung irgendwo zentral zu speichern und dann > > mittels eines Identifikators eine neue Verbindung zu erstellen, z.B. > > Richtig; momentan stört mich auch, dass das Passwort im Klartext in > den Queries steht. Fraglich bei der Registry-Lösung ist vermutlich, > wie man das am einfachsten umsetzt. Hast Du Ideen hierzu? > > Christian spontant würde ich sagen, irgendwo die Connection-Strings zu speichern (z.B. in einer Systemdatenbank - so könnte man ohne SSH oder Kommando- zeile BaseX einstellen... aber das ist ein anderes Thema). Man könnte auch die MD5 Summen statt die Passwörte speichern oder sogar an die Funktion angeben. Grüße, Dimitar

3 2

Who is using... ?
by Fabrice Etanchaud 26 May '12

26 May '12

Dear Christian, 1. Will BaseX clients still able to connect to the BaseX server serving HHTP requests ? This XQuery Client API Module is a very good idea ! 2. IMHO, BaseX should concentrate on processing xml, I stopped using ZIP module when I realized db:add() was doing the job, in streaming mode. Great job ! Best regards, Fabrice ________________________________ Hi all, we are interested in your opinion! As one of our main focus is to keep BaseX small and cute (..sad but true: the BaseX core library already exceeds 2mb.. ;), we're currently contemplating the removal of two existing BaseX features, namely.. 1) the BaseXHTTP client mode (flag -c). Currently, we offer three operation modes: default, local, and client [1]. In client mode, all requests are supposed to be passed on to yet another BaseX server instance. Currently, however, WebDAV is the only service which is 100% compatible with this mode, while RESTXQ isn't tailored at all to handle remote database requests. Instead, we are contemplating on adding a light-weight XQuery Client API Module, which can then be used to explicitly address remote BaseX instances from within XQuery whenever databases are not stored locally anyway. 2) the current ZIP Module. The functionality won't disappear, though: we will replace it with a new, more flexible proposal specified by Matthias Branter (28msec) and me [2]. In a nutshell: who of you is using either 1) or 2) and would run into big troubles if those features were ousted and replaced with the approaches we believe are more promising for future development? Thanks as usual for your feedback, either to the list (preferred) or in private, Christian [1] http://docs.basex.org/wiki/Startup_Options#BaseX_HTTP_Server [2] http://spex.basex.org/index.php?title=ZIP_Module Fabrice ETANCHAUD Senior Software Engineer edital Berkenlaan 1 B-1831 Brussels, Belgium <skype:fabrice.etanchaud?call> +32 2 716 32 32 general +32 2 716 32 20 fax fetanchaud(a)edital.com corsearch.com <http://www.corsearch.com/> | edital.com <http://www.edital.com/> <outbind://899-00000000E439C83B50BF17409D8C27B7AE0425AF07005AA868536216F 543BAB9422EAEA5338B00000011020900005AA868536216F543BAB9422EAEA5338B00000 13AA36B0000/cid:image001.gif@01CC0F02.2C91B6C0> Confidentiality Notice: This email and its attachments (if any) contain confidential information of the sender. The information is intended only for the use by the direct addressees of the original sender of this email. If you are not an intended recipient of the original sender (or responsible for delivering the message to such person), you are hereby notified that any review, disclosure, copying, distribution or the taking of any action in reliance of the contents of and attachments to this email is strictly prohibited. If you have received this email in error, please immediately notify the sender at the address shown herein and permanently delete any copies of this email (digital or paper) in your possession.

2 1

Re: [basex-talk] Too Many Open Files
by Andreas Rulle 26 May '12

26 May '12

Hello Andreas, thank you very much for your valuable hints. sudo lsof -p basex-ps-no delivers many lines of java 11489 root 962u sock 0,6 0t0 10449645 can't identify protocol and (1) sudo lsof -p 11489 | wc -l gets reports an increasing number of used "files". The increase of the number in (1) in a given time interval correlates to the number of requests that the LVM load balancer sends to the basex port 1984 in that interval. There are about 4 requests from the LVM load balancer per minute. At the time of this writing (1) has the value of 1076 ... The parameter keepalive has the value KEEPALIVE = 600 but it does not seem to stop the increase of the value in (1). This information opens the way to workarounds - decrease the number of requests from the LVM load balancer, - increase the ulimit -n - restart the basex -server before the number in one reaches ulimt -n. But we really would prefer solutions to the increase of the figure in (1) The almost identical settings of the load balancer do work for a MySQL-Database without any problems. And without the load balancer an instance of the basex runs since April 10 without hitting ulimit -n = 1024. Any hints on this are very welcome! With kind regards, Andreas > I just searched our mailing list, cause there was something about too many open files some time ago: > >> On typical Linux installations, die open filedescriptor limit is 1024: >> >> $ grep 'open files' /proc/self/limits >> Max open files 1024 1024 files >> >> In Java, if a file-based object (FileWriter, FileReader, etc.) is not closed, the underlying file descriptor is not closed. Bug detectors (FindBugs) check for that intraprocedurally. I recommend you run both BaseX as well as this application through it. >> >> If an open file-based object becomes unreachable, the finalizer will eventually close it - but it's possible to run out of open file descriptors simply due to unreachable, but not yet finalized objects. (Of course, if the object is leaked, it won't be closed ever.) >> >> In Linux, use 'ps' to find out the pid of the Java JVM process, then do a ls -l /proc/<pid>/fds to see which file descriptors the process in question has open; or use the 'lsof' command. >> >> - Godmar > Probably that helps to investigate the issue. > > -- Nexoma GmbH Theodorus Weg 7 59755 Arnsberg Tel. + 49 (0) 52 51 1613-0 Fax + 49 (0) 52 51 1613-99 mailto:andreas.rulle@nexoma.de Geschäftsführer: Guido Sauerland Sitz der Gesellschaft: Arnsberg Registergericht: Arnsberg, HRB 9365

6 11

XML file with xmlns attribute and XPATH
by Philippe Rathé 25 May '12

25 May '12

Hello there, I just want to do a simple xpath expression but I can't get it works when I use this XML document <?xml version="1.0" encoding='utf-8'?> <feed xmlns="http://example.org" xmlns:abc="http://abc.example.org"> <title>Projects</title> </feed> The expression "exquery //title" does not return anything from the CLI once I open the database containing only that file. But it works with this file <?xml version="1.0" encoding='utf-8'?> <feed xmlns="http://example.org"> <title>Projects</title> </feed> It looks like having both xmlns and xmlns:abc namespace attribute creates a problem. My real use case is atompub feed document which the 2 first lines looks like this: <?xml version="1.0" encoding='utf-8'?> <feed xmlns="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app"> Any idea? I'm using a 7.2.2 snapshot, thanks.

3 6

whitespace handling
by Godmar Back 25 May '12

25 May '12

Hi, I'm confused about whitespace. I've written an XQuery that returns an XML fragment based on some computation on an underlying XML document. The returned XML contains insignificant whitespace, which then adversely affects my program. I'm not sure if I'm causing it or BaseX, or even what the rules are. My XQuery contains code such as this one (I'm using color to emphasize the relevant parts). declare function local:buildSubTree($id) { let $feed := doc($doc_name)/atom:feed let $parent := $feed/atom:entry[atom:id=$id] let $children := data($parent//libx:entry/@src) return if (fn:count($children) = 0) then () else for $child_id in $children return if (local:isInThisFeed($feed, $child_id)) then <node id='{$child_id}' type='{local:getType($feed, $child_id)}' title='{local:getTitle($feed, $child_id)}'>{ local:buildSubTree($child_id) }</node> else if (functx:is-absolute-uri($child_id)) then <node id='{$child_id}' title='Unpublished Entry' type='external' /> else <node id='{$child_id}' title='Unpublished Entry' type='import' /> }; but the result contains fragments such as this, which is an 2-space indented formatting of the XML, with insignificant whitespace added: <node id="180" title="Put Google Book results into Addison" type="libapp"> <node id="181" title="Search Google Books while searching Addison" type="module"/> <node id="182" title="Display HTML notifications via an embedded panel using jGrowl" type="module"/> </node> If I say <node>{ ... }</node> in the XQuery etc., should there be insignificant whitespace in the response? Thank you for any help/pointers. (Note that I have considered simply stripping insignificant whitespace, but I do not like this solution since for some queries, I'd like to preserve it, whereas for others I absolutely cannot have insignificant whitespace since I'm performing traversals of the resulting XML DOM Tree.) Thank you! - Godmar

2 1

file:list - a question
by Hans-Juergen Rennau 24 May '12

24 May '12

Hello, a question about the function file:list. The documenation says: "The returned paths are relative to the provided path." My understanding is that if the path is a/b/c and this dir contains a file fool.xml, the returned path would be "fool.xml", rather thatn "a/b/c/fool.xml". What confuses me is that I observe two different behaviours in two different BaseX versions, once with and once without path: version 7.2 with path, version 7.2.2 without path. (The OS also differs in the environments, but I suppose this is irrelevant.) My question: is the output without path (more generally: with a path relative to the stated dir) the correct one and will be produced in all future versions? Thank you. Kind regards, Hans-Juergen

2 1

Managing databases via XQuery
by Andy Bunce 24 May '12

24 May '12

I sometimes find myself, in code, wanting to create a database, or more specifically test if a database exists and create it if does not. The database module appears not have functions to create (or delete databases). Is this a deliberate design decision? I can do this via the command interface, although test and create is a bit messy; I can do it via the REST interface, and I think via the WebDAV interface, but not, it seems, via XQuery directly. Adding additional functions to the database module would be a solution, or maybe a util:command() that would execute any BaseX command? /Andy

3 4

RESTXQ and parameters in the media-type
by Philippe Rathé 22 May '12

22 May '12

Hello, I would like to know if the parameters in the media-type should be in the "produces" annotation or elsewhere. Because I can't get it to work with the media-type of an atom feed member. %rest:produces("application/atom+xml;type=entry") The valid syntax can be found in rfc2616 section 3.7 Media Types: http://tools.ietf.org/html/rfc2616#section-3.7 In the RESTXQ wiki page, we can see the media type of an atome feed %rest:produces("application/atom+xml") But I did not found anything about using a parameter in the media type. Thanks.

2 3

Open Source-based Business
by Yu, Yiqing 22 May '12

22 May '12

Hi open source software developer, I am a research assistant at Institute for Information Systems and New Media, University of Munich. Driven by the interest in open source software, I want to investigate open source-based business by connecting the decision-making theory to the software development practice. Based on this idea, I designed an online survey "OS-based business": http://www.unipark.de/uc/osb/ Through "advanced search" and your profile at Github I find you may fit into the survey's target group. I gladly invite you to it and look forward to your participation. Some points in the survey may inspire your research or business ideas. :) I would deeply appreciate your support! Best Regards, Yours, Yiqing Yu Institute for Information Systems and New Media Ludwig-Maximilians-Universität München www.wim.bwl.lmu.de<http://www.wim.bwl.lmu.de/>

1 0

Disable or control query caching
by Manuel Bernhardt 22 May '12

22 May '12

Hello, we're getting an apparent deadlock (followed by a GC overhead limit exceeded) on one machine, when starting some processing on a collection of over 800 000 records. Going after it with YourKit yields the following: application-akka.actor.default-dispatcher-110 <--- Frozen for at least 3m 7s org.basex.server.Query.cache(InputStream) org.basex.server.ClientQuery.cache() org.basex.server.Query.more() eu.delving.basex.client.Implicits$RichClientQuery.hasNext() scala.collection.Iterator$$anon$19.hasNext() scala.collection.Iterator$$anon$29.hasNext() scala.collection.Iterator$class.foreach(Iterator, Function1) scala.collection.Iterator$$anon$29.foreach(Function1) core.processing.CollectionProcessor$$anonfun$process$2.apply(ClientSession) core.processing.CollectionProcessor$$anonfun$process$2.apply(Object) core.storage.BaseXStorage$$anonfun$withSession$1.apply(ClientSession) core.storage.BaseXStorage$$anonfun$withSession$1.apply(Object) eu.delving.basex.client.BaseX$$anonfun$withSession$1.apply(ClientSession) eu.delving.basex.client.BaseX$$anonfun$withSession$1.apply(Object) eu.delving.basex.client.BaseX.withSession(Function1) eu.delving.basex.client.BaseX.withSession(String, Function1) core.storage.BaseXStorage$.withSession(Collection, Function1) core.processing.CollectionProcessor.process(Function0, Function1, Function1, Function3) core.processing.DataSetCollectionProcessor$.process(DataSet) actors.Processor$$anonfun$receive$1.apply(Object)<2 recursive calls> akka.actor.Actor$class.apply(Actor, Object) actors.Processor.apply(Object) akka.actor.ActorCell.invoke(Envelope) akka.dispatch.Mailbox.processMailbox(int, long) akka.dispatch.Mailbox.run() akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec() akka.jsr166y.ForkJoinTask.doExec() akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinTask) akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool$WorkQueue) akka.jsr166y.ForkJoinWorkerThread.run() In the server logs I can observe: 09:10:45.129 [192.168.1.214:47530]: dimcon____geheugen-van-nederland QUERY(3) for $i in /record[@version = 0] order by $i/system/index return $i OK 0.06 ms 09:10:45.129 [192.168.1.214:47530]: dimcon____geheugen-van-nederland QUERY(3) OK 0.03 ms 09:13:23.155 [192.168.1.214:47530]: dimcon____geheugen-van-nederland ITER(3) Error: Connection reset 09:13:23.155 [192.168.1.214:47530]: dimcon____geheugen-van-nederland LOGOUT admin OK I looked up the code, and it looks as though the whole (?) query is cached in memory upon retrieval. Given the DB is over 1.2 GB in size, our client server has a hard time (it only has 1,5 GB of Xmx). Is there any preferred way of dealing with this? What I am going to do for the moment, I think, is to override or intercept the creation of the ClientQuery and make an implementation that has a different caching strategy. Another approach may probably be to limit the query output and implement some custom iteration behavior - but if it that can be handled directly at the query level, I think it would make things easier. Manuel

2 6

db:add and ADD logic
by Fabrice Etanchaud 22 May '12

22 May '12

Dear all at BaseX, I don't understand the ADD/db:add behaviour when adding two documents with the same name : previous document is not replaced, but copied, and I did not find a way to correct, and delete only one of the documents, because they all have the same name... So, thanks to the db:replace function, I wrote a updating function which emulates desired behaviour (that adds or replaces the document if it already exists in the database). But if I try to insert the same document twice with this function (in the same updating query), I still get two documents in the database, because function is 'updating' and because of the pending update list... Are db:XXX functions really 'updating' ? Is there a way to delete only one of the duplicated documents ? Thank you again for your fantastic job. Regards, Fabrice ETANCHAUD Senior Software Engineer edital Berkenlaan 1 B-1831 Brussels, Belgium <skype:fabrice.etanchaud?call> +32 2 716 32 32 general +32 2 716 32 20 fax fetanchaud(a)edital.com corsearch.com <http://www.corsearch.com/> | edital.com <http://www.edital.com/> <outbind://899-00000000E439C83B50BF17409D8C27B7AE0425AF07005AA868536216F 543BAB9422EAEA5338B00000011020900005AA868536216F543BAB9422EAEA5338B00000 13AA36B0000/cid:image001.gif@01CC0F02.2C91B6C0> Confidentiality Notice: This email and its attachments (if any) contain confidential information of the sender. The information is intended only for the use by the direct addressees of the original sender of this email. If you are not an intended recipient of the original sender (or responsible for delivering the message to such person), you are hereby notified that any review, disclosure, copying, distribution or the taking of any action in reliance of the contents of and attachments to this email is strictly prohibited. If you have received this email in error, please immediately notify the sender at the address shown herein and permanently delete any copies of this email (digital or paper) in your possession.

2 1

Returned mail: see transcript for details
by sandeep＠marlabs.com 21 May '12

21 May '12

The original message was received at Mon, 21 May 2012 11:27:17 -0500 from [65.1.203.108] ----- The following addresses had permanent fatal errors ----- basex-talk(a)mailman.uni-konstanz.de

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk