BaseX-Talk

basex-talk@mailman.uni-konstanz.de

1 participants
5252 discussions

should BXCollection.listResources always throw an exception?
by Meredith Gregory 17 Apr '11

17 Apr '11

Dear BaseXers, So far, BaseX has been a lot of fun to work with! Thanks for all your efforts on this codebase. i've got a couple of questions, though. - Whether i've got no documents in a freshly created BXCollection, or i've ensured that there is at least one document present, a call to listResources always throws an IndexOutOfBounds exception. Is that the intended behavior? - The reason i'm using this API is that i need to be able to update a single document in a collection guaranteed to have exactly one document. So, i thought i would simply ask the BXCollection to list it's resources, grab the first one (since there should only be one) and then use the id from the list returned by that API in subsequent queries. Maybe that's not the preferred approach? Is there a better one? Any help would be greatly appreciated. Best wishes, --greg override def getCollection( createIfMissing : Boolean )( xmlCollStr : String ) : Option[Collection] = { try { // BUGBUG -- LGM the semantics of BXCollection a little // different than createIfMissing; and requires catching the // database not found exception Some( new BXCollection( xmlCollStr, true ) ) } catch { case e : XMLDBException => { val bxColl = new BXCollection( xmlCollStr, false ) val document = bxColl.createResource( null, XMLResource.RESOURCE_TYPE ).asInstanceOf[XMLResource] document.setContent( "<database></database>" ) bxColl.storeResource( document ) Some( bxColl ) } case _ => None } } override def update( xmlCollStr : String )( cnxn : CnxnCtxtLabel[Namespace,Var,String] ) : Unit = { for( xmlColl <- getCollection( true )( xmlCollStr ) ) { val resourceList : Array[String] = xmlColl.listResources if ( resourceList.size == 0 ) { val document = xmlColl.createResource( null, XMLResource.RESOURCE_TYPE ).asInstanceOf[XMLResource] document.setContent( xmlIfier.asXML( cnxn ).toString ) xmlColl.storeResource( document ) } else { val cnxnElem = xmlIfier.asXML( cnxn ) cnxnElem match { case cElem : Elem => { delete( xmlCollStr, resourceList( 0 ) )( cElem ) insert( xmlCollStr, resourceList( 0 ) )( cElem ) } case _ => { throw new Exception( "record is not an XML element" ) } } } } } <tweet>storing to db : org.basex.api.xmldb.BXDatabase@77dc754 pair : record(contactMeUsing(email), {"com.biosimilarity.lift.model.store.MonadicTermTypes_-Ground":{"v":{"@class":"string","$":"for the win Mikail.Mauz@gmail.com"},"_-outer":{"@class":"com.biosimilarity.lift.model.store.PersistedMonadicTS$TheMTT$"}}}) in coll : Squibble</tweet> java.lang.IndexOutOfBoundsException at java.io.RandomAccessFile.readBytes(Native Method) at java.io.RandomAccessFile.read(RandomAccessFile.java:322) at java.io.RandomAccessFile.readFully(RandomAccessFile.java:381) at org.basex.io.DataAccess.cursor(DataAccess.java:225) at org.basex.io.DataAccess.readToken(DataAccess.java:147) at org.basex.data.DiskData.txt(DiskData.java:213) at org.basex.data.DiskData.text(DiskData.java:181) at org.basex.api.xmldb.BXCollection.listResources(BXCollection.java:128) at com.biosimilarity.lift.model.store.xml.BaseXCnxnStorage$$anonfun$update$1.apply(BaseXXMLPersist.scala:313) at com.biosimilarity.lift.model.store.xml.BaseXCnxnStorage$$anonfun$update$1.apply(BaseXXMLPersist.scala:312) at scala.Option.foreach(Option.scala:185) at com.biosimilarity.lift.model.store.xml.BaseXCnxnStorage$class.update(BaseXXMLPersist.scala:312) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction.update(PersistedMonadicTermStore.scala:321) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction$$anonfun$putInStore$3$$anonfun$apply$12$$anonfun$apply$13.apply(PersistedMonadicTermStore.scala:468) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction$$anonfun$putInStore$3$$anonfun$apply$12$$anonfun$apply$13.apply(PersistedMonadicTermStore.scala:459) at scala.Option.foreach(Option.scala:185) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction$$anonfun$putInStore$3$$anonfun$apply$12.apply(PersistedMonadicTermStore.scala:459) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction$$anonfun$putInStore$3$$anonfun$apply$12.apply(PersistedMonadicTermStore.scala:458) at scala.Option.foreach(Option.scala:185) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction$$anonfun$putInStore$3.apply(PersistedMonadicTermStore.scala:458) at com.biosimilarity.lift.model.store.PersistedTermStoreScope$PersistedMonadicGeneratorJunction$$anonfun$putInStore$3.apply(PersistedMonadicTermStore.scala:458) at scala.util.continuations.package$.run(package.scala:30) at scala.concurrent.cpsops$$anonfun$spawn$1.apply$mcV$sp(cpsops.scala:9) at scala.concurrent.FJTaskRunners$FJTaskRunner$$anon$1.compute(FJTaskRunners.scala:21) at jsr166y.forkjoin.RecursiveAction.exec(RecursiveAction.java:247) at jsr166y.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:286) -- L.G. Meredith Managing Partner Biosimilarity LLC 7329 39th Ave SW Seattle, WA 98136 +1 206.650.3740 http://biosimilarity.blogspot.com

2 6

Can not generate fulltext search indexes for wikipedia XML dB
by Erol Akarsu 14 Apr '11

14 Apr '11

Ok, I was able to run full text search with another XML Database. I am primarily interested in how Basex will play with big XML file Wikipedia. Actually, Database create of wikipedia is fine. But when I add full tet search and indexes for it, it always throws out of memory exception error. I have changed -Xmx with 6GB that is still not enough to generate indexes for Wikipedia. Can you help me on how to generate indexes with a machine that case 6GB for Basex process? Thanks Erol Akarsu On Sun, Apr 10, 2011 at 4:07 AM, Andreas Weiler < andreas.weiler(a)uni-konstanz.de> wrote: > Hi, > > the following query could work for you: > > declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; > for $i in doc("enwiki-latest-pages-articles")//w:sitename > return $i[. contains text "Wikipedia"]/.. > > -- Andreas > > Am 09.04.2011 um 20:43 schrieb Erol Akarsu: > > Hi > > I am having difficulty in running full text operators. This script gives > siteinfo below > declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; > let $d := fn:doc ("enwiki-latest-pages-articles") > return ($d//w:siteinfo)[1] > > But return $d//w:siteinfo[w:sitename contains text 'Wikipedia'] does NOT > give same node > Why "contains" ft operator behave incorrectly? I remember it was working > fine. I just dropped and recreated database and turn all indexes. Can you > help me? > Query info is here: > > Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; > Compiling: > - pre-evaluating fn:doc("enwiki-latest-pages-articles") > - adding text() step > - optimizing descendant-or-self step(s) > - removing path with no index results > - pre-evaluating (())[1] > - binding static variable $res > - pre-evaluating fn:doc("enwiki-latest-pages-articles") > - binding static variable $d > - adding text() step > - optimizing descendant-or-self step(s) > - removing path with no index results > - simplifying flwor expression > Result: () > Timing: > - Parsing: 0.46 ms > - Compiling: 0.42 ms > - Evaluating: 0.17 ms > - Printing: 0.1 ms > - Total Time: 1.15 ms > Query plan: > <sequence size="0"/> > > > > <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" > http://www.w3.org/2001/XMLSchema-instance"> > <sitename>Wikipedia</sitename> > <base>http://en.wikipedia.org/wiki/Main_Page</base> > <generator>MediaWiki 1.17wmf1</generator> > <case>first-letter</case> > <namespaces> > <namespace key="-2" case="first-letter">Media</namespace> > <namespace key="-1" case="first-letter">Special</namespace> > <namespace key="0" case="first-letter"/> > <namespace key="1" case="first-letter">Talk</namespace> > <namespace key="2" case="first-letter">User</namespace> > <namespace key="3" case="first-letter">User talk</namespace> > <namespace key="4" case="first-letter">Wikipedia</namespace> > <namespace key="5" case="first-letter">Wikipedia talk</namespace> > <namespace key="6" case="first-letter">File</namespace> > <namespace key="7" case="first-letter">File talk</namespace> > <namespace key="8" case="first-letter">MediaWiki</namespace> > <namespace key="9" case="first-letter">MediaWiki talk</namespace> > <namespace key="10" case="first-letter">Template</namespace> > <namespace key="11" case="first-letter">Template talk</namespace> > <namespace key="12" case="first-letter">Help</namespace> > <namespace key="13" case="first-letter">Help talk</namespace> > <namespace key="14" case="first-letter">Category</namespace> > <namespace key="15" case="first-letter">Category talk</namespace> > <namespace key="100" case="first-letter">Portal</namespace> > <namespace key="101" case="first-letter">Portal talk</namespace> > <namespace key="108" case="first-letter">Book</namespace> > <namespace key="109" case="first-letter">Book talk</namespace> > </namespaces> > </siteinfo> > > On Mon, Apr 4, 2011 at 7:31 AM, Erol Akarsu <eakarsu(a)gmail.com> wrote: > >> I imported wikipedia xml into basex and tried to search it. >> >> But searching it takes longer. >> >> I tried to search one element that is first child of whole document and it >> took 52 sec. >> I know the XML file is very big 31GB. How can I optimize the search? >> >> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> >> let $d := fn:doc ("enwiki-latest-pages-articles")//w:siteinfo >> return $d >> >> Database info: >> >> > open enwiki-latest-pages-articles >> Database 'enwiki-latest-pages-articles' opened in 778.49 ms. >> > info database >> Database Properties >> Name: enwiki-latest-pages-articles >> Size: 23356 MB >> Nodes: 228090153 >> Height: 6 >> >> Database Creation >> Path: /mnt/hgfs/C/tmp/enwiki-latest-pages-articles.xml >> Time Stamp: 03.04.2011 12:29:15 >> Input Size: 30025 MB >> Encoding: UTF-8 >> Documents: 1 >> Whitespace Chopping: ON >> Entity Parsing: OFF >> >> Indexes >> Up-to-date: true >> Path Summary: ON >> Text Index: ON >> Attribute Index: ON >> Full-Text Index: OFF >> > >> >> >> Timing info: >> >> Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> Compiling: >> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >> - optimizing descendant-or-self step(s) >> - binding static variable $d >> - removing variable $d >> - simplifying flwor expression >> Result: element siteinfo { ... } >> Timing: >> - Parsing: 1.4 ms >> - Compiling: 52599.0 ms >> - Evaluating: 0.28 ms >> - Printing: 0.62 ms >> - Total Time: 52601.32 ms >> Query plan: >> <DBNode name="enwiki-latest-pages-articles" pre="5"/> >> >> >> >> Result of query: >> >> <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" >> http://www.w3.org/2001/XMLSchema-instance"> >> <sitename>Wikipedia</sitename> >> <base>http://en.wikipedia.org/wiki/Main_Page</base> >> <generator>MediaWiki 1.17wmf1</generator> >> <case>first-letter</case> >> <namespaces> >> <namespace key="-2" case="first-letter">Media</namespace> >> <namespace key="-1" case="first-letter">Special</namespace> >> <namespace key="0" case="first-letter"/> >> <namespace key="1" case="first-letter">Talk</namespace> >> <namespace key="2" case="first-letter">User</namespace> >> <namespace key="3" case="first-letter">User talk</namespace> >> <namespace key="4" case="first-letter">Wikipedia</namespace> >> <namespace key="5" case="first-letter">Wikipedia talk</namespace> >> <namespace key="6" case="first-letter">File</namespace> >> <namespace key="7" case="first-letter">File talk</namespace> >> <namespace key="8" case="first-letter">MediaWiki</namespace> >> <namespace key="9" case="first-letter">MediaWiki talk</namespace> >> <namespace key="10" case="first-letter">Template</namespace> >> <namespace key="11" case="first-letter">Template talk</namespace> >> <namespace key="12" case="first-letter">Help</namespace> >> <namespace key="13" case="first-letter">Help talk</namespace> >> <namespace key="14" case="first-letter">Category</namespace> >> <namespace key="15" case="first-letter">Category talk</namespace> >> <namespace key="100" case="first-letter">Portal</namespace> >> <namespace key="101" case="first-letter">Portal talk</namespace> >> <namespace key="108" case="first-letter">Book</namespace> >> <namespace key="109" case="first-letter">Book talk</namespace> >> </namespaces> >> </siteinfo> >> >> >> >> Thanks >> >> Erol Akarsu >> >> > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > >

2 3

Re: [basex-talk] lemmatisation
by Christian Grün 14 Apr '11

14 Apr '11

Dear Xavier, thanks for your request; due to time constraints, I'm passing this on to the list.. Hope someone helps; otherwise, please ask again, Christian ___________________________ On Thu, Apr 14, 2011 at 11:49 AM, xavier-laurent salvador <xavier-laurent.salvador(a)univ-paris13.fr> wrote: > Sorry for the send, here is my right xquery routine: > > <texte> { > for $dic in db:open('dictionary')//entry[pos contains text "Advb." ftor > "Conj." any word]/orth return > <res> > > Resultat: {$dic}: > { > let $s:= ft:mark(//*[text() contains text {$dic} using case > insensitive using wildcards], 'advb') > let $sv := $s//advb > return <ff>{$s}</ff>} > </res> > } > </texte> > > > Hello again, > could you give me an advice, please ? > I'm trying to integrate basex in a lexicographic project. Server an >> bindings are useful and ok. >> What i would like to do is crossing the datas of a db (="dictionary") >> with the tokens of another db(="text"): for all the adverbs of my >> dictionary, i want a mark around the token in the text. >> What i get is a new instance of my text, with <mark/> around the lemma, >> for every adverbs of the dictionary. 15 adverbs, 15 times the text. >> >> 'dictionary is <entry><orth>xxx</orth><pos>Advb.</pos></entry> >> 'text' is <root><texte><localisation>1</localisation>Donc dist alors li >> heros a ses amis</texte></root> >> >> here is my routine ('text' is already db:open): >> >> <texte> { >> for $dic in db:open('dictionary')//entry[pos contains text "Advb." ftor >> "Conj." any word]/orth return >> <res> >> >> Resultat: {$dic}: >> { >> let $s:= ft:mark(//*[text() contains text {$dic} using case >> insensitive using wildcards], 'advb') >> let $sv := $s//advb >> return <ff>{$s}</ff>} >> </res> >> } >> </texte> >> >> I tried with tumbling window, but couldn't with basex. >> Do you have an advice, please ? >> >> Thank you by advance and congratulations again for your great work, >> >> xls >> >> > > > >

1 0

Possible bug ?
by xavier-laurent salvador 14 Apr '11

14 Apr '11

Hi, here is a feedback for a "possible bug". I was trying to ft:mark with a "\s" wildcard. Hope I helped, Version: BaseX 6.6.1 Java: Sun Microsystems Inc., 1.6.0_18 OS: Linux, i386 Stack Trace: java.lang.NegativeArraySizeException org.basex.util.ft.WesternTokenizer.orig(WesternTokenizer.java:201) org.basex.util.ft.WesternTokenizer.get(WesternTokenizer.java:187) org.basex.util.ft.WesternTokenizer.nextToken(WesternTokenizer.java:113) org.basex.util.ft.FTLexer.nextToken(FTLexer.java:116) org.basex.query.ft.FTTokenizer.cache(FTTokenizer.java:105) org.basex.query.ft.FTWords.contains(FTWords.java:208) org.basex.query.ft.FTWords.item(FTWords.java:130) org.basex.query.ft.FTContains.item(FTContains.java:75) org.basex.query.ft.FTContains.item(FTContains.java:1) org.basex.query.expr.ParseExpr.ebv(ParseExpr.java:85) org.basex.query.expr.ParseExpr.test(ParseExpr.java:98) org.basex.query.expr.Preds.preds(Preds.java:99) org.basex.query.path.IterStep$1.next(IterStep.java:49) org.basex.query.iter.NodeIter.next(NodeIter.java:1) org.basex.query.path.IterPath$1.next(IterPath.java:67) org.basex.query.iter.NodeIter.next(NodeIter.java:1) org.basex.query.func.FNFt$1.next(FNFt.java:97) org.basex.query.iter.Iter.finish(Iter.java:65) org.basex.query.expr.ParseExpr.value(ParseExpr.java:76) org.basex.query.expr.Let$1.next(Let.java:94) org.basex.query.expr.FLWR$1.next(FLWR.java:62) org.basex.query.expr.Constr.<init>(Constr.java:55) org.basex.query.expr.CElem.item(CElem.java:116) org.basex.query.expr.CElem.item(CElem.java:1) org.basex.query.expr.CFrag.item(CFrag.java:1) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:52) org.basex.query.QueryContext.iter(QueryContext.java:329) org.basex.query.expr.Constr.<init>(Constr.java:54) org.basex.query.expr.CElem.item(CElem.java:116) org.basex.query.expr.CElem.item(CElem.java:1) org.basex.query.expr.CFrag.item(CFrag.java:1) org.basex.query.expr.ParseExpr.iter(ParseExpr.java:52) org.basex.query.QueryContext.iter(QueryContext.java:329) org.basex.query.QueryContext.iter(QueryContext.java:294) org.basex.query.QueryContext.eval(QueryContext.java:254) org.basex.query.QueryProcessor.execute(QueryProcessor.java:113) org.basex.core.cmd.AQuery.query(AQuery.java:83) org.basex.core.cmd.XQuery.run(XQuery.java:22) org.basex.core.Command.run(Command.java:292) org.basex.core.Command.exec(Command.java:274) org.basex.core.Command.execute(Command.java:67) org.basex.gui.GUI.exec(GUI.java:390) org.basex.gui.GUI$5.run(GUI.java:351)

3 2

Apache Lucene for full text search ?
by Erol Akarsu 13 Apr '11

13 Apr '11

I have another question. Does Basex use Apache Lucene for full text search or have its own search engine? Thanks On Mon, Apr 11, 2011 at 12:35 PM, Erol Akarsu <eakarsu(a)gmail.com> wrote: > Ok, > > I was able to run full text search with another XML Database. > > I am primarily interested in how Basex will play with big XML file > Wikipedia. > > Actually, Database create of wikipedia is fine. But when I add full tet > search and indexes for it, it always throws out of memory exception error. > > I have changed -Xmx with 6GB that is still not enough to generate indexes > for Wikipedia. > > Can you help me on how to generate indexes with a machine that case 6GB > for Basex process? > > Thanks > > Erol Akarsu > > > > On Sun, Apr 10, 2011 at 4:07 AM, Andreas Weiler < > andreas.weiler(a)uni-konstanz.de> wrote: > >> Hi, >> >> the following query could work for you: >> >> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> for $i in doc("enwiki-latest-pages-articles")//w:sitename >> return $i[. contains text "Wikipedia"]/.. >> >> -- Andreas >> >> Am 09.04.2011 um 20:43 schrieb Erol Akarsu: >> >> Hi >> >> I am having difficulty in running full text operators. This script gives >> siteinfo below >> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> let $d := fn:doc ("enwiki-latest-pages-articles") >> return ($d//w:siteinfo)[1] >> >> But return $d//w:siteinfo[w:sitename contains text 'Wikipedia'] does NOT >> give same node >> Why "contains" ft operator behave incorrectly? I remember it was working >> fine. I just dropped and recreated database and turn all indexes. Can you >> help me? >> Query info is here: >> >> Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >> Compiling: >> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >> - adding text() step >> - optimizing descendant-or-self step(s) >> - removing path with no index results >> - pre-evaluating (())[1] >> - binding static variable $res >> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >> - binding static variable $d >> - adding text() step >> - optimizing descendant-or-self step(s) >> - removing path with no index results >> - simplifying flwor expression >> Result: () >> Timing: >> - Parsing: 0.46 ms >> - Compiling: 0.42 ms >> - Evaluating: 0.17 ms >> - Printing: 0.1 ms >> - Total Time: 1.15 ms >> Query plan: >> <sequence size="0"/> >> >> >> >> <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" >> http://www.w3.org/2001/XMLSchema-instance"> >> <sitename>Wikipedia</sitename> >> <base>http://en.wikipedia.org/wiki/Main_Page</base> >> <generator>MediaWiki 1.17wmf1</generator> >> <case>first-letter</case> >> <namespaces> >> <namespace key="-2" case="first-letter">Media</namespace> >> <namespace key="-1" case="first-letter">Special</namespace> >> <namespace key="0" case="first-letter"/> >> <namespace key="1" case="first-letter">Talk</namespace> >> <namespace key="2" case="first-letter">User</namespace> >> <namespace key="3" case="first-letter">User talk</namespace> >> <namespace key="4" case="first-letter">Wikipedia</namespace> >> <namespace key="5" case="first-letter">Wikipedia talk</namespace> >> <namespace key="6" case="first-letter">File</namespace> >> <namespace key="7" case="first-letter">File talk</namespace> >> <namespace key="8" case="first-letter">MediaWiki</namespace> >> <namespace key="9" case="first-letter">MediaWiki talk</namespace> >> <namespace key="10" case="first-letter">Template</namespace> >> <namespace key="11" case="first-letter">Template talk</namespace> >> <namespace key="12" case="first-letter">Help</namespace> >> <namespace key="13" case="first-letter">Help talk</namespace> >> <namespace key="14" case="first-letter">Category</namespace> >> <namespace key="15" case="first-letter">Category talk</namespace> >> <namespace key="100" case="first-letter">Portal</namespace> >> <namespace key="101" case="first-letter">Portal talk</namespace> >> <namespace key="108" case="first-letter">Book</namespace> >> <namespace key="109" case="first-letter">Book talk</namespace> >> </namespaces> >> </siteinfo> >> >> On Mon, Apr 4, 2011 at 7:31 AM, Erol Akarsu <eakarsu(a)gmail.com> wrote: >> >>> I imported wikipedia xml into basex and tried to search it. >>> >>> But searching it takes longer. >>> >>> I tried to search one element that is first child of whole document and >>> it took 52 sec. >>> I know the XML file is very big 31GB. How can I optimize the search? >>> >>> declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >>> >>> let $d := fn:doc ("enwiki-latest-pages-articles")//w:siteinfo >>> return $d >>> >>> Database info: >>> >>> > open enwiki-latest-pages-articles >>> Database 'enwiki-latest-pages-articles' opened in 778.49 ms. >>> > info database >>> Database Properties >>> Name: enwiki-latest-pages-articles >>> Size: 23356 MB >>> Nodes: 228090153 >>> Height: 6 >>> >>> Database Creation >>> Path: /mnt/hgfs/C/tmp/enwiki-latest-pages-articles.xml >>> Time Stamp: 03.04.2011 12:29:15 >>> Input Size: 30025 MB >>> Encoding: UTF-8 >>> Documents: 1 >>> Whitespace Chopping: ON >>> Entity Parsing: OFF >>> >>> Indexes >>> Up-to-date: true >>> Path Summary: ON >>> Text Index: ON >>> Attribute Index: ON >>> Full-Text Index: OFF >>> > >>> >>> >>> Timing info: >>> >>> Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; >>> Compiling: >>> - pre-evaluating fn:doc("enwiki-latest-pages-articles") >>> - optimizing descendant-or-self step(s) >>> - binding static variable $d >>> - removing variable $d >>> - simplifying flwor expression >>> Result: element siteinfo { ... } >>> Timing: >>> - Parsing: 1.4 ms >>> - Compiling: 52599.0 ms >>> - Evaluating: 0.28 ms >>> - Printing: 0.62 ms >>> - Total Time: 52601.32 ms >>> Query plan: >>> <DBNode name="enwiki-latest-pages-articles" pre="5"/> >>> >>> >>> >>> Result of query: >>> >>> <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" >>> http://www.w3.org/2001/XMLSchema-instance"> >>> <sitename>Wikipedia</sitename> >>> <base>http://en.wikipedia.org/wiki/Main_Page</base> >>> <generator>MediaWiki 1.17wmf1</generator> >>> <case>first-letter</case> >>> <namespaces> >>> <namespace key="-2" case="first-letter">Media</namespace> >>> <namespace key="-1" case="first-letter">Special</namespace> >>> <namespace key="0" case="first-letter"/> >>> <namespace key="1" case="first-letter">Talk</namespace> >>> <namespace key="2" case="first-letter">User</namespace> >>> <namespace key="3" case="first-letter">User talk</namespace> >>> <namespace key="4" case="first-letter">Wikipedia</namespace> >>> <namespace key="5" case="first-letter">Wikipedia talk</namespace> >>> <namespace key="6" case="first-letter">File</namespace> >>> <namespace key="7" case="first-letter">File talk</namespace> >>> <namespace key="8" case="first-letter">MediaWiki</namespace> >>> <namespace key="9" case="first-letter">MediaWiki talk</namespace> >>> <namespace key="10" case="first-letter">Template</namespace> >>> <namespace key="11" case="first-letter">Template talk</namespace> >>> <namespace key="12" case="first-letter">Help</namespace> >>> <namespace key="13" case="first-letter">Help talk</namespace> >>> <namespace key="14" case="first-letter">Category</namespace> >>> <namespace key="15" case="first-letter">Category talk</namespace> >>> <namespace key="100" case="first-letter">Portal</namespace> >>> <namespace key="101" case="first-letter">Portal talk</namespace> >>> <namespace key="108" case="first-letter">Book</namespace> >>> <namespace key="109" case="first-letter">Book talk</namespace> >>> </namespaces> >>> </siteinfo> >>> >>> >>> >>> Thanks >>> >>> Erol Akarsu >>> >>> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk(a)mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> >

2 1

Basex performance
by Erol Akarsu 11 Apr '11

11 Apr '11

I imported wikipedia xml into basex and tried to search it. But searching it takes longer. I tried to search one element that is first child of whole document and it took 52 sec. I know the XML file is very big 31GB. How can I optimize the search? declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; let $d := fn:doc ("enwiki-latest-pages-articles")//w:siteinfo return $d Database info: > open enwiki-latest-pages-articles Database 'enwiki-latest-pages-articles' opened in 778.49 ms. > info database Database Properties Name: enwiki-latest-pages-articles Size: 23356 MB Nodes: 228090153 Height: 6 Database Creation Path: /mnt/hgfs/C/tmp/enwiki-latest-pages-articles.xml Time Stamp: 03.04.2011 12:29:15 Input Size: 30025 MB Encoding: UTF-8 Documents: 1 Whitespace Chopping: ON Entity Parsing: OFF Indexes Up-to-date: true Path Summary: ON Text Index: ON Attribute Index: ON Full-Text Index: OFF > Timing info: Query: declare namespace w="http://www.mediawiki.org/xml/export-0.5/"; Compiling: - pre-evaluating fn:doc("enwiki-latest-pages-articles") - optimizing descendant-or-self step(s) - binding static variable $d - removing variable $d - simplifying flwor expression Result: element siteinfo { ... } Timing: - Parsing: 1.4 ms - Compiling: 52599.0 ms - Evaluating: 0.28 ms - Printing: 0.62 ms - Total Time: 52601.32 ms Query plan: <DBNode name="enwiki-latest-pages-articles" pre="5"/> Result of query: <siteinfo xmlns="http://www.mediawiki.org/xml/export-0.5/" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance"> <sitename>Wikipedia</sitename> <base>http://en.wikipedia.org/wiki/Main_Page</base> <generator>MediaWiki 1.17wmf1</generator> <case>first-letter</case> <namespaces> <namespace key="-2" case="first-letter">Media</namespace> <namespace key="-1" case="first-letter">Special</namespace> <namespace key="0" case="first-letter"/> <namespace key="1" case="first-letter">Talk</namespace> <namespace key="2" case="first-letter">User</namespace> <namespace key="3" case="first-letter">User talk</namespace> <namespace key="4" case="first-letter">Wikipedia</namespace> <namespace key="5" case="first-letter">Wikipedia talk</namespace> <namespace key="6" case="first-letter">File</namespace> <namespace key="7" case="first-letter">File talk</namespace> <namespace key="8" case="first-letter">MediaWiki</namespace> <namespace key="9" case="first-letter">MediaWiki talk</namespace> <namespace key="10" case="first-letter">Template</namespace> <namespace key="11" case="first-letter">Template talk</namespace> <namespace key="12" case="first-letter">Help</namespace> <namespace key="13" case="first-letter">Help talk</namespace> <namespace key="14" case="first-letter">Category</namespace> <namespace key="15" case="first-letter">Category talk</namespace> <namespace key="100" case="first-letter">Portal</namespace> <namespace key="101" case="first-letter">Portal talk</namespace> <namespace key="108" case="first-letter">Book</namespace> <namespace key="109" case="first-letter">Book talk</namespace> </namespaces> </siteinfo> Thanks Erol Akarsu

5 8

Re: [basex-talk] BaseX 6.6.1 Update - Soft-Files.com
by Alexander Holupirek 11 Apr '11

11 Apr '11

Hi Gary, the changelog can be found here: https://github.com/BaseXdb/basex/blob/master/changelog Kind regards, Alex On 11.04.2011, at 14:56, Gary Larsen wrote: > Could someone point me to a change log to learn what’s in this update? > > > > Thanks, > > gary > > > > > > > > From: basex-talk-bounces(a)mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] On Behalf Of Soft-Files Team > Sent: Monday, April 11, 2011 1:00 AM > To: basex-talk(a)mailman.uni-konstanz.de > Subject: [basex-talk] BaseX 6.6.1 Update - Soft-Files.com > > > > Dear Developer, > > Your product BaseX 6.6.1 has been updated. > > Your product review page is located at: http://www.soft-files.com/basex_72311.html. > > Sincerely, > The Soft-Files Team > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk(a)mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Alexander Holupirek |-- http://www.informatik.uni-konstanz.de/~holupire |-- Database & Information Systems Group, U Konstanz `-- Room E 209, 0049 7531 88 4449 (phone) 3577 (fax)

1 1

BaseX 6.6.1 Update - Soft-Files.com
by Soft-Files Team 11 Apr '11

11 Apr '11

Dear Developer, Your product BaseX 6.6.1 has been updated. Your product review page is located at: http://www.soft-files.com/basex_72311.html. Sincerely, The Soft-Files Team

2 1

Puzzling XQuery result
by Meredith Gregory 09 Apr '11

09 Apr '11

Dear BaseXers, i've got a result from an XQuery that i don't understand and am looking for insight and guidance. i've got a collection called 'Squibble' with three documents in it. They are listed below. The query for $rcrd in collection( 'Squibble' )//record where $rcrd/*[1] = <contentChannel><String>email</String></contentChannel> return $rcrd returns record elements from the first *two* documents. This is surprising to me because the first child of the first record element in the first document is <contactMeUsing>...</contactMeUsing>, not <contentChannel>...</contentChannel>. i was expecting it to return just the record element from the second document. i confess, i don't know anything about XQuery. So, it could be that i don't understand how the where clause is supposed to work. Any clues would be most appreciated. i'm running my tests using - BaseX 6.6 - on Mac OS X 10.6.7 Best wishes, --greg <?xml version="1.0" encoding="UTF-8"?> <record> <contactMeUsing> <String>email</String> </contactMeUsing> <String>{&quot;com.biosimilarity.lift.model.store.MonadicTermTypes_-Ground&quot;:{&quot;v&quot;:{&quot;@class&quot;:&quot;string&quot;,&quot;$&quot;:&quot;for the win Mikail.Mauz(a)gmail.com &quot;},&quot;_-outer&quot;:{&quot;@class &quot;:&quot;com.biosimilarity.lift.model.store.PersistedMonadicTS$TheMTT$&quot;}}}</String> </record> <?xml version="1.0" encoding="UTF-8"?> <record> <contentChannel> <String>email</String> </contentChannel> <String>{&quot;com.biosimilarity.lift.model.store.MonadicTermTypes_-Ground&quot;:{&quot;v&quot;:{&quot;@class&quot;:&quot;string&quot;,&quot;$&quot;:&quot;for the get Donald.Duque(a)yahoo.com &quot;},&quot;_-outer&quot;:{&quot;@class &quot;:&quot;com.biosimilarity.lift.model.store.PersistedMonadicTS$TheMTT$&quot;}}}</String> </record> <?xml version="1.0" encoding="UTF-8"?> <record> <phone> <var>cell</var> </phone> <String>{&quot;com.biosimilarity.lift.model.store.MonadicTermTypes_-Ground&quot;:{&quot;v&quot;:{&quot;@class &quot;:&quot;string&quot;,&quot;$&quot;:&quot;2xx.6xx.3xxx&quot;},&quot;_-outer&quot;:{&quot;@class &quot;:&quot;com.biosimilarity.lift.model.store.PersistedMonadicTS$TheMTT$&quot;}}}</String> </record> -- L.G. Meredith Managing Partner Biosimilarity LLC 7329 39th Ave SW Seattle, WA 98136 +1 206.650.3740 http://biosimilarity.blogspot.com

2 7

tf/idf
by Wiard Vasen 06 Apr '11

06 Apr '11

Dear sirs of Basex, I am doing my Master thesis on the letters of Vincent van Gogh at the University of Amsterdam. For that purpose I use BaseX to analyze the letters. I wonder, is there the possibility to generate a tf/idf score automatically? In your faq I noticed there needs to be a special term like 'SET SCORING 0' to be able to get a tf/idf score. This information I get from the following page: http://docs.basex.org/wiki/Full-Text Could you help me with this? I would be very grateful. Kind regards,

4 25

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk