BaseX-Talk

basex-talk@mailman.uni-konstanz.de

6 participants
5269 discussions

Re: [basex-talk] Setting multiple CSV EXPORTER options is not documented on docs.basex.org
by Christian Grün 03 Oct '24

03 Oct '24

Hi Navel, > BaseX Xquery CSV module is for dealing with CSV files which we don't have. With this module, you can serialize XML data as CSV (and parse CSV back to XML). But that’s certainly just one way to do it. Hi Owen, > However, I wonder how BaseX might deal with more than a million records. Feel free to get back to us once you encounter any limits. Best, Christian Naval Sarda <nsarda(a)epicomm.net> schrieb am Do., 3. Okt. 2024, 14:45: > Hi Owen, Christian, > > We are working on Stratml files which are in XML format. BaseX Xquery CSV > module is for dealing with CSV files which we don't have. > > We can scale up BaseX by adding more servers and splitting the data > between more servers if the record counts goes high. > > Naval > > > On 02/10/24 10:54 pm, Owen Ambur wrote: > > Naval, can you answer Christian's question? > > My sense is that a lot of features have been built into BaseX that we > could use but I don't have the knowledge or expertise to understand how > best to do so. At least, I hope we're not reinventing capabilities that > are already available to address our technical objectives > <https://aboutthem.info/SQS.xml> as well as our broader conceptual > objectives <https://aboutthem.info/ATI.xml>. > > Christian, this morning we had a very encouraging Zoom meeting with folks > at GSA who are finally getting around to figuring out how to help Uncle > Sam's agencies comply with section 10 > <https://www.linkedin.com/pulse/trustworthy-institutions-owen-ambur/> of > the GPRA Modernization Act. I encouraged them not to reinvent the StratML > schemas and it would be great if the BaseX community could help us > demonstrate the benefits of using them, not only by U.S. federal agencies > but agencies at all levels of government, worldwide, as well as tax-favored > organizations and others whose plans and reports should be matters of > public record. > > There are >5.8K plans in the StratML collection > <https://stratml.us/drybridge/index.htm> and indexed in the query > service. Their URLs are listed in sitemap format at > https://stratml.us/docs/sitemap.xml > > I am contemplating whether to hire someone to convert the relevant > elements of the IRS Form 990 database > <https://www.irs.gov/charities-non-profits/tax-exempt-organization-search-bu…> > to StratML format, using the templates at > https://stratml.us/drybridge/index.htm#MPR4CO However, I wonder how > BaseX might deal with more than a million records. > > Owen Ambur > https://www.linkedin.com/in/owenambur/ > > > On Wednesday, October 2, 2024 at 12:18:16 PM EDT, Christian Grün > <christian.gruen(a)gmail.com> <christian.gruen(a)gmail.com> wrote: > > > Hi Owen, > > Is it based on the BaseX XQuery CSV Module? > > Best, > Christian > > > > > Owen Ambur <owen.ambur(a)verizon.net> schrieb am Mi., 2. Okt. 2024, 18:12: > > I'm not sure how this exchange might relate but my developer has provided > CSV export capabilities for query results listings at StratML > <https://search.aboutthem.info/> > > > StratML <https://search.aboutthem.info/> > > > > Any comments or suggestions on how we might enhance the functionality of > our StratML-enabled query service would be most welcome. > > Owen Ambur > https://www.linkedin.com/in/owenambur/ > > > On Tuesday, October 1, 2024 at 04:42:12 AM EDT, Christian Grün < > christian.gruen(a)gmail.com> wrote: > > > Hi Omar, > > Thanks for the observation. For the SERIALIZER option, there was a > corresponding note which I have added to the EXPORTER option, along with > a little example for exporting CSV [1]. > > Best, > Christian > > [1] https://docs.basex.org/main/Options#exporter > > > > On Mon, Sep 30, 2024 at 4:04 PM Omar Siam <Omar.Siam(a)oeaw.ac.at> wrote: > > Hi, > > I just tried after some time to script a CSV export with a BXS file. I > forgot how to set multiple options for the csv serialization. In the end > this worked: > <set > option="EXPORTER">method=csv,csv=header=true,,lax=false,,quotes=true</set> > I can not find it on docs.basex.org. Perhaps this should be added and > explained. > > Best regards > > -- > Mag. Ing. Omar Siam > Austrian Center for Digital Humanities and Cultural Heritage > Österreichische Akademie der Wissenschaften | Austrian Academy of Sciences > Stellvertretende Behindertenvertrauensperson | Deputy representative for > disabled persons > Bäckerstraße 13, 1010 Wien, Österreich | Vienna, Austria > T: +43 1 51581-7295 > omar.siam(a)oeaw.ac.at | www.oeaw.ac.at/acdh > >

1 0

Setting multiple CSV EXPORTER options is not documented on docs.basex.org
by Omar Siam 02 Oct '24

02 Oct '24

Hi, I just tried after some time to script a CSV export with a BXS file. I forgot how to set multiple options for the csv serialization. In the end this worked: <set option="EXPORTER">method=csv,csv=header=true,,lax=false,,quotes=true</set> I can not find it on docs.basex.org. Perhaps this should be added and explained. Best regards -- Mag. Ing. Omar Siam Austrian Center for Digital Humanities and Cultural Heritage Österreichische Akademie der Wissenschaften | Austrian Academy of Sciences Stellvertretende Behindertenvertrauensperson | Deputy representative for disabled persons Bäckerstraße 13, 1010 Wien, Österreich | Vienna, Austria T: +43 1 51581-7295 omar.siam(a)oeaw.ac.at | www.oeaw.ac.at/acdh

3 4

Equivalent of eXist's util:wait in BaseX?
by Joe Wicentowski 27 Sep '24

27 Sep '24

Hi all, I'm retrieving content from a list of URLs with BaseX (via the http:send-request function) and appear to be hitting rate limits on the remote server. I'd like to pause for 5 seconds between requests. In eXist, I'd use util:wait [1] to handle this pause. Is there an equivalent in BaseX? Or is there a different approach in BaseX? Thanks! Joe [1] https://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/x…

2 2

Re: [basex-talk] Error with function-lookup and static variable
by Rob Stapper 21 Sep '24

21 Sep '24

Perfect. Cascaded function-calls through variables within self-recursive constructs work fine now. mvgr. Rob Stapper

1 0

Error with function-lookup and static variable
by Amanda Galtman 20 Sep '24

20 Sep '24

Hi, all. I'm seeing an error in BaseX when I use function-lookup in both a global variable and a function, where the function relies on the variable. I reduced the situation to the following small query: xquery version "3.1"; declare variable $local:lookup := function-lookup(QName("nonexistent", "nonexistent"), 1); declare function local:myfcn() { let $f := ($local:lookup, function-lookup(QName('http://www.w3.org/2005/xpath-functions','string'), 1))[1] return $f('a') }; local:myfcn() When I run it with BaseX 11.1, I get [XQDY0054] Static variable depends on itself: $local:lookup When I run it with Saxon-HE 12, I don't get this error. Is there anything I can do in my code to avoid this error? Thanks, Amanda

4 13

Unknown function db:open
by Kendall Shaw 17 Sep '24

17 Sep '24

I'm using basex 11.2 under WSL ubuntu 24.04 when I use the command xquery count(db:open('some')/*), I see an error message about db prefix being undefined. The same thing from an editor tab. import module namespace db = 'http://basex.org/modules/db'; count(db:open('some')/*) I get the same error message. Is this to be expected? How can I supply what is needed? Thanks, Kendall Shaw

2 2

Re: [basex-talk] BaseX-Talk Digest, Vol 177, Issue 2
by Rob Stapper 12 Sep '24

12 Sep '24

> Op 11-09-2024 12:00 CEST schreef basex-talk-request(a)mailman.uni-konstanz.de: > > > Send BaseX-Talk mailing list submissions to > basex-talk(a)mailman.uni-konstanz.de > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > or, via email, send a message with subject or body 'help' to > basex-talk-request(a)mailman.uni-konstanz.de > > You can reach the person managing the list at > basex-talk-owner(a)mailman.uni-konstanz.de > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of BaseX-Talk digest..." > > > Today's Topics: > > 1. terrible performance of webserver under BaseX11 (Rob Stapper) > 2. Re: terrible performance of webserver under BaseX11 > (Christian Gr?n) > 3. Re: terrible performance of webserver under BaseX11 > (Christian Gr?n) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 10 Sep 2024 15:31:52 +0200 (CEST) > From: Rob Stapper <r.stapper(a)lijbrandt.nl> > To: "basex-talk(a)mailman.uni-konstanz.de" > <basex-talk(a)mailman.uni-konstanz.de> > Subject: [basex-talk] terrible performance of webserver under BaseX11 > Message-ID: <1197257124.2620908.1725975112156(a)kpc.webmail.kpnmail.nl> > Content-Type: text/plain; charset="utf-8" > > Hi Christian, > > My webserver-app that performed reasenably well under BaseX10 performs terribly under Basex11. Chrome even gives me at the end of the process an 'out of memory' message. So it is not only performance but also resource-usage. I have absolutely no clou how to address this, so all suggestions are very welcome. > > mvgr. > > Rob Stapper >

2 1

Re: [basex-talk] terrible performance of webserver under BaseX11
by Christian Grün 10 Sep '24

10 Sep '24

(cc to the list) For a more helpful response, we would certainly need to look at your code and run it by ourselves. I recommend you to run your function… $SWM:dialog.new …outside your web application, e.g. in the BqseX GUI. This may help you to isolate the problematic part of the code.

1 0

terrible performance of webserver under BaseX11
by Rob Stapper 10 Sep '24

10 Sep '24

Hi Christian, My webserver-app that performed reasenably well under BaseX10 performs terribly under Basex11. Chrome even gives me at the end of the process an 'out of memory' message. So it is not only performance but also resource-usage. I have absolutely no clou how to address this, so all suggestions are very welcome. mvgr. Rob Stapper

2 1

Unexpected error: Improper use? Potential bug? Your feedback is welcome:
by Rob Stapper 03 Sep '24

03 Sep '24

Hi Christian, After the hollidaybreak I found the courage to try to port my application to BaseX's version 112. I get the error-dump shown below. Any idea what is going wrong here? error-dump: ========= Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk(a)mailman.uni-konstanz.de Version: BaseX 11.2 Java: Eclipse Adoptium, 17.0.8 OS: Windows 10, amd64 Stack Trace: java.lang.ArrayIndexOutOfBoundsException: Index 3 out of bounds for length 3 at org.basex.query.func.Closure.setSignature(Closure.java:418) at org.basex.query.func.StaticFuncs$FuncCache.init(StaticFuncs.java:268) at org.basex.query.func.StaticFuncs.check(StaticFuncs.java:83) at org.basex.query.QueryParser.check(QueryParser.java:270) at org.basex.query.QueryParser.parseLibrary(QueryParser.java:198) at org.basex.query.QueryContext.lambda$parseLibrary$1(QueryContext.java:236) at org.basex.query.QueryContext.run(QueryContext.java:763) at org.basex.query.QueryContext.parseLibrary(QueryContext.java:234) at org.basex.query.QueryContext.parse(QueryContext.java:178) at org.basex.http.web.WebModule.qc(WebModule.java:100) at org.basex.http.web.WebModule.parse(WebModule.java:57) at org.basex.http.web.WebModules.parse(WebModules.java:376) at org.basex.http.web.WebModules.parse(WebModules.java:367) at org.basex.http.web.WebModules.parse(WebModules.java:367) at org.basex.http.web.WebModules.cache(WebModules.java:336) at org.basex.http.web.WebModules.find(WebModules.java:149) at org.basex.http.web.WebModules.restxq(WebModules.java:116) at org.basex.http.restxq.RestXqServlet.run(RestXqServlet.java:44) at org.basex.http.BaseXServlet.service(BaseXServlet.java:69) at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1410) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:529) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1381) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1553) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1303) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.Server.handle(Server.java:563) at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) at java.base/java.lang.Thread.run(Unknown Source) mvgr. Rob Stapper

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk