- BaseX-Talk - mailman.uni-konstanz.de

Question about JSON parsing/serializing
by Hans-Juergen Rennau 22 Mar '21

22 Mar '21

Dear BaseX team, there is something I don't understand about the parsing/serializing of JSON. When I serialize a document without options, backslashes are escaped. So for example ... <url>https://help.openconnectors.ext.hana.ondemand.com/home/google-analytics</url> becomes"url":"https:\/\/help.openconnectors.ext.hana.ondemand.com\/home\/google-analytics" Only using serialization options - map{"escape": "no"} - do I get the output as expected, corresponding to the input. Why are slashes by default escaped when serializing JSON? Kind regards,Hans-Jürgen PS: When parsing, the slashes are not escaped, so it is "asymmetrical" - I need to parse without options, yet to serialize with options.

4 12

Re: [basex-talk] Call for install/setup stories from users
by Graydon 17 Mar '21

17 Mar '21

On Wed, Mar 17, 2021 at 10:43:07PM +0100, Ben Engbers scripsit: > To my knowledge there is no need to remove the data directory. I have never > done this and have never had any problems Well, data/ is in the archive empty and I need to retain the existing data from the incumbent basex when I update BaseX. I find it's simpler to promote data/ to being a sibling of basex/ and use a symbolic link from inside basex/ to find it so I don't have to argue with the defaults. (I am far from sure you can argue with the defaults to the extent of putting the data directory outside of basex/.) So, if you have no need to retain data = no need to do anything with data/. But if you do need to retain data/, giving whatever archive tool a chance to drop an empty data/ over it on upgrade is maybe not the best plan. -- Graydon Saunders | graydonish(a)gmail.com Þæs oferéode, ðisses swá mæg. -- Deor ("That passed, so may this.")

1 0

Preferences do not open, Exception in thread AWT-EventQueue-0
by commandline-be 17 Mar '21

17 Mar '21

Hey, To split off this bug report from the 'first post on this mailinglist' thread. Noteworthy but probably not relevant is every time is start basexgui i see [warning] /usr/bin/basexgui: Unable to locate /usr/share/java/tagsoup.jar in /usr/share/java [warning] /usr/bin/basexgui: Unable to locate /usr/share/java/jline.jar in /usr/share/java This is the output i see in the terminal when running the command basexgui and selecting Preferences in the window. This repeats every time i select Preferences. Exception in thread "AWT-EventQueue-0" java.lang.IndexOutOfBoundsException: Index: 3, Tab count: 3 at java.desktop/javax.swing.JTabbedPane.checkIndex(JTabbedPane.java:1758) at java.desktop/javax.swing.JTabbedPane.setSelectedIndex(JTabbedPane.java:595) at org.basex.gui.dialog.DialogPrefs.<init>(DialogPrefs.java:44) at org.basex.gui.dialog.DialogPrefs.show(DialogPrefs.java:56) at org.basex.gui.GUIMenuCmd$50.execute(GUIMenuCmd.java:778) at org.basex.gui.GUIMenu.lambda$newItem$0(GUIMenu.java:77) at java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967) at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308) at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) at java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) at java.desktop/javax.swing.AbstractButton.doClick(AbstractButton.java:369) at java.desktop/javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:1020) at java.desktop/javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:1064) at java.desktop/java.awt.Component.processMouseEvent(Component.java:6635) at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3342) at java.desktop/java.awt.Component.processEvent(Component.java:6400) at java.desktop/java.awt.Container.processEvent(Container.java:2263) at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5011) at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2321) at java.desktop/java.awt.Component.dispatchEvent(Component.java:4843) at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918) at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547) at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488) at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2307) at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2772) at java.desktop/java.awt.Component.dispatchEvent(Component.java:4843) at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772) at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721) at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95) at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:745) at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:743) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:742) at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90) Best, Joris- - - mailto:commandline@protonmail.com

6 16

first post on this mailinglist
by commandline-be 17 Mar '21

17 Mar '21

Dear, With a need to dig into ms eventlog converted to xml i landed with BaseX and was amazed by the functionality presented with this tool. I'm new to many things here so i'm going thru a learning curve here. This mail is mostly to propose improvement and feature requests, some minor other may be more elaborate * Map visualisation : a mode in which the keys are listed in hierarchical yet isometric way so they are not abreviated but listed for visual acessibility. * Forms builder : a simple form builder to datamine xml files would be nice, for my current purpose i'm to write a number of queries to identify specific eventid, attributes and values in relation to one another, being able to simply select a query in a drop down would be nice to review results etc * Possible bug : the preferences dialog no longer shows up * upgrade path : working with a linux distro prohibits simple upgrading, an upgrade dialog for drop in out of band upgrades could prove welcome to not flood this mailing list old bugs Thank you, Joris

2 3

Simple xqdoc to HTML converter
by Omar Siam 15 Mar '21

15 Mar '21

Hi, I brushed up the 2006 xqdoc to HTML XQuery a bit to document some of my code. Maybe this is also interesting to others. https://github.com/acdh-oeaw/xqdoc I also found https://github.com/lcahlander/xqDoc-eXist-db mentioned on this mailing list. I plan to look into getting this to work on BaseX. Best regards Omar Siam

3 2

Options for ft:search
by Sebastian Zimmer 15 Mar '21

15 Mar '21

Hi, I'd like to request a feature concerning the ft:search method: An `lserror` option that works exactly like the global option `LSERROR`. It would be nice if we could set the maximum Levenshtein distance specifically for each fuzzy search without having to adjust the global option. Another question on ft:search: Is there a reason why it doesn't have a `case` option just like ft:contains has one? Best regards, Sebastian

2 3

Histogramming a large dataset
by Ron Katriel 10 Mar '21

10 Mar '21

Hi, I would appreciate your advice on optimizing a query against a large BaseX (9.2.4) database. It is loaded with data from the FDA’s Adverse Event Reporting System (FAERS). Currently this is just the 2020 dataset which comprises 12 documents stored as 308,870,597 nodes (6,565 MB). The queries below effectively - though not necessarily efficiently - implement a histogram. The first, which is applied to patient gender (sex), returns the results (3 items) in 52 seconds: 2 893694 1 583999 0 198 The second does this for patient weight - rounded to the closest 10 lbs increment. It takes 580 seconds to place the data into 67 bins. Initially I tried running it on the rounded weights but aborted the run as it was a taking an inordinate amount of time (there are 217 distinct weights in the dataset). Is there a way to improve the performance of this type of query? Thanks, Ron (: 3 items - 52 sec :) let $safetyreport := db:open('FAERS')/ichicsr/safetyreport for $value in distinct-values($safetyreport/patient/patientsex) return concat($value, " ", count(index-of($safetyreport/patient/patientsex, $value))) (: 67 items - 580 sec :) let $safetyreport := db:open('FAERS')/ichicsr/safetyreport for $value in distinct-values($safetyreport/patient/patientweight ! (. div 10.0) ! round(.) ! (. * 10)) return concat($value, " ", count(index-of($safetyreport/patient/patientweight ! (. div 10.0) ! round(.) ! (. * 10), $value))) -- The information in this email and any attachments are intended solely for the recipient(s) to whom it is addressed, and may be confidential and/or privileged. Any unauthorized distribution or copying of this transmittal or its attachments is prohibited. If you are not a named recipient or have received this email in error: (i) you should not read, disclose, or copy it, (ii) please notify the sender of your receipt by reply email and delete this email and all attachments.

3 6

Question about Staircase Join
by Yasir B 03 Mar '21

03 Mar '21

Hello, I am studying the paper "Storing and Querying Large XML Instances". [https://files.basex.org/publications/Gruen%20%5B2010%5D,%20Storing%20and%20…] This is very interesting and helpful to read! I apologize I fail to understand the following in 3.4.2.1 Staircase Join: "In the partitioned plane, the scanned areas are made disjunct, i.e., scanning is canceled whenever the pre value of the currently scanned node equals the pre value of the next context node." Since pre values are unique (?), I couldn't follow how the next context node would have the same pre value. For the partitioning step, I had the impression the next context node is not the same as the current context node, nor would it be a descendant of the current context node. Thank you, Yasir

2 1

map-entries sequence
by Rob Stapper 02 Mar '21

02 Mar '21

Hi Christian, In my casus I’m exporting and importing xQuery maps where the export-format is a xQuery-file or a json-file. Hereby the sequence of the exported map-entries is important: I want the sequence in which the entries are generated to stay in tact when exporting. I can’t get this working. My current work-around is to place the map-entries in an array. It does the trick but it feels redundant. Is there a way to have this kind of control over map-entry sequence? Thnx in advance, Rob Stapper Sent from Mail for Windows 10 -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

2 1

Write several XML files via one XQuery
by Brune, Nicole 01 Mar '21

01 Mar '21

Dear all, please excuse me if this has been covered in the documentation and I just haven't found it. I am trying to harvest some information from XML files in a database (all XML files) and create several new XML files with the information. I am aware of the function write:file(path, data). Is it possible to write several files in one go and via one XQuery? I have tried exchanging "../test.xml" with "../test[1 to 2000].xml, ..." but, unfortunately, it did not work. Thank you kindly in advance for your help! Best, Nicole

2 1

BaseX 9.5: The Spring Edition
by Christian Grün 25 Feb '21

25 Feb '21

Dear all, we are more than happy to provide you with a new and fresh version of BaseX, our XML framework, database system and XQuery 3.1 processor: https://basex.org/ BaseX is getting faster and faster! We have spent most of our time on query rewritings and optimizations, which will speed up the execution of your XQuery code and RESTXQ apps. You can find various other new enhancements in our release: RESTXQ - the client IP address behind a proxy is added to the logs - stack traces in the error message can now be suppressed - the default error code has been changed from 400 to 500 - error messages are more user-friendly now (#1708) - content negotiation was improved (#1991) - inline arguments of called function (#1982) XQUERY FUNCTIONS - Utility: new functions for sequences, arrays and maps - Higher-Order Functions: hof:drop-while - Jobs: More options for job intervals XQUERY PERFORMANCE - revised scoring propagation (all items will take less memory) - default function inlining limit reduced from 100 to 50 - axis path rewritings, better typing (#1910, #1976, #1979) - better typing (#1906, #1908, #1909, #1944, #1945) - faster data()/string() checks (#1975) - rewrite map:merge to map:put (#1969) - rewrite order by to fn:sort (#1966) - rewrite group by to fn:distinct-values (#1932) - rewrite quantifiers to FLWOR expressions (#1961) - optimizations of arithmetic expressions (#1938, #1972) - optimizations of lookup operator (#1929, #1984) - optimizations of counts (#1965, #1973, #1974) - optimizations of regular lists (#1919, #1924) - optimizations of switch expression (#1920) - optimizations of instance of (#1939) - distinct values (#1930, #1967) - positional checks (#1937) - existence checks (#1971) For a more comprehensive list of added and updated features, check out our documentation (docs.basex.org) and check out the GitHub issues (github.com/BaseXdb/basex/issues). Have fun, Your BaseX Team

3 2

BaseX, xinclude and xpointer
by pascaljoubaud＠gmail.com 19 Feb '21

19 Feb '21

Hello BaseX-Community, I'm a beginner with baseX so my questions may sound a bit weird. I hope not too much. I have a problem using xinclude in a baseX database. Basically i have a main file which contains this xinclude instruction : <xi:include href="EP15.xml" xpointer="element(EP15)"><xi:fallback><p>Pièce non encodée pour le moment.</p></xi:fallback></xi:include> In my baseX database i have my main file and the EP15.xml file which contains this instruction : <div type="Piece" xml:id="EP15">blabla<div> When i 'm looking to my main file in base X, i always have the fallback instruction displayed : <p>Pièce non encodée pour le moment.</p> I'm using baseX 9.3.2., the local option XINCLUDE is turned on. I have also tried to use direct xpath instead of a named element in my xpointer xpointer="element(/TEI/text/body/div[1])"> but the resultat is stille the same I'm a bit stuck at the moment.. Any help would be very much appreciated ! Thanks a lot Pascal

2 10

Nasty trap with RESTXQ content negotiation
by Marco Lettere 19 Feb '21

19 Feb '21

Dear all, First of all, I hope everything is fine and all of you are staying healthy! When I read again the docs about Content Negotiation in RestXQ I noticed the sentence: Functions can be restricted to specific Media Types. The default type is|*/*|. So I wrote the two RestXQ signaures [1] with the expectation of *fallback()* behaving as a catch-all for mime-types not declared in *specific(**)*. At that point, when calling the service with something stating "Content-Type: application/xml" BaseX yields the following error: Stopped at /....: [basex:restxq] Several functions found for path "test": - ccp_ui:test [text/plain] - ccp_ui:test2 [text/plain] If I specify an explicit %rest:consumes("*/*") in the annotations of fallback(), specific() correctly intercepts the request. Unfortunately at that point calling with "Content-Type: text/plain" which should slip through the functions and be catched by fallback() yields instead: No function found that matches the request. So there is no combination, to my knowledge, to implement my use-case correctly ... besides handling mime-types explicitly in code (which is rather uncomfortable). I'm using BaseX 9.4.5 at the moment. Do you have any hint on this? Thank you very much. Marco. [1] declare %rest:path("test") %rest:POST("{$body}") %rest:consumes("text/csv", "application/xml", "application/json") %output:method("text") function s:specific($body as document-node()) { "Treated as document" }; declare %rest:path("test") %rest:POST("{$body}") %output:method("text") function s:fallback($body as item()) { "Treated as binary" };

2 2

vanishing whitespace with fn:doc()
by Jos van den Oever 17 Feb '21

17 Feb '21

Dear all, First off: BaseX is great to work with. I use it for a few statically generated websites. But I recently found what might be a bug. Some whitespace vanishes when loading xml files. E.g. this xml file: ```test.xml <a> a b <a> c </a> d e </a> ``` run like this: doc('test.xml') gives: <a>a b<a>c</a>d e</a> But running this: ``` parse-xml('<a> a b <a> c </a> d e </a>') ``` retains the whitespace. I've tested this with BaseX 7.0, 8.0, 9.0 and 9.4.6. Running this in saxon-he-10.3.jar retains the whitespace. I can work around this issue by placing xml:space="preserve" in the document element. I cannot come up with a scenario in which discarding whitespace during is parsing is ok when no DTD or XML Schema is provided. Best regards, Jos

2 7

Re: [basex-talk] Jobs and file modules
by Christian Grün 16 Feb '21

16 Feb '21

There may be various solutions ;) A minimized code example that demonstrates your procedure could be helpful here. On Tue, Feb 16, 2021 at 3:21 PM Tim Thompson <timathom(a)gmail.com> wrote: > > Yes, the insert operations also involve lookups against a couple of full-text indexes (using ft:search()). The entries in the full-text indexes include corresponding unique identifiers (URIs). The query: > > iterates through a set of library catalog records > filters on certain elements > runs full-text lookups on the text (which are name or subject strings) > inserts a new element containing the URI from a match in the index. > Logs the result of the each lookup (using file:append-text-lines()) > > > I'm guessing that the index access is what slows things down the most. What would be the approach for combining updates? > > Thank you, > Tim > > > -- > Tim A. Thompson > Discovery Metadata Librarian > Yale University Library > > > > On Tue, Feb 16, 2021 at 4:53 AM Christian Grün <christian.gruen(a)gmail.com> wrote: >> >> Hi Tim, >> >> > Is it possible to put updating expressions in a library module function (with the name of the database hard coded) and then call from the function within jobs:eval() in a main module? >> >> Yes, it should be. Here’s a little example (create database 'x' first): >> >> lib.xqm: >> module namespace lib = 'lib'; >> declare %updating function lib:delete() { >> prof:sleep(1000), >> delete node db:open('x')//x >> }; >> >> query.xq: >> let $id := jobs:eval(" >> import module namespace lib='lib' at 'lib.xqm'; >> lib:delete() >> ") >> return (prof:sleep(500), jobs:list-details($id)) >> >> The query result looks something like this: >> >> <job id="job29" type="QueryJob" state="running" user="admin" >> duration="PT0.498S" reads="(none)" writes="x" >> time="2021-02-16T10:40:57.605+01:00">import module namespace lib='lib' >> at 'lib.xqm'; lib:delete()</job> >> >> If the name of the database is found in the "writes" attribute, you’ll >> know that only this database will be write-locked. >> >> However, please note that, due to random file access patterns, >> parallel writes are often slower than consecutive ones (even with >> SSDs), so my guess is that you won’t save a lot. >> >> 48 hours sounds a lot indeed. It’s usually much, much faster to run a >> single XQuery expression that performs 1000 updates than running 1000 >> independent queries. Could you give us more information on the insert >> operations you need to perform? Is there any chance to combine >> updates? >> >> Best, >> Christian

1 0

Jobs and file modules
by Tim Thompson 16 Feb '21

16 Feb '21

Hello, I was experimenting with the jobs module and wondering why there's a difference between *file:write() *and *file:write-text()* in the query below: for $i in 1 to 5 return ( jobs:eval(' declare variable $iter external; file:write-text("~/Desktop/job"|| $iter ||".txt", (prof:sleep(5000), string(current-dateTime()))) ', map {"iter": $i} ) ) With *file:write-text()*, if I don't wrap *current-dateTime()* in *string()*, nothing is written and no files are created. With *file:write()*, text is always written and files always get created. Thanks! Tim -- Tim A. Thompson Metadata Librarian Yale University Library

2 6

Feature requests: Backups, Indexes, Execution Times
by BIRKNER Michael 12 Feb '21

12 Feb '21

Hello BaseX-Community, I am working a while with BaseX now and I like it very much. I think it is a great piece of software ... and it's Open Source ... WOW!!! But every now and then I discover some small cumbersome issues that could maybe solved with some new functionalities. That is why I would like to propose the following features for BaseX: * Description for database backups: Right now, the titles of database backups are like "dbname-2021-02-02-17-45-44". It would be great if one could add a description to such a backup. The reason is: While developing I often update databases step by step and in between these steps, I make backups of the whole database, so in case of errors or if I have to look up something, I could revert to a former state of the database. With a meaningful description, it would be much easier to pick the right backup. * RegEx in index searches: It would be great to have the possibility to search for RegEx patterns in the index, e. g. like this: db:text("db-name", "REGEX-PATTERN-HERE") or db:attribute("db-name", "REGEX-PATTERN-HERE") * Index searches with a path: I sometimes had the need to do an index search within only a certain path, e. g.: db:text('db-name', '/PATH/WITHIN/DB', 'search-word') I don't know if this is even possible to realize, but if yes, it would sometimes be very useful to me. * Selective indexing based on attribute values: It would be great to have selective indexing based on attributes and their values. Imagine an XML structure like this: <items> <item name="id">ABC</item> <item name="title">XYZ</item> <item name="person">Jane Doe</item> ... </items> If one would like to index e. g. only the ID values, a functionality to do selective indexing just for "<item>" nodes with attribute "name" that contains the value "id" would be useful. This would allow to have a comparatively small index for very large XML data sets, which could have impact on query performance. * Human readable execution times in GUI: Maybe a small change but - at least in my case - it would make developing performant xQueries much easier: Having the "Timing" section in the Info-View of the GUI display human readable times. Right now, the values are displayed only in milliseconds like 175713.28 ms. But an additional display in a more human readable format, e. g. hh:mm:ss.ms would sometimes be very useful. I hope at least some of the mentioned features do make sense and will be considered for implementation [:-)] Best regards, Michael Dieses Mail ist ausschließlich für die Verwendung durch die/den darin genannten AdressatInnen bestimmt und kann vertrauliche bzw rechtlich geschützte Informationen enthalten, deren Verwendung ohne Genehmigung durch den/ die AbsenderIn rechtswidrig sein kann. Falls Sie dieses Mail irrtümlich erhalten haben, informieren Sie uns bitte und löschen Sie die Nachricht. UID: ATU 16209706 I https://wien.arbeiterkammer.at/datenschutz

3 4

xquery:eval() required in replace()?
by Graydon Saunders 06 Feb '21

06 Feb '21

Hello -- So as part of building tests, I'm regularizing the text contents of some Word documents into single strings. (Which makes it relatively easy to make sure no words have gotten lost or changed order when compared to other stages of the process.) Regularization is a tactful way to put this particular atrocity: let $stringTidy as function(xs:string+) as xs:string := function($in as xs:string+) as xs:string {$in => string-join(' ') => replace(xquery:eval($menuMatch),'') => replace('
',' ') => replace('	',' ') => replace('',' ') => replace('\p{Zs}',' ') => replace(' +',' ') => replace(' ([,\.;:])','$1') => replace('^ ','') => replace(' $','')} $menuMatch gets stripped out of the Word because it's added by processing, rather than being present in the source file which generates the other half of the compare. (It's currently U+1405, ᐅ, though I devoutly hope this doesn't matter!) It gets read from an XSL source document, which I've included in minimal form, along with some sample data and a minimal-ish query. If I use $menuMatch in the replace, it doesn't work, in the sense that the ᐅ character is NOT removed from the string. If I xquery:eval() it, as here, the replace does work to remove the ᐅ from the string. I don't expect to need xquery:eval to use a variable as the second argument of replace(). Am I wrong? Has the pile of arrow operators exceeded the bounds of reason? Thanks! Graydon

3 2

Re: [basex-talk] FLOATS
by Christian Grün 06 Feb '21

06 Feb '21

Hi Adam, It’s recommendable to use xs:decimal instead of xs:float. Here’s one solution… declare function local:sumVec($seq) { let $count := max(($seq ! string() ! replace(., '[^_]', '') ! string-length())) + 1 let $sums := ( for $c in 1 to $count return sum( for $item in $seq let $tokens := tokenize(string($item), '_') return try { xs:decimal($tokens[$c]) } catch * { 0 } ) ) return string-join($sums, '_') }; …and another one (presumably more cryptic): declare function local:sumVec($seq) { let $numbers := ( for $item in $seq return array { for $token in tokenize(string($item), '_') return try { xs:decimal($token) } catch * { 0 } } ) return string-join( for $c in 1 to max($numbers ! array:size(.)) return sum($numbers[array:size(.) >= $c]?($c)) , '_') }; Hope this helps, Christian

1 0

Re: [basex-talk] BaseX-Talk Digest, Vol 134, Issue 3
by Kourouma Ismael 06 Feb '21

06 Feb '21

Hi, Please, how these queries should be helpful in my journey for learning BaseX ? Best regards, Ismael Send BaseX-Talk mailing list submissions to basex-talk(a)mailman.uni-konstanz.de To subscribe or unsubscribe via the World Wide Web, visit https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk or, via email, send a message with subject or body 'help' to basex-talk-request(a)mailman.uni-konstanz.de You can reach the person managing the list at basex-talk-owner(a)mailman.uni-konstanz.de When replying, please edit your Subject line so it is more specific than "Re: Contents of BaseX-Talk digest..." Today's Topics: 1. Re: Feature requests: Backups, Indexes, Execution Times (Christian Gr?n) ---------------------------------------------------------------------- Message: 1 Date: Sat, 6 Feb 2021 11:12:01 +0100 From: Christian Gr?n <christian.gruen(a)gmail.com> To: Graydon Saunders <graydonish(a)gmail.com> Cc: BIRKNER Michael <Michael.BIRKNER(a)akwien.at>, "basex-talk(a)mailman.uni-konstanz.de" <basex-talk(a)mailman.uni-konstanz.de> Subject: Re: [basex-talk] Feature requests: Backups, Indexes, Execution Times Message-ID: <CAP94bnN_4iLWvPG+S3C068jPx=0QXH_d1OtZrwJ+RLH-frAC6A(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Execution times will now be shown in the MM:SS.mm or HH:MM:SS.mm format, but only? ? in the header of the info view, and ? if the measured time exceeds 60 seconds. A new snapshot is available [1]. Have fun, Christian [1] https://files.basex.org/releases/latest/ On Thu, Feb 4, 2021 at 6:15 PM Graydon <graydonish(a)gmail.com> wrote: > > On Thu, Feb 04, 2021 at 05:21:24PM +0100, Christian Gr?n scripsit: > > > - *Human readable execution times in GUI*: Maybe a small change but > > > - at least in my case - it would make developing performant xQueries > > > much easier: Having the "Timing" section in the Info-View of the GUI > > > display human readable times. Right now, the values are displayed > > > only in milliseconds like 175713.28 ms. But an additional display in > > > a more human readable format, e. g. hh:mm:ss.ms would sometimes be > > > very useful. > > > > > Sounds reasonable and doable. I?ll think about it. > > Please leave the time formatting switchable! I'd strongly prefer > straight milliseconds to a human-readable format. > > -- > Graydon Saunders | graydonish(a)gmail.com > ??s ofer?ode, ?isses sw? m?g. > -- Deor ("That passed, so may this.") End of BaseX-Talk Digest, Vol 134, Issue 3 ******************************************

1 0

Re: [basex-talk] FLOATS
by Adam Law 04 Feb '21

04 Feb '21

Many thanks Christian - I don't know why I didn't think of decimal. I will review my xquery types and arrays. Thanks again - much appreciated :) Adam ! lobotomy() On Thu, Feb 4, 2021 at 11:57 PM Christian Grün <christian.gruen(a)gmail.com> wrote: > Hi Adam, > > It’s recommendable to use xs:decimal instead of xs:float. Here’s one > solution… > > declare function local:sumVec($seq) { > let $count := max(($seq ! string() ! > replace(., '[^_]', '') ! string-length())) + 1 > let $sums := ( > for $c in 1 to $count > return sum( > for $item in $seq > let $tokens := tokenize(string($item), '_') > return try { xs:decimal($tokens[$c]) } catch * { 0 } > ) > ) > return string-join($sums, '_') > }; > > …and another one (presumably more cryptic): > > declare function local:sumVec($seq) { > let $numbers := ( > for $item in $seq > return array { > for $token in tokenize(string($item), '_') > return try { xs:decimal($token) } catch * { 0 } > } > ) > return string-join( > for $c in 1 to max($numbers ! array:size(.)) > return sum($numbers[array:size(.) >= $c]?($c)) > , '_') > }; > > Hope this helps, > Christian > > > > On 2/4/21, Adam Law <adamjameslaw(a)gmail.com> wrote: > > (: > > Hello I have a hopefully not too silly question ... > > Is there an easy way to remove the exponent notation and return > > 30000201_30000000 > > without working out where the E is in the parts and padding zeros > > ====> <a> > > <d>3.0000201E7_3.0E7</d> > > </a> > > :) > > declare function local:sumVec($a,$opt){ > > let $a := <x>{for $w at $pos in $a > > return > > for $v at $pos in tokenize(xs:string($w),"_") > > return element {'a'} { (attribute {'pos'} {$pos}, try > > {xs:float($v)} catch * {0} )} > > }</x> > > let $b := string-join(for $w in distinct-values($a//@pos) > > return if ($opt=0) then sum($a/a[@pos=$w]) ,"_") > > return $b > > }; > > > > let $a := '' > > return <a> > > > > > <d>{local:sumVec(('10000001_10000000','10000000_10000000','10000000_10000000','100_',100),0)}</d> > > </a> > > >

1 0

FLOATS
by Adam Law 04 Feb '21

04 Feb '21

(: Hello I have a hopefully not too silly question ... Is there an easy way to remove the exponent notation and return 30000201_30000000 without working out where the E is in the parts and padding zeros ====> <a> <d>3.0000201E7_3.0E7</d> </a> :) declare function local:sumVec($a,$opt){ let $a := <x>{for $w at $pos in $a return for $v at $pos in tokenize(xs:string($w),"_") return element {'a'} { (attribute {'pos'} {$pos}, try {xs:float($v)} catch * {0} )} }</x> let $b := string-join(for $w in distinct-values($a//@pos) return if ($opt=0) then sum($a/a[@pos=$w]) ,"_") return $b }; let $a := '' return <a> <d>{local:sumVec(('10000001_10000000','10000000_10000000','10000000_10000000','100_',100),0)}</d> </a>

1 0

Mapping and Basex
by Ben Pracht 29 Jan '21

29 Jan '21

Hi, I've seen this geo module and wondered what this is used for. I'd like to see if there's a way of extracting addresses from a website and their corresponding school. I could be way off but I was hoping this would be a start. Thanks, Ben

1 0

Fwd: Securing All Endpoints
by Adam Law 27 Jan '21

27 Jan '21

Hi Christian, Thanks for the permissions pointer- it works well. As an aside I had to use let $vbFailure1 := ($perm?authorization != 'example' or exists( $perm?authorization )=false()) Rather than let $vbFailure1 := ($perm?authorization != 'example' ) To handle the case of no Authorisation. I am not sure why. I should be able to use request:cookie for ws (when I need to) which is great. I wasn't able to work out a catchall rest:path solution but it is not really important for me. Thanks again for the pointer. Adam On Mon, Jan 18, 2021 at 8:33 PM Christian Grün <christian.gruen(a)gmail.com> wrote: > You could have a look at the "Catch HTTP Errors" section in our > documentation. > > [1] https://docs.basex.org/wiki/RESTXQ#Error_Handling > > On Mon, Jan 18, 2021 at 1:30 PM Adam Law <adamjameslaw(a)gmail.com> wrote: > > > > Another thought - a 'catch all' path for fail2ban to catch attempted > accessess to undefined paths. > > I haven't tested this to see if there is such a thing in BaseX > > https://stackoverflow.com/questions/50282541/catch-all-url-in-golang > > > > On Mon, Jan 18, 2021 at 5:26 PM Christian Grün < > christian.gruen(a)gmail.com> wrote: > >> > >> Hi Adam, > >> > >> You could have a loot at our Permissions layer [1]. It can be used for > >> all kinds of checks, not only those specific to authentication. I > >> think we should generalize the concept in a future version of BaseX > >> and introduce additional post-hooks. > >> > >> Hope this helps? > >> Christian > >> > >> [1] https://docs.basex.org/wiki/Permissions > >> > >> > >> > >> On Mon, Jan 18, 2021 at 3:19 AM Adam Law <adamjameslaw(a)gmail.com> > wrote: > >> > > >> > Golang has great middleware functionality for endpoints that allows a > person to readily secure all of them. I was wondering there would be > anyway doing similar in BaseX for restxq & websockets. For example by > requiring all requests to have a particular query parameter (very basic > example). ... to protect say against browser / javascript port scanning. > >> > > >> > Eg > https://stackoverflow.com/questions/26204485/gorilla-mux-custom-middleware > and check that request contains a particular parameter (very basic security) > >> > > >> > It seems that I have to secure each endpoint individually. > >> > > >> > Adam >

2 4

Shrinking view in Manage Database Dialog
by Markus Schmid 27 Jan '21

27 Jan '21

Hallo in Basex v9.4.5 im Dialog Manage Database wird die linke View zusammengeschrumpft wenn die rechte Detail View wegen langer Pfad- oder Datenbanknamen viel Platz braucht. Die Views ändern die Breite während man die Einträge in der Liste der Datenbanken durchläuft und auswählt. Beides nervt leicht. Freundliche Grüße Markus S.

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk