BaseX-Talk

basex-talk@mailman.uni-konstanz.de

5251 discussions

Re: [basex-talk] Usage of doc's in BaseX
by Fabrice ETANCHAUD 14 Jun '18

14 Jun '18

Hi Bram, You are welcome, BaseX is such a versatile tool ! Keep index and data content separate could help in various ways. The main one is that, considering you still have to index your index database, you will index only needed data, and you might even set autoupdate to forget about index management. As it seems your data is 'append only', you should build your index on 'pre' node values instead of 'id' node values, in order to gain direct access to data. When working on XML documents, I try to replace 'database' with 'collection', and that seems to help in making effective decisions. Best regards, Fabrice Etanchaud [1] http://docs.basex.org/wiki/Node_Storage [2] http://docs.basex.org/wiki/Database_Module#db:node-pre [2] http://docs.basex.org/wiki/Database_Module#db:open-pre De : Bram Vanroy [mailto:Bram.Vanroy@UGent.be] Envoyé : jeudi 14 juin 2018 13:48 À : Fabrice ETANCHAUD Objet : RE: Usage of doc's in BaseX Hi Fabrice Thanks for the reply. The values that I am trying to index are indeed computed from the original XML but not present in it. Thanks for the limitations link, I had seen it but couldn't find it when googling for 'basex limitations' or others. Considering that my databases will be static and don't need updating, would you still argue for a separation? Thanks again Bram Van: BaseX-Talk <basex-talk-bounces(a)mailman.uni-konstanz.de> Namens Fabrice ETANCHAUD Verzonden: donderdag 14 juni 2018 11:18 Aan: BaseX <basex-talk(a)mailman.uni-konstanz.de> Onderwerp: Re: [basex-talk] Usage of doc's in BaseX Hello Bram, IMHO the main argument for data/index separation is the ease of index recreation, and the ease of reindexation of your index database. Is there still a need for ad hoc indexing, now that BaseX let us index only a node name selection ? I guess you need to index computed values ? For current BaseX limitations, you will find them in [1], but you might have already read that page. I hit the database node number limit once working with the European Patent Office DOCDB collection. So I had to set up a database naming politics to dispatch the documents. Hoping it helps, Best regards, Fabrice Etanchaud Senior Data Specialist CERFrance PCH [1] http://docs.basex.org/wiki/Statistics De : BaseX-Talk [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Bram Vanroy Envoyé : jeudi 14 juin 2018 10:47 À : BaseX Objet : [basex-talk] Usage of doc's in BaseX Dear BaseX team I am planning an update on our previous custom indexing system [1]. But to do this I have a couple of questions. The major ones will be how to write an efficient custom indexing query in XQuery, but that'll be for another email. (In fact, we have a dual indexing system, so two index files per main file.) For now I am mainly interested in different documents in a single databases, and the doc() functionality. Intuitively, I'd say that documents that are related to each other should be put in the same database. E.g. one database with different documents for plants, and one database with different documents for animals. But when I was scrolling through the documentation of BaseX, I noticed that when creating custom indices you do not put those in the same db as the original content, so you have on database for the content and one for the index [2]. Is this the way it's typically done? More generally, the questions that I have are the following: * What is the actual difference in BaseX between using separate documents in a single database, or using different databases all together? * Is there a performance difference when I would put my index file in the same database as the content, vs. when using different databases altogether? * What is the max allowed size for a document in a database and a database itself respectively? (I have files that are 100's of GB in size. It might not be plausible to have a file and its index file in the same database.) Thank you in advance Kind regards Bram Vanroy Doctoral Research at Ghent University, Belgium https://www.lt3.ugent.be/people/bram-vanroy/ [1] https://biblio.ugent.be/publication/8534144 [2] http://docs.basex.org/wiki/Indexes#Custom_Index_Structures

1 0

doc path in basex
by Martin Lourduswamy 14 Jun '18

14 Jun '18

Hi, I have a query that creates DB db:create("database", "db.xml") I have put the db.xml in the bin dir. This works for unix/Linux, but on windows I need to specifically give bin dir lke db:create("database", "bin/db.xml") Otherwise it does not work. I do not want to hard code path in the script. Is there a general way to specify the path, Thanks, Regards Martin Lourduswamy

1 0

Usage of doc's in BaseX
by Bram Vanroy 14 Jun '18

14 Jun '18

Dear BaseX team I am planning an update on our previous custom indexing system [1]. But to do this I have a couple of questions. The major ones will be how to write an efficient custom indexing query in XQuery, but that'll be for another email. (In fact, we have a dual indexing system, so two index files per main file.) For now I am mainly interested in different documents in a single databases, and the doc() functionality. Intuitively, I'd say that documents that are related to each other should be put in the same database. E.g. one database with different documents for plants, and one database with different documents for animals. But when I was scrolling through the documentation of BaseX, I noticed that when creating custom indices you do not put those in the same db as the original content, so you have on database for the content and one for the index [2]. Is this the way it's typically done? More generally, the questions that I have are the following: * What is the actual difference in BaseX between using separate documents in a single database, or using different databases all together? * Is there a performance difference when I would put my index file in the same database as the content, vs. when using different databases altogether? * What is the max allowed size for a document in a database and a database itself respectively? (I have files that are 100's of GB in size. It might not be plausible to have a file and its index file in the same database.) Thank you in advance Kind regards Bram Vanroy Doctoral Research at Ghent University, Belgium https://www.lt3.ugent.be/people/bram-vanroy/ [1] https://biblio.ugent.be/publication/8534144 [2] http://docs.basex.org/wiki/Indexes#Custom_Index_Structures

2 1

Issue with selecting child nodes
by Shaun Flynn 13 Jun '18

13 Jun '18

Hello there, I have the following query: declare variable $queryPath external := "D:\NROD Projects\xqCrews\diagrams.v3\"; declare variable $testSet := doc($queryPath || "tests\testSet1.xml"); $testSet where testSet1.xml is: <test> <test1> <content>Foobar</content> </test1> <test1> <content>Foobar</content> </test1> </test> When I run $testSet/*, I would expect to get the following result: <test1> <content>Foobar</content> </test1> <test1> <content>Foobar</content> </test1> However, I get the complete document in testSet1.xml When I run this against testSet1.xml $testSet/*/* then I get what would I expect to see: <test1> <content>Foobar</content> </test1> <test1> <content>Foobar</content> </test1> Am I misunderstanding something? Shaun

2 1

Nugets?
by Peter Villadsen 08 Jun '18

08 Jun '18

Hello, I am building a system that relies on a Basex backend, and for that I have chosen to use the TCP protocol. My system is built in managed code, so I went ahead and copied the code provided here: https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/c%23 which satisfied my needs very nicely. However, since I have not written this code myself, it is subject to expensive and time consuming pre-shipping legal audits where it is checked that the code does not contain references to other public domain code. Engineers developing managed code today expect APIs to be shipped as Nuget packages. This has the advantages that it is extremely easy to uptake, you can regard it as a black box (but get the source if you need to), and it supports dependencies (none, in your case) and you can go back and use previously shipped versions (which is important in your case, since you have just changed the authentication scheme I believe). Could you consider offering the API as both Nugets and source on Github? Best Regards Peter Villadsen Principal Architect Microsoft Business Applications Group

5 4

EXECUTE syntax
by Ben Engbers 08 Jun '18

08 Jun '18

Hi, I want to use my R-client to insert csv in a database. These lines works: csv_add_run <- 'RUN "./DataScience/RBaseX/CSVexample.xq"' Session$command(csv_add_run) When I take the content from CSVexample.xq and incorporate that into a EXECUTE command, I get this: csv_add_exe <- 'EXECUTE "let $root := '/home/bengbers/DataScience/RBaseX/Examples/Parse/;)'; for $path in file:children($root)[ends-with(., '.csv')] return db:add('CSV_test', $path, 'CSV_API', map { 'parser': 'csv', 'csvparser': map { 'header': 'yes', 'separator': ';' }) "' Session$command(csv_add_exe) Stopped at , 1/9: Unknown command: 'EXECUTE. Did you mean 'EXECUTE'? My question is how to define the input for the EXECUTE-command? Cheers, Ben

2 1

Re: [basex-talk] Long queries on big data timing out.
by Christian Grün 08 Jun '18

08 Jun '18

Dear Peter, 1. and 2. If you are working with the client/server architecture of BaseX, you can adjust the default timeout of 30 seconds by assigning another number of seconds to the TIMEOUT option [1]. This can e.g. be done by editing the .basex configuration file (please find more information in the top of the referenced Wiki document). 3. There is a CACHETIMEOUT option for caching the results of jobs:eval; see [2] for more details. 4. The jobs:list-details function will also list finalized jobs with enabled 'cache' option: <job id="job1" duration="PT1.32S" state="cached" type="QueryJob" ... By checking the @state attribute, you will see if the query is still "running", or if a "cached" result is available. Hope this helps, Christian PS: Your mail did not survive my spam check; maybe something is wrong with the mail configuration? [1] http://docs.basex.org/wiki/Options#TIMEOUT [2] http://docs.basex.org/wiki/Jobs_Module#jobs:eval > Hello, > > We have some complicated queries that are submitted to a database that is really, really big. The system will time out, just bringing back a message saying "Interrupted". > > 1. I know that you can provide a timeout parameter when using jobs:eval etc for batch jobs, but is there a setting that we can tweak to set the default for interactive (i.e. non jobs:eval) queries? > 2. What is the default value for the timeout value? > 3. How long will the server hang on to results from jobs:eval()? I think I remember reading that this is configurable, but how? Is it per job or globally for the server? > 4. Can you consider adding an API that tells how long the job has been running?

1 0

Long queries on big data timing out.
by Peter Villadsen 07 Jun '18

07 Jun '18

Hello, We have some complicated queries that are submitted to a database that is really, really big. The system will time out, just bringing back a message saying "Interrupted". 1. I know that you can provide a timeout parameter when using jobs:eval etc for batch jobs, but is there a setting that we can tweak to set the default for interactive (i.e. non jobs:eval) queries? 2. What is the default value for the timeout value? 3. How long will the server hang on to results from jobs:eval()? I think I remember reading that this is configurable, but how? Is it per job or globally for the server? 4. Can you consider adding an API that tells how long the job has been running? Best Regards Peter Villadsen Principal Architect Microsoft Business Applications Group

1 0

XML well-formed error
by DK Singh 06 Jun '18

06 Jun '18

Hi All, I have a XML file when i apply xslt using xslt:transform function it is giving well formed error, even i tried on BaseX UI using xslt:transform function [FODC0002] "" (Line 1): The markup in the document following the root element must be well-formed. but when the same file i transform on the oxygeng editor it getting transformed, can anyone suggest what going wrong. Regards Dharmendra Kumar Singh

4 8

Script tags
by Philip Spaelti 02 Jun '18

02 Jun '18

Hi, I'm using the GUI to generate html documents. The xquery includes a <script src="my_script.js"></script> tag which Basex simplifies to <script src="my_script.js"/> which causes problems in the browser. Am I doing something wrong, or is this a bug? best Philip --------------------------------------------------- Philip Spaelti Kobe Shoin Women's University, Graduate school ---------------------------------------------------

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

BaseX-Talk