How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi,
do you mean something like a repo variable ?
If you define a module name in your repo:
1- module namespace myNs = 'http://www.xxxxx.fr/xxxx-repo%E2%80%99 http://www.xxxxx.fr/xxxx-repo%E2%80%99;
You can define in your repo a new variable :
1- declare variable $myNs:testid := db:open(‘xxxxx')/utilisateurs/entry/sessions/session/id=session:id();
Then import your module in your restxq:
1- import module namespace isi = 'http://www.isilex.fr/isi-repo%E2%80%99 http://www.isilex.fr/isi-repo%E2%80%99;
And access your variables:
1- $myNs:testid
More here: http://docs.basex.org/wiki/XQuery_3.0#External_Variables http://docs.basex.org/wiki/XQuery_3.0#External_Variables
Sorry If I’m wrong and didn’t understand,
XLS
Le 5 sept. 2017 à 01:37, Erik Peterson ep@ardec.com a écrit :
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
A problem with that is that it evaluates the variable each time. If you need it to only evaluate once (expensive operation), something else needs to be done. I would like to know an answer to this so that I can schedule jobs on start up without involving processes external to basex.
From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Xavier-Laurent SALVADOR xavierlaurent.salvador@gmail.com Date: Monday, September 4, 2017 at 11:42 PM To: Erik Peterson ep@ardec.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Server Variables, cached vars, etc
Hi,
do you mean something like a repo variable ?
If you define a module name in your repo:
1- module namespace myNs = 'http://www.xxxxx.fr/xxxx-repo%E2%80%99https://urldefense.proofpoint.com/v2/url?u=http-3A__www.xxxxx.fr_xxxx-2Drepo-25E2-2580-2599&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=6BbM4k_uDRdhjURAbaUe8SgS19qfgxyIM03BQbY2xzk&s=vnUoDRkXSxl-NAs4wmOr8q2saq5Y9K0_k6l71w-jrrM&e=;
You can define in your repo a new variable :
1- declare variable $myNs:testid := db:open(‘xxxxx')/utilisateurs/entry/sessions/session/id=session:id();
Then import your module in your restxq:
1- import module namespace isi = 'http://www.isilex.fr/isi-repo%E2%80%99https://urldefense.proofpoint.com/v2/url?u=http-3A__www.isilex.fr_isi-2Drepo-25E2-2580-2599&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=6BbM4k_uDRdhjURAbaUe8SgS19qfgxyIM03BQbY2xzk&s=7iT9husSt3pxwCXxx8Kf7Sf1KKDDurqvyhDHERP1p2g&e=;
And access your variables:
1- $myNs:testid
More here: http://docs.basex.org/wiki/XQuery_3.0#External_Variableshttps://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_XQuery-5F3.0-23External-5FVariables&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=6BbM4k_uDRdhjURAbaUe8SgS19qfgxyIM03BQbY2xzk&s=tfVbcjLW6lGgTfWHzfgWtKLPja7kdFONHW74KF_CdiA&e=
Sorry If I’m wrong and didn’t understand,
XLS Le 5 sept. 2017 à 01:37, Erik Peterson <ep@ardec.commailto:ep@ardec.com> a écrit :
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
1) Use session http://docs.basex.org/wiki/Session_Module
2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
3) Use the file system.
Hope this helps [cit] ;-)
Marco.
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
1) Use session http://docs.basex.org/wiki/Session_Module
2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
3) Use the file system.
Hope this helps [cit] ;-)
Marco.
To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
1) Use session http://docs.basex.org/wiki/Session_Module
2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
3) Use the file system.
Hope this helps [cit] ;-)
Marco.
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD < fetanchaud@pch.cerfrance.fr> wrote:
To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto: basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
Use session http://docs.basex.org/wiki/Session_Module
Use a database which is builtin in BaseX and is very lightweight.
Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
- Use the file system.
Hope this helps [cit] ;-)
Marco.
--
*Erik Peterson* President, Ardec LLC 281-804-9023 | ep@ardec.com
I thought it might be useful to mention advice I was given about startup hooks:
From: "Kirsten, Dirk" Dirk.Kirsten@senacor.commailto:Dirk.Kirsten@senacor.com
,,,
there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup... for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Erik Peterson ep@ardec.com Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr, "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD <fetanchaud@pch.cerfrance.frmailto:fetanchaud@pch.cerfrance.fr> wrote: To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
1) Use session http://docs.basex.org/wiki/Session_Modulehttps://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_Session-5FModule&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=DfAtFmXgoUqMIP116Hn78cCdB0klwv2oiZZ8v8di0Ew&s=VUiY8Im77MBCa2uPCzDsV7HRZEC-Sqx2WNzt_CD9Iok&e=
2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
3) Use the file system.
Hope this helps [cit] ;-)
Marco. -- Erik Peterson President, Ardec LLC 281-804-9023 | ep@ardec.commailto:ep@ardec.com
Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing?
On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw kendall.shaw@workday.com wrote:
I thought it might be useful to mention advice I was given about startup hooks:
From: "Kirsten, Dirk" Dirk.Kirsten@senacor.com
,,,
there is currently no way to do this using BaseX itself. But I also
don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to- run-java-program-automatically-on-tomcat-startup/ for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
*From: *basex-talk-bounces@mailman.uni-konstanz.de on behalf of Erik Peterson ep@ardec.com *Date: *Tuesday, September 5, 2017 at 7:02 AM *To: *Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr, " basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de *Subject: *Re: [basex-talk] Server Variables, cached vars, etc
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD < fetanchaud@pch.cerfrance.fr> wrote:
To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto: basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni- konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
- Use session http://docs.basex.org/wiki/Session_Module
- Use a database which is builtin in BaseX and is very lightweight.
Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
- Use the file system.
Hope this helps [cit] ;-)
Marco.
--
*Erik Peterson*
President, Ardec LLC
281-804-9023 <(281)%20804-9023> | ep@ardec.com
The servlet could populate your singleton just once upon startup, or run xquery etc. The load-on-startup configuration means that the servlet is initialized after basex has been loaded. So, if you restart jetty or whatever web server/web container you are using basex restarts and then your servlet’s init method is invoked.
Kendall
From: Erik Peterson ep@ardec.com Date: Saturday, September 9, 2017 at 4:16 AM To: Kendall Shaw kendall.shaw@workday.com Cc: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing?
On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw <kendall.shaw@workday.commailto:kendall.shaw@workday.com> wrote: I thought it might be useful to mention advice I was given about startup hooks:
From: "Kirsten, Dirk" Dirk.Kirsten@senacor.commailto:Dirk.Kirsten@senacor.com
,,,
there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup...https://urldefense.proofpoint.com/v2/url?u=http-3A__crunchify.com_how-2Dto-2Drun-2Djava-2Dprogram-2Dautomatically-2Don-2Dtomcat-2Dstartup_&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=fPChED25tvjfNbUFkPSuyRBFfpYGFrqPI1DLNeEovLI&s=0q_rTZpxZNOefv3G77E1Vna_JmrxWcl0OcoKc1EmHAc&e= for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
From: <basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de> on behalf of Erik Peterson <ep@ardec.commailto:ep@ardec.com> Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD <fetanchaud@pch.cerfrance.frmailto:fetanchaud@pch.cerfrance.fr>, "basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de" <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD <fetanchaud@pch.cerfrance.frmailto:fetanchaud@pch.cerfrance.fr> wrote: To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.demailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
1) Use session http://docs.basex.org/wiki/Session_Modulehttps://urldefense.proofpoint.com/v2/url?u=http-3A__docs.basex.org_wiki_Session-5FModule&d=DwMFaQ&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=JgwnBEpN1c-DDmq-Up2QMq9rrGyfWK0KtSpT7dxRglA&m=DfAtFmXgoUqMIP116Hn78cCdB0klwv2oiZZ8v8di0Ew&s=VUiY8Im77MBCa2uPCzDsV7HRZEC-Sqx2WNzt_CD9Iok&e=
2) Use a database which is builtin in BaseX and is very lightweight. Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
3) Use the file system.
Hope this helps [cit] ;-)
Marco. -- Erik Peterson President, Ardec LLC 281-804-9023tel:(281)%20804-9023 | ep@ardec.commailto:ep@ardec.com
-- Erik Peterson President, Ardec LLC 281-804-9023 | ep@ardec.commailto:ep@ardec.com
Hi Erik,
I think that Xavier-Laurent, Marco, Fabrice and Kendall have already given excellent feedback.
In our own projects, we store all global data in databases, or in local configuration files. One advantage is that this data requires no initialization and will automatically be available after a restart.
I don’t know anything about »server variables« in MarkLogic so far, so @Erik: feel free to pass me on a link to the documentation, and I can check if a similar solution could make sense for BaseX.
Talking about the start script server option: The basexserver command comes with a -c flag, which allows you run initial commands [1]. We could add such a flag for basexhttp, or even allow an initial input for both startup commands (similar to basex/basexclient). Would this be helpful for some of you reading this? Quite obviously, this requires BaseX to be run via these scripts (it wouldn’t have any effect if BaseX is deployed as servlet).
Cheers, Christian
[1] http://docs.basex.org/wiki/Command-Line_Options#Server
On Sun, Sep 10, 2017 at 1:56 AM, Kendall Shaw kendall.shaw@workday.com wrote:
The servlet could populate your singleton just once upon startup, or run xquery etc. The load-on-startup configuration means that the servlet is initialized after basex has been loaded. So, if you restart jetty or whatever web server/web container you are using basex restarts and then your servlet’s init method is invoked.
Kendall
From: Erik Peterson ep@ardec.com Date: Saturday, September 9, 2017 at 4:16 AM To: Kendall Shaw kendall.shaw@workday.com Cc: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing?
On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw kendall.shaw@workday.com wrote:
I thought it might be useful to mention advice I was given about startup hooks:
From: "Kirsten, Dirk" Dirk.Kirsten@senacor.com
,,,
there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup... for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Erik Peterson ep@ardec.com Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr, "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr wrote:
To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
Use session http://docs.basex.org/wiki/Session_Module
Use a database which is builtin in BaseX and is very lightweight.
Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
- Use the file system.
Hope this helps [cit] ;-)
Marco.
--
Erik Peterson
President, Ardec LLC
281-804-9023 | ep@ardec.com
--
Erik Peterson
President, Ardec LLC
281-804-9023 | ep@ardec.com
Hi Christian, welcome back to the list! Having the possibility to run a script or call any sort of cuntvionality at startup is quite a useful thing. Doing it in a declarative way would be great maybe exploiting the .basex file or the web.xml?
Btw, hooking up on Server variables, a colleague of mine has just come up with another (somewhat exotic) way of passing content from one RestXQ call to another using proc:system to set and get environment variables variables. ;-) Regards, Marco.
On 18/09/2017 16:19, Christian Grün wrote:
Hi Erik,
I think that Xavier-Laurent, Marco, Fabrice and Kendall have already given excellent feedback.
In our own projects, we store all global data in databases, or in local configuration files. One advantage is that this data requires no initialization and will automatically be available after a restart.
I don’t know anything about »server variables« in MarkLogic so far, so @Erik: feel free to pass me on a link to the documentation, and I can check if a similar solution could make sense for BaseX.
Talking about the start script server option: The basexserver command comes with a -c flag, which allows you run initial commands [1]. We could add such a flag for basexhttp, or even allow an initial input for both startup commands (similar to basex/basexclient). Would this be helpful for some of you reading this? Quite obviously, this requires BaseX to be run via these scripts (it wouldn’t have any effect if BaseX is deployed as servlet).
Cheers, Christian
[1] http://docs.basex.org/wiki/Command-Line_Options#Server
On Sun, Sep 10, 2017 at 1:56 AM, Kendall Shaw kendall.shaw@workday.com wrote:
The servlet could populate your singleton just once upon startup, or run xquery etc. The load-on-startup configuration means that the servlet is initialized after basex has been loaded. So, if you restart jetty or whatever web server/web container you are using basex restarts and then your servlet’s init method is invoked.
Kendall
From: Erik Peterson ep@ardec.com Date: Saturday, September 9, 2017 at 4:16 AM To: Kendall Shaw kendall.shaw@workday.com Cc: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing?
On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw kendall.shaw@workday.com wrote:
I thought it might be useful to mention advice I was given about startup hooks:
From: "Kirsten, Dirk" Dirk.Kirsten@senacor.com
,,,
there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat-startup... for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Erik Peterson ep@ardec.com Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr, "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr wrote:
To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
Use session http://docs.basex.org/wiki/Session_Module
Use a database which is builtin in BaseX and is very lightweight.
Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
- Use the file system.
Hope this helps [cit] ;-)
Marco.
--
Erik Peterson
President, Ardec LLC
281-804-9023 | ep@ardec.com
--
Erik Peterson
President, Ardec LLC
281-804-9023 | ep@ardec.com
Hello Christian !
Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Christian Grün Envoyé : lundi 18 septembre 2017 16:20 À : Kendall Shaw; coach3pete Cc : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
Hi Erik,
I think that Xavier-Laurent, Marco, Fabrice and Kendall have already given excellent feedback.
In our own projects, we store all global data in databases, or in local configuration files. One advantage is that this data requires no initialization and will automatically be available after a restart.
I don’t know anything about »server variables« in MarkLogic so far, so @Erik: feel free to pass me on a link to the documentation, and I can check if a similar solution could make sense for BaseX.
Talking about the start script server option: The basexserver command comes with a -c flag, which allows you run initial commands [1]. We could add such a flag for basexhttp, or even allow an initial input for both startup commands (similar to basex/basexclient). Would this be helpful for some of you reading this? Quite obviously, this requires BaseX to be run via these scripts (it wouldn’t have any effect if BaseX is deployed as servlet).
Cheers, Christian
[1] http://docs.basex.org/wiki/Command-Line_Options#Server
On Sun, Sep 10, 2017 at 1:56 AM, Kendall Shaw kendall.shaw@workday.com wrote:
The servlet could populate your singleton just once upon startup, or run xquery etc. The load-on-startup configuration means that the servlet is initialized after basex has been loaded. So, if you restart jetty or whatever web server/web container you are using basex restarts and then your servlet’s init method is invoked.
Kendall
From: Erik Peterson ep@ardec.com Date: Saturday, September 9, 2017 at 4:16 AM To: Kendall Shaw kendall.shaw@workday.com Cc: "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thanks Kendal for your reply. What would be the advantage of creating a servlet over a singleton class to do the same thing?
On Fri, Sep 8, 2017 at 11:12 AM, Kendall Shaw kendall.shaw@workday.com wrote:
I thought it might be useful to mention advice I was given about startup hooks:
From: "Kirsten, Dirk" Dirk.Kirsten@senacor.com
,,,
there is currently no way to do this using BaseX itself. But I also don’t think that should be the job of BaseX. Instead you can write a servlet and deploy it using Tomcat which runs some Java application, e.g. which could trigger some BaseXX command. See http://crunchify.com/how-to-run-java-program-automatically-on-tomcat- startup/ for an example how to do this.
I switched from using a cron job, to doing this in order to schedule jobs. I have very simple servlet that is configured with <load-on-startup>2</load-on-startup> (basex has load-on-startup 2). It runs a shell script which schedules the jobs, soon after basex is loaded.
Kendall
From: basex-talk-bounces@mailman.uni-konstanz.de on behalf of Erik Peterson ep@ardec.com Date: Tuesday, September 5, 2017 at 7:02 AM To: Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr, "basex-talk@mailman.uni-konstanz.de" basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Server Variables, cached vars, etc
Thank you all for your replys. It looks like a main memory database is the best "built in" option. However, I have created Jar file to drop him/lib with a Java Singleton object...holding a map. That should be accessible across requests and sessions. The question is how to populate this just once upon start up? Perhaps I could do a job that would do that? Also I could memoize the variables in a global script. That way the expensive operation is only run the first time it is needed.
Any other suggestions welcome. Recommend that a standard built-in feature be added to handle these scenarios.
On Tue, Sep 5, 2017 at 1:33 AM Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr wrote:
To be confirmed : there is no 'start script' server option. I do manually create and populate the mainmem db in the dba query interface.
Best regards, Fabrice
-----Message d'origine----- De : Fabrice ETANCHAUD Envoyé : mardi 5 septembre 2017 09:29 À : 'Marco Lettere'; basex-talk@mailman.uni-konstanz.de Objet : RE: [basex-talk] Server Variables, cached vars, etc
Hi all,
Another solution is to share a main memory database, that behaves like a memory cache. In Client/Server mode, any main memory created by one client is available to all the other ones.
Best regards, Fabrice
-----Message d'origine----- De : basex-talk-bounces@mailman.uni-konstanz.de [mailto:basex-talk-bounces@mailman.uni-konstanz.de] De la part de Marco Lettere Envoyé : mardi 5 septembre 2017 09:14 À : basex-talk@mailman.uni-konstanz.de Objet : Re: [basex-talk] Server Variables, cached vars, etc
On 05/09/2017 01:37, Erik Peterson wrote:
How can I create a variable that is evaluated only once but accessed across many RestXQ requests and sessions. I'm trying to cache data that comes from an integration with an expensive operation. Does BaseX support something similar to server variables like Mark Logic?
Hi Erik,
AFAIK you have the following possibilities to keep a variable live accross multiple RestXQ calls:
Use session http://docs.basex.org/wiki/Session_Module
Use a database which is builtin in BaseX and is very lightweight.
Especially if your data is serializable to XML you could benefit also from indexes to speed up access to your cached objects.
- Use the file system.
Hope this helps [cit] ;-)
Marco.
--
Erik Peterson
President, Ardec LLC
281-804-9023 | ep@ardec.com
--
Erik Peterson
President, Ardec LLC
281-804-9023 | ep@ardec.com
Fabrice, Marco, thanks for your asseessment. After I have realized that it’ll take some more time to make this happen, I have added yet another issue for this feature request [1].
[1] https://github.com/BaseXdb/basex/issues/1499
On Mon, Sep 18, 2017 at 5:03 PM, Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr wrote:
Hello Christian !
Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection.
Best regards, Fabrice
…done [1]. A new 8.7 snapshot is available [1].
[1] http://docs.basex.org/wiki/Command-Line_Options#HTTP_Server [2] http://files.basex.org/releases/latest/
On Wed, Sep 20, 2017 at 7:37 PM, Christian Grün christian.gruen@gmail.com wrote:
Fabrice, Marco, thanks for your asseessment. After I have realized that it’ll take some more time to make this happen, I have added yet another issue for this feature request [1].
[1] https://github.com/BaseXdb/basex/issues/1499
On Mon, Sep 18, 2017 at 5:03 PM, Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr wrote:
Hello Christian !
Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection.
Best regards, Fabrice
Great! M.
On 21/09/2017 12:52, Christian Grün wrote:
…done [1]. A new 8.7 snapshot is available [1].
[1] http://docs.basex.org/wiki/Command-Line_Options#HTTP_Server [2] http://files.basex.org/releases/latest/
On Wed, Sep 20, 2017 at 7:37 PM, Christian Grün christian.gruen@gmail.com wrote:
Fabrice, Marco, thanks for your asseessment. After I have realized that it’ll take some more time to make this happen, I have added yet another issue for this feature request [1].
[1] https://github.com/BaseXdb/basex/issues/1499
On Mon, Sep 18, 2017 at 5:03 PM, Fabrice ETANCHAUD fetanchaud@pch.cerfrance.fr wrote:
Hello Christian !
Yes, a -c option for the basexhttp would help, as mentioned earlier, for example creating a shared mainmem collection.
Best regards, Fabrice
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Erik Peterson -
Fabrice ETANCHAUD -
Kendall Shaw -
Marco Lettere -
Xavier-Laurent SALVADOR