Dear all, has anyone experience with exploiting the persistent jobs (services) feature under Tomcat? We have installed services with the proper jobs.xml file. When running jobs:services() in the DBA after startup, we get the services listed. Nevertheless none of the XQuery scripts inside the jobs is executed at startup of the Tomcat. When running standalone with Jetty, the scripts are executed even if, in this case, we are getting empty results when running jobs:services() in the DBA after startup. Any clue on what we should double-check? Thanks, Marco.
Hi Marco, I see your question didn’t get an answer yet. Maybe we can first try to find out what Jetty does (or does not):
When running standalone with Jetty, the scripts are executed even if, in this case, we are getting empty results when running jobs:services() in the DBA after startup.
Some questions back: 1. By 'standalone', do you mean that you are running 'basexhttp', or do you run Jetty and deploy BaseX as WAR file? 2. How does the jobs.xml file in the database directory look like? Cheers, Christian
Hi Christian,
thank you as usual for your attention.
With standalone I mean basexhttp which is what we usually do. This time
we have requirement for running inside tomcat...
The jobs.xml file looks like:
<jobs>
<job id="startup">if (db:exists('onedb')) then () else
db:create('ondedb')</job>
</jobs>
but we have also tried with:
<jobs>
<job id="startup">admin:write-log(current-dateTime())</job>
</jobs>
which presents the exact same issues: basexhttp runs ok but
jobs:services() doesn't list them whereas tomcat doesn't run the scripts
but services are listed.
Thank you again,
Marco.
On 15/10/2018 15:25, Christian Grün wrote:
> Hi Marco,
>
> I see your question didn’t get an answer yet. Maybe we can first try
> to find out what Jetty does (or does not):
>
>> When running standalone with Jetty, the scripts are executed even if, in
>> this case, we are getting empty results when running jobs:services() in
>> the DBA after startup.
> Some questions back:
> 1. By 'standalone', do you mean that you are running 'basexhttp', or
> do you run Jetty and deploy BaseX as WAR file?
> 2. How does the jobs.xml file in the database directory look like?
>
> Cheers,
> Christian
Hi Marco,
I tried to reproduce the problem:
1. I downloaded http://files.basex.org/releases/9.0.2/BaseX902.zip
2. I unzipped the archive
3. I added a jobs.xml file in the basex/data directory:
<jobs>
<job>admin:write-log('!')</job>
</jobs>
4. I started basexhttp
5. As expected, the log entry is listed in the DBA logs panel.
6. jobs:services() returns <job>admin:write-log('!')</job>
Could you do the same and report back to us what you did differently?
Thanks in advance
Christian
> Hi Christian,
> thank you as usual for your attention.
> With standalone I mean basexhttp which is what we usually do. This time
> we have requirement for running inside tomcat...
> The jobs.xml file looks like:
>
> <jobs>
> <job id="startup">if (db:exists('onedb')) then () else
> db:create('ondedb')</job>
> </jobs>
>
> but we have also tried with:
>
> <jobs>
> <job id="startup">admin:write-log(current-dateTime())</job>
> </jobs>
>
> which presents the exact same issues: basexhttp runs ok but
> jobs:services() doesn't list them whereas tomcat doesn't run the scripts
> but services are listed.
> Thank you again,
> Marco.
>
> On 15/10/2018 15:25, Christian Grün wrote:
> > Hi Marco,
> >
> > I see your question didn’t get an answer yet. Maybe we can first try
> > to find out what Jetty does (or does not):
> >
> >> When running standalone with Jetty, the scripts are executed even if, in
> >> this case, we are getting empty results when running jobs:services() in
> >> the DBA after startup.
> > Some questions back:
> > 1. By 'standalone', do you mean that you are running 'basexhttp', or
> > do you run Jetty and deploy BaseX as WAR file?
> > 2. How does the jobs.xml file in the database directory look like?
> >
> > Cheers,
> > Christian
>
>
Ok, actually on my personal PC jobs are correctly listed also with
basexhttp (need to investigate why on my colleagues PC this doesn't work).
Back to the more important issue...
I put the following jobs.xml[1] file in the data folder and get [2] as
first lines of log in dba after reboot.
I put the same file into the "dbpath" folder of my app running as tomcat
webapp but the log looks like [3].
The feeling is that services are scheduled only in accordance to the
startup of the http server which doesn't occur when running inside Tomcat.
In general we think that when working with tomcat one usually has to
link an external dbpath in order to avoid getting his data removed when
redeploying the webapp. The requirement to put jobs.xml inside the
datapath somehow breaks the atomicity of a deployable webapp.
A workaround could be to have the jobs.xml somehow redefined as a
web.xml variable. Might this be possible?
Thanks for your support.
M.
[1]
<jobs>
<job>admin:write-log('!')</job>
<job id="startup" >admin:write-log("1=1")</job>
</jobs>
[2]
17:39:05.100 0:0:0:0:0:0:0:1:42740 admin 302 745.90 [GET]
http://localhost:8984/dba
17:39:02.715 SERVER admin INFO NaN !
17:39:02.715 SERVER admin INFO NaN 1=1
17:39:02.468 SERVER admin OK NaN HTTP Server was started (port: 8984).
17:39:02.454 SERVER admin OK NaN Server was started (port: 1984).
[3]
17:29:04.933 0:0:0:0:0:0:0:1:52332 admin 302 1497.81 [GET]
http://localhost:8081/contract-repository/dba
17:28:43.218 SERVER admin OK NaN Server was started (port: 1984).
On 15/10/2018 16:59, Christian Grün wrote:
> Hi Marco,
>
> I tried to reproduce the problem:
>
> 1. I downloaded http://files.basex.org/releases/9.0.2/BaseX902.zip
> 2. I unzipped the archive
> 3. I added a jobs.xml file in the basex/data directory:
>
> <jobs>
> <job>admin:write-log('!')</job>
> </jobs>
>
> 4. I started basexhttp
> 5. As expected, the log entry is listed in the DBA logs panel.
> 6. jobs:services() returns <job>admin:write-log('!')</job>
>
> Could you do the same and report back to us what you did differently?
>
> Thanks in advance
> Christian
>
>
>
>
>> Hi Christian,
>> thank you as usual for your attention.
>> With standalone I mean basexhttp which is what we usually do. This time
>> we have requirement for running inside tomcat...
>> The jobs.xml file looks like:
>>
>> <jobs>
>> <job id="startup">if (db:exists('onedb')) then () else
>> db:create('ondedb')</job>
>> </jobs>
>>
>> but we have also tried with:
>>
>> <jobs>
>> <job id="startup">admin:write-log(current-dateTime())</job>
>> </jobs>
>>
>> which presents the exact same issues: basexhttp runs ok but
>> jobs:services() doesn't list them whereas tomcat doesn't run the scripts
>> but services are listed.
>> Thank you again,
>> Marco.
>>
>> On 15/10/2018 15:25, Christian Grün wrote:
>>> Hi Marco,
>>>
>>> I see your question didn’t get an answer yet. Maybe we can first try
>>> to find out what Jetty does (or does not):
>>>
>>>> When running standalone with Jetty, the scripts are executed even if, in
>>>> this case, we are getting empty results when running jobs:services() in
>>>> the DBA after startup.
>>> Some questions back:
>>> 1. By 'standalone', do you mean that you are running 'basexhttp', or
>>> do you run Jetty and deploy BaseX as WAR file?
>>> 2. How does the jobs.xml file in the database directory look like?
>>>
>>> Cheers,
>>> Christian
>>
Thanks for your summary on the behavior of Tomcat.
In general we think that when working with tomcat one usually has to link an external dbpath in order to avoid getting his data removed when redeploying the webapp. The requirement to put jobs.xml inside the datapath somehow breaks the atomicity of a deployable webapp. A workaround could be to have the jobs.xml somehow redefined as a web.xml variable. Might this be possible?
To understand your use case better: 1. Do you currently define an external database path when working with Tomcat? 2. Would you prefer to have a servlet-specific jobs.xml file and an external database directory, or should both of them be placed internally or externally? Just a guess, but maybe it would generally be a good idea to be able to define an external database directory for Tomcat? Or would you indeed like to have servlet-specific databases removod if On Mon, Oct 15, 2018 at 5:47 PM Marco Lettere <m.lettere@gmail.com> wrote:
Ok, actually on my personal PC jobs are correctly listed also with basexhttp (need to investigate why on my colleagues PC this doesn't work).
Back to the more important issue... I put the following jobs.xml[1] file in the data folder and get [2] as first lines of log in dba after reboot. I put the same file into the "dbpath" folder of my app running as tomcat webapp but the log looks like [3].
The feeling is that services are scheduled only in accordance to the startup of the http server which doesn't occur when running inside Tomcat.
In general we think that when working with tomcat one usually has to link an external dbpath in order to avoid getting his data removed when redeploying the webapp. The requirement to put jobs.xml inside the datapath somehow breaks the atomicity of a deployable webapp. A workaround could be to have the jobs.xml somehow redefined as a web.xml variable. Might this be possible?
Thanks for your support. M.
[1] <jobs> <job>admin:write-log('!')</job> <job id="startup" >admin:write-log("1=1")</job> </jobs>
[2] 17:39:05.100 0:0:0:0:0:0:0:1:42740 admin 302 745.90 [GET] http://localhost:8984/dba 17:39:02.715 SERVER admin INFO NaN ! 17:39:02.715 SERVER admin INFO NaN 1=1 17:39:02.468 SERVER admin OK NaN HTTP Server was started (port: 8984). 17:39:02.454 SERVER admin OK NaN Server was started (port: 1984). [3] 17:29:04.933 0:0:0:0:0:0:0:1:52332 admin 302 1497.81 [GET] http://localhost:8081/contract-repository/dba 17:28:43.218 SERVER admin OK NaN Server was started (port: 1984).
On 15/10/2018 16:59, Christian Grün wrote:
Hi Marco,
I tried to reproduce the problem:
1. I downloaded http://files.basex.org/releases/9.0.2/BaseX902.zip 2. I unzipped the archive 3. I added a jobs.xml file in the basex/data directory:
<jobs> <job>admin:write-log('!')</job> </jobs>
4. I started basexhttp 5. As expected, the log entry is listed in the DBA logs panel. 6. jobs:services() returns <job>admin:write-log('!')</job>
Could you do the same and report back to us what you did differently?
Thanks in advance Christian
Hi Christian, thank you as usual for your attention. With standalone I mean basexhttp which is what we usually do. This time we have requirement for running inside tomcat... The jobs.xml file looks like:
<jobs> <job id="startup">if (db:exists('onedb')) then () else db:create('ondedb')</job> </jobs>
but we have also tried with:
<jobs> <job id="startup">admin:write-log(current-dateTime())</job> </jobs>
which presents the exact same issues: basexhttp runs ok but jobs:services() doesn't list them whereas tomcat doesn't run the scripts but services are listed. Thank you again, Marco.
On 15/10/2018 15:25, Christian Grün wrote:
Hi Marco,
I see your question didn’t get an answer yet. Maybe we can first try to find out what Jetty does (or does not):
When running standalone with Jetty, the scripts are executed even if, in this case, we are getting empty results when running jobs:services() in the DBA after startup.
Some questions back: 1. By 'standalone', do you mean that you are running 'basexhttp', or do you run Jetty and deploy BaseX as WAR file? 2. How does the jobs.xml file in the database directory look like?
Cheers, Christian
On 15/10/2018 17:56, Christian Grün wrote:
Thanks for your summary on the behavior of Tomcat.
In general we think that when working with tomcat one usually has to link an external dbpath in order to avoid getting his data removed when redeploying the webapp. The requirement to put jobs.xml inside the datapath somehow breaks the atomicity of a deployable webapp. A workaround could be to have the jobs.xml somehow redefined as a web.xml variable. Might this be possible?
To understand your use case better: 1. Do you currently define an external database path when working with Tomcat? Yes. It's necessary. Otherwise, for each redeploy of the webapp, the data is lost. 2. Would you prefer to have a servlet-specific jobs.xml file and an external database directory, or should both of them be placed internally or externally? I think there is nothing against how dbpath is handled now. One has just to uncomment the dbpath variable inside the web.xml of the webapps that require persistent data and change it. That's good enough for us. At the same time the unfortunate consequence is that jobs.xml, which to our feeling is something that lives with code and not with data, has to be copied manually to the external dbpath in scenarios where you redeploy a webapp after some fixes or tuning. So our suggestion is to have something like a "jobs" variable in web.xml that points to an servlet internal jobs.xml (maybe in the WEB-INF folder?) to be used instead of the default one.
In any case I don't know if this solves the current misbehaviour we have with tomcat which simply and rudely ignores the content of jobs.xml altogether. I hope things are slightly clearer now. M.
Just a guess, but maybe it would generally be a good idea to be able to define an external database directory for Tomcat? Or would you indeed like to have servlet-specific databases removod if
On Mon, Oct 15, 2018 at 5:47 PM Marco Lettere <m.lettere@gmail.com <mailto:m.lettere@gmail.com>> wrote:
Ok, actually on my personal PC jobs are correctly listed also with basexhttp (need to investigate why on my colleagues PC this doesn't work).
Back to the more important issue... I put the following jobs.xml[1] file in the data folder and get [2] as first lines of log in dba after reboot. I put the same file into the "dbpath" folder of my app running as tomcat webapp but the log looks like [3].
The feeling is that services are scheduled only in accordance to the startup of the http server which doesn't occur when running inside Tomcat.
In general we think that when working with tomcat one usually has to link an external dbpath in order to avoid getting his data removed when redeploying the webapp. The requirement to put jobs.xml inside the datapath somehow breaks the atomicity of a deployable webapp. A workaround could be to have the jobs.xml somehow redefined as a web.xml variable. Might this be possible?
Thanks for your support. M.
[1] <jobs> <job>admin:write-log('!')</job> <job id="startup" >admin:write-log("1=1")</job> </jobs>
[2] 17:39:05.100 0:0:0:0:0:0:0:1:42740 admin 302 745.90 [GET] http://localhost:8984/dba 17:39:02.715 SERVER admin INFO NaN ! 17:39:02.715 SERVER admin INFO NaN 1=1 17:39:02.468 SERVER admin OK NaN HTTP Server was started (port: 8984). 17:39:02.454 SERVER admin OK NaN Server was started (port: 1984).
[3] 17:29:04.933 0:0:0:0:0:0:0:1:52332 admin 302 1497.81 [GET] http://localhost:8081/contract-repository/dba 17:28:43.218 SERVER admin OK NaN Server was started (port: 1984).
On 15/10/2018 16:59, Christian Grün wrote:
Hi Marco,
I tried to reproduce the problem:
1. I downloadedhttp://files.basex.org/releases/9.0.2/BaseX902.zip 2. I unzipped the archive 3. I added a jobs.xml file in the basex/data directory:
<jobs> <job>admin:write-log('!')</job> </jobs>
4. I started basexhttp 5. As expected, the log entry is listed in the DBA logs panel. 6. jobs:services() returns <job>admin:write-log('!')</job>
Could you do the same and report back to us what you did differently?
Thanks in advance Christian
Hi Christian, thank you as usual for your attention. With standalone I mean basexhttp which is what we usually do. This time we have requirement for running inside tomcat... The jobs.xml file looks like:
<jobs> <job id="startup">if (db:exists('onedb')) then () else db:create('ondedb')</job> </jobs>
but we have also tried with:
<jobs> <job id="startup">admin:write-log(current-dateTime())</job> </jobs>
which presents the exact same issues: basexhttp runs ok but jobs:services() doesn't list them whereas tomcat doesn't run the scripts but services are listed. Thank you again, Marco.
On 15/10/2018 15:25, Christian Grün wrote:
Hi Marco,
I see your question didn’t get an answer yet. Maybe we can first try to find out what Jetty does (or does not):
When running standalone with Jetty, the scripts are executed even if, in this case, we are getting empty results when running jobs:services() in the DBA after startup.
Some questions back: 1. By 'standalone', do you mean that you are running 'basexhttp', or do you run Jetty and deploy BaseX as WAR file? 2. How does the jobs.xml file in the database directory look like?
Cheers, Christian
At the same time the unfortunate consequence is that jobs.xml, which to our feeling is something that lives with code and not with data, has to be copied manually to the external dbpath in scenarios where you redeploy a webapp after some fixes or tuning. So our suggestion is to have something like a "jobs" variable in web.xml that points to an servlet internal jobs.xml (maybe in the WEB-INF folder?) to be used instead of the default one.
I see! We’ll keep this in mind and think about some alternatives.
Hi all, I've been using BaseX for a couple of years now for a quite well visited assesment site, where the role of BaseX is to store and query the assessment results. The site functioned very well last year. This summer I upgraded to BaseX 9.2, and since then I see BaseX hanging on a request every once in a while, causing the assessment site to not respond at all anymore. In the log I see entries without further information, except 'Digest authentication expected', and with fast increasing response times, see attached picture. I know this is very little information, but does anyone have an idea what might be causing this? And what to do about those 'Digest authentication expected' messages? I DID configure BaseX for Digest authentication, and as far as I know I am using it for every request. Paul
Hi Paul, . This summer I upgraded to BaseX
9.2, and since then I see BaseX hanging on a request every once in a while, causing the assessment site to not respond at all anymore.
So this means that some of your request with digest authentication work, and others don’t? In the log I see entries without further information, except 'Digest
authentication expected', and with fast increasing response times, see attached picture.
Could you give us more information how your requests look like? Are they all sent from the browser, or from any other system? And what to do about those 'Digest authentication expected' messages? I
DID configure BaseX for Digest authentication, and as far as I know I am using it for every request.
Do you have more details on the requests that cause this error message? Cheers Christian
participants (3)
-
Christian Grün -
Marco Lettere -
Paul Swennenhuis