As discussed in a parallel thread, I’ve determined that the problem is caused by trying to transform a massive doc in the database using an XSLT transform that either never completes or fails in a way that is not reflected in the log.
I performed a full-scale test on my development server, meaning that this was the only process running on it. I could see through my server status dashboard that the process doing the XSLT was running and it took a *long* time to finish (at least 24 hours—I stopped watching it closely but checking it now it has completed successfully).
So the issue is with my attempt to transform a 300MB document, not an issue with BaseX itself.
The solution is clearly to not try to transform a 300MB document, which I’m pursing now.
I’ve yet to find a way, short of connecting BaseX to an interactive debugger, to get visibility into what it’s doing on the server—for example, I orchestrate my processes by creating a sequence of jobs but there doesn’t seem to be a way to see what job is running or get insight into what it’s actively doing.
I’m not sure it’s even possible to provide that kind of visibility but it does make knowing what’s happening a challenge. Of course I can use messages in my code to get some feedback in the log, and I do do that, but for just monitoring a running server it would be nice to have more visibility.
Cheers,
E.
_____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368
servicenow
servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Xhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Instagramhttps://www.instagram.com/servicenow
From: Christian Grün christian.gruen@gmail.com Date: Thursday, April 24, 2025 at 8:11 AM To: Eliot Kimber eliot.kimber@servicenow.com Cc: Andy Bunce bunce.andy@gmail.com, basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] Re: Recovering database in bad state [External Email]
________________________________ Hi Eliot, I’m sorry I cannot give a simple answer that leads to a positive result. Sometimes, the INSPECT command gives you some hints what may have gone wrong (but it doesn’t help you fix it). > I think this is because the database was being written to and something happened i This message needs your attention
* Someone new is on this email.
Provided by ServiceNow DT (Employee Portal KB0077950) - This banner is visible only to ServiceNow employees. CGBANNERINDICATOR Hi Eliot,
I’m sorry I cannot give a simple answer that leads to a positive result. Sometimes, the INSPECT command gives you some hints what may have gone wrong (but it doesn’t help you fix it).
I think this is because the database was being written to and something happened that left it in a bad state.
Can you recollect what happened?
Best, Christian
On Fri, Apr 18, 2025 at 10:31 PM Eliot Kimber via BaseX-Talk <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> wrote: This is with BaseX 11.6 on RHEL.
Cheers,
E.
_____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368
servicenow
servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Xhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Instagramhttps://www.instagram.com/servicenow
From: Eliot Kimber <eliot.kimber@servicenow.commailto:eliot.kimber@servicenow.com> Date: Friday, April 18, 2025 at 3:30 PM To: Andy Bunce <bunce.andy@gmail.commailto:bunce.andy@gmail.com> Cc: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Recovering database in bad state I tried deleting the udp.basex file but it appeared to have no effect, unfortunately.
Cheers,
E.
_____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368
servicenow
servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Xhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Instagramhttps://www.instagram.com/servicenow
From: Andy Bunce <bunce.andy@gmail.commailto:bunce.andy@gmail.com> Date: Friday, April 18, 2025 at 11:48 AM To: Eliot Kimber <eliot.kimber@servicenow.commailto:eliot.kimber@servicenow.com> Cc: basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> Subject: Re: [basex-talk] Recovering database in bad state [External Email]
________________________________ Hi Eliot,
That's not fun.
I see your directory has a ` upd.basex`you could try the suggestion at: https://docs.basex.org/main/Transaction_Management#update_operationshttps://docs.basex.org/main/Transaction_Management#update_operations
Maybe you get lucky. /Andy
On Fri, 18 Apr 2025 at 15:59, Eliot Kimber via BaseX-Talk <basex-talk@mailman.uni-konstanz.demailto:basex-talk@mailman.uni-konstanz.de> wrote: I have a database that shows having the expected 2 documents in it in the DBA databases list but when I go to the database in the DBA it shows 0 items.
I think this is because the database was being written to and something happened that left it in a bad state.
Is there any possibility of recovering this database so it is usable?
The practical problem is that it takes several hours to populate this database, so recovering it rather than attempting another multi-hour load would be nice.
Here’s what I see for files in the database’s directory on the server:
-rw-r--r-- 1 eliot.kimber.adm domain users 73039559 Apr 18 14:50 atv.basex -rw-r--r-- 1 eliot.kimber.adm domain users 7544382 Apr 18 14:52 atvl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 2730980 Apr 18 14:52 atvr.basex -rw-r--r-- 1 eliot.kimber.adm domain users 236 Apr 18 04:17 ftxx.basex -rw-r--r-- 1 eliot.kimber.adm domain users 218072 Apr 18 04:17 ftxy.basex -rw-r--r-- 1 eliot.kimber.adm domain users 1603678 Apr 18 04:17 ftxz.basex -rw-r--r-- 1 eliot.kimber.adm domain users 15 Apr 18 04:17 idp.basex -rw-r--r-- 1 eliot.kimber.adm domain users 15193 Apr 18 04:17 inf.basex -rw-r--r-- 1 eliot.kimber.adm domain users 67 Apr 18 04:17 pth.basex -rw-r--r-- 1 eliot.kimber.adm domain users 28 Apr 18 04:17 swl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 83701760 Apr 18 14:52 tbl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 9 Apr 18 04:17 tbli.basex -rw-r--r-- 1 eliot.kimber.adm domain users 112380335 Apr 18 14:52 tokl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 2829520 Apr 18 14:52 tokr.basex -rw-r--r-- 1 eliot.kimber.adm domain users 1718692 Apr 18 04:17 txt.basex -rw-r--r-- 1 eliot.kimber.adm domain users 351326 Apr 18 04:17 txtl.basex -rw-r--r-- 1 eliot.kimber.adm domain users 285610 Apr 18 02:38 txtr.basex -rw-r--r-- 1 eliot.kimber.adm domain users 0 Apr 18 04:18 upd.basex
In the Databases list I see: _temp_lrk_yokohama_link_recordshttp://mirabel.corp.service-now.com:8984/dba/database?name=_temp_lrk_yokohama_link_records 2 274 MB 2025-04-18, 14:52:42
But then when I go to the database itself it just say “0 entries”.
Thanks,
Eliot
_____________________________________________ Eliot Kimber Sr. Staff Content Engineer O: 512 554 9368
servicenow
servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Xhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Instagramhttps://www.instagram.com/servicenow