Christian,

I will explore those questions.

I did more work with prof:dump()—that seems to be the most effective way to narrow the point at which the problem starts.

Based on that work it appears I may be causing problems by trying to load large numbers of nodes using db:open-id(). Based on that I probably need to just rethink how I’m doing this particular process now that I have a better understand of how best to do things.

I also found putting a debugger break point on prof:dump() itself was useful—it at least lets me see where I am in the running code from within the debugger.

Cheers,

E.

_____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow

From: Christian Grün christian.gruen@gmail.com Date: Thursday, March 3, 2022 at 1:39 AM To: Eliot Kimber eliot.kimber@servicenow.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] How Best to Determine What BaseX Is Doing When It Pegs the Processor? [External Email]

Hi Eliot,

It’s hard to judge if the freeze-up is due to the query, RESTXQ or your general configuration. Let me see…

• Do you get the full result if you invoke the query in the GUI or on command line? How long does that take/how large is the result? • How do you send the HTTP request? Does it make a difference if you use the browser or cURL/Wget? • You could check if the execution time of your request is included in the database logs (e.g. via the DBA interface). If yes, the problem must occur after the actual query execution. • Do you get the execution time shown if you wrap your RESTXQ function body with prof:time?

Feel free to share a minimized example of the RESTXQ entry point with us (including the annotations you use).

Cheers (pun intended), Christian

On Wed, Mar 2, 2022 at 4:35 PM Eliot Kimber eliot.kimber@servicenow.com wrote:

I have a RESTXQ page that makes a query call to construct a report and then displays it. When I run this page something is triggering BaseX to peg the processor and it basically never returns from whatever it is doing.

Using proc:dump() I can see that the call to construct the report runs and returns the expected value—this takes about 3 seconds, and then the page never returns and top shows that the java process for the BaseX HTTP server is pegged at 100%.

The behavior is consistent so I know it must be something I’m doing or the effect of some configuration setting I have but no idea what might be going on. I get the behavior both on my development machine and on my production server, so it must be something in my code or some reproducible issue with the databases I’m managing.

I’ve tried connecting the BaseX HTTP server to the Eclipse debugger, which works, but I’m not sure where I should be focusing my attention in the code.

Turning on the -d flag isn’t that helpful either as the log is flooded with exceptions from BaseX trying to load Java classes for my XQuery modules, which swamps any more detailed debugging log messages and what messages I see appear to be entirely about the HTTP interaction, not what’s happening in the core BaseX server.

Is there a particular place in the code I can focus on in the interactive debugger that would be a likely source of the behavior or another debugging technique I haven’t considered?

Thanks,

E.

---

Eliot Kimber

Sr Staff Content Engineer

O: 512 554 9368

M: 512 554 9368

servicenow.com

LinkedIn | Twitter | YouTube | Facebook

It’s the second form that I have that seems to be the point where things hang up, meaning I see a prof:dump() message before that call and not the prof:dump() message that follows it, but I’m also passing in potentially 10s of 1000s of node IDs, which is probably not optimal.

I tried rewriting the code to do them individually but that didn’t really seem to help, which was not a surprise.

My current code is taking Oxygen validation reports, which consist of sets of “incidents”, where each incident is associated with a specific document. I have a separate index that associates documents to their containing “bundle” DITA maps. I then build a map of incident element node IDs to bundle names from which I can then build an HTML report of incidents grouped by bundle. So my code is iterating over the keys in this map (bundle names), getting the set of node IDs of incident elements, and then processing those incident elements to build the HTML report entries.

I now realize that this map-based approach is suboptimal and I should just build the map as a separate XML document that I can build in the background and persist along with my other indexes and then just render it. That is, whenever I ingest a validation report I can just build the bundle-to-incident document then and not try to reconstruct it when users request the report.

I’ll pursue this re-implementation.

Cheers,

E. _____________________________________________ Eliot Kimber Sr Staff Content Engineer O: 512 554 9368 M: 512 554 9368 servicenow.comhttps://www.servicenow.com LinkedInhttps://www.linkedin.com/company/servicenow | Twitterhttps://twitter.com/servicenow | YouTubehttps://www.youtube.com/user/servicenowinc | Facebookhttps://www.facebook.com/servicenow

From: Christian Grün christian.gruen@gmail.com Date: Thursday, March 3, 2022 at 9:16 AM To: Eliot Kimber eliot.kimber@servicenow.com Cc: basex-talk@mailman.uni-konstanz.de basex-talk@mailman.uni-konstanz.de Subject: Re: [basex-talk] How Best to Determine What BaseX Is Doing When It Pegs the Processor? [External Email]

Based on that work it appears I may be causing problems by trying to load large numbers of nodes using db:open-id().

If you have numerous single db:open-id calls, you could potentially reduce them to a single call:

db:open-id($db, $sequence-of-ids)