Let me tell you what I've done this far:

- attempt to decouple the libs as much as possible; - disabled the automatic parsing of XQuery files-flag; - only name the public files in file: expath-pkg; - annotate as much functions as private as possible;

Thanx for the offer. I'll send you the packages later.

Rob

-----Oorspronkelijk bericht----- Van: Christian Grün [mailto:christian.gruen@gmail.com] Verzonden: donderdag 10 maart 2016 10:53 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath

Hi Rob,

you could try to disable automatic parsing of XQuery files via the Preference Dialog (see screenshot). Does this help?

Apart from that, feel free to directly send me your sources, and I can have a second look.

Cheers, Christian

On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper r.stapper@lijbrandt.nl wrote:

Hi Christian,

New tools, new challenges.

I’ve rearranged my source-code in packages/libraries conform the eXPath-specs in BaseX now.

But since than the GUI-editor is becoming very slow, even unworkably slow.

What is causing this and can you give me any tips how speed up the GUI-editor?

I’ve got 7 libraries now. The one with the largest interface counts 11 entries in file:expath-pkg.xml, all public entries, the smallest library counts one entry.

TIA,

Rob

Hi Christian,

Thanx for having a closer look at it.

I have already abandoned the ex-path package mechanism, it just was to slow to work with on my computer. Now I can type away once again. Once in a while my computer has to take a breath, probably updating the project files, and I'll have to wait for a second, but this doesn't bother me. The problem with the ex-path mechanism was that I had to wait with every keystroke. It looks to me that the timing-mechanism when using of ex-path is different than without using ex-path.

So if you can do something about the timing, just like you proposed below, I could again use ex-path.

In the meanwhile I've been thinking about reorganizing my modules. Maybe it would be good practice if each package is accessed by just one interface module which addresses the other modules through a relative path. I haven't tested this

Let me know if you have figured something out, I'll be happy to test it for you.

Take care, Rob

-----Oorspronkelijk bericht----- Van: Christian Grün [mailto:christian.gruen@gmail.com] Verzonden: vrijdag 1 april 2016 14:06 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath

Hi Rob,

I managed to play around with BaseX on an older machine (Celeron, 1.3 GHz, 224 MB RAM), and I was surprised to see it worked pretty well. Maybe it was due to Windows XP, though, and the fact that it was a fresh installation.

I had a closer look at the files you sent to me, and the most noticeable delays I noticed happened when editing an XQuery file with numerous module imports. The reason for that is that all the imported modules need to be parsed with each key click.

Spontaneously, I got two ideas how this could be improved:

* XQuery parsing could be delayed for some milliseconds. This way, it would only be parsed if a user has (presumably) finished his input.

* An obsolete query parsing thread could be stopped before a new parsing thread is started.

* The least thrilling solution: A new option in the Preferences dialog to disable realtime parsing, and only parse if a file is saved.

I’ll see what can be done.

Cheers, Christian

On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper r.stapper@lijbrandt.nl wrote:

Hi Christian,

New tools, new challenges.

I’ve rearranged my source-code in packages/libraries conform the eXPath-specs in BaseX now.

But since than the GUI-editor is becoming very slow, even unworkably slow.

What is causing this and can you give me any tips how speed up the GUI-editor?

I’ve got 7 libraries now. The one with the largest interface counts 11 entries in file:expath-pkg.xml, all public entries, the smallest library counts one entry.

TIA,

Rob

Hi Christian,

Absolutely brilliant, no hick-ups what so ever. Double-checked it.

Have a goof weekend, Rob

-----Oorspronkelijk bericht----- Van: Christian Grün [mailto:christian.gruen@gmail.com] Verzonden: vrijdag 1 april 2016 15:47 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath

Hi Rob,

I think I’ve found a solution that does justice to both older and newer systems: query parsing will now be generally delayed for some ms), and only one query parsing process will be active at the same time (before, every key click triggered an extra thread).

Even if you have switched to another mechanism now, could you give it a try [1]?

Looking forward to your feedback, Christian

[1] http://files.basex.org/releases/latest

On Fri, Apr 1, 2016 at 3:11 PM, Rob Stapper r.stapper@lijbrandt.nl wrote:

Hi Christian,

Thanx for having a closer look at it.

I have already abandoned the ex-path package mechanism, it just was to slow to work with on my computer. Now I can type away once again. Once in a while my computer has to take a breath, probably updating the project files, and I'll have to wait for a second, but this doesn't bother me. The problem with the ex-path mechanism was that I had to wait with every keystroke. It looks to me that the timing-mechanism when using of ex-path is different than without using ex-path.

So if you can do something about the timing, just like you proposed below, I could again use ex-path.

In the meanwhile I've been thinking about reorganizing my modules. Maybe it would be good practice if each package is accessed by just one interface module which addresses the other modules through a relative path. I haven't tested this

Let me know if you have figured something out, I'll be happy to test it for you.

Take care, Rob

-----Oorspronkelijk bericht----- Van: Christian Grün [mailto:christian.gruen@gmail.com] Verzonden: vrijdag 1 april 2016 14:06 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath

Hi Rob,

I managed to play around with BaseX on an older machine (Celeron, 1.3 GHz, 224 MB RAM), and I was surprised to see it worked pretty well. Maybe it was due to Windows XP, though, and the fact that it was a fresh installation.

I had a closer look at the files you sent to me, and the most noticeable delays I noticed happened when editing an XQuery file with numerous module imports. The reason for that is that all the imported modules need to be parsed with each key click.

Spontaneously, I got two ideas how this could be improved:

• XQuery parsing could be delayed for some milliseconds. This way, it would only be parsed if a user has (presumably) finished his input.

• An obsolete query parsing thread could be stopped before a new parsing thread is started.

• The least thrilling solution: A new option in the Preferences dialog to disable realtime parsing, and only parse if a file is saved.

I’ll see what can be done.

Cheers, Christian

On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper r.stapper@lijbrandt.nl wrote:

...

Hi Christian,

New tools, new challenges.

I’ve rearranged my source-code in packages/libraries conform the eXPath-specs in BaseX now.

But since than the GUI-editor is becoming very slow, even unworkably slow.

What is causing this and can you give me any tips how speed up the GUI-editor?

I’ve got 7 libraries now. The one with the largest interface counts 11 entries in file:expath-pkg.xml, all public entries, the smallest library counts one entry.

TIA,

Rob

Hello -

I've got a problem I'm not sure how to best approach.

I've got triplets of names -- class.operation.specifier -- that I need to match against much longer sequences of names. (Which are in attributes in an XML hierarchy; each sequence of names derives from a path to a leaf element.)

If there is a match (as there usually is not) one of the names in the sequence of names will match to the class, a subsequent name to the operation, and a name subsequent to that match to the specifier. (All simple string values.)

The naive n^2 version is much too slow for the amount of data involved.

Is there an efficient way to do this kind of matching?

Thanks! Graydon

Hi Christian --

I can't give you a real example because it's the client's health care data, and I would be fed to the rippy fish if I posted any of it.

I've created an example with trivial data, zipped that up, and attached it. (Create a database from example.xml, and run threePartMatchesEG.xq against that DB.)

(I think) the difficult part is:

let $found := //*[@name eq $match(1)][./descendant::*[@name eq $match(2)][./descendant::*[@name eq $match(3)]]]

This works, but it's going over the entire database for every three part class-operation-specifier compound name. I can't shake the feeling that there's a more efficient way to do this, but I can't see what it might be.

Thanks! Graydon

On Fri, Apr 1, 2016 at 12:04 PM, Christian Grün christian.gruen@gmail.com wrote:

Hi Graydon,

Do you think there’d be a chance for us to get a minimized, self-contained example, which demonstrates the n^2 solution?

Thanks in advance, Christian

On Fri, Apr 1, 2016 at 5:24 PM, Graydon Saunders graydonish@gmail.com wrote:

...

Hello -

I've got a problem I'm not sure how to best approach.

I've got triplets of names -- class.operation.specifier -- that I need to match against much longer sequences of names. (Which are in attributes

in an

...

XML hierarchy; each sequence of names derives from a path to a leaf element.)

If there is a match (as there usually is not) one of the names in the sequence of names will match to the class, a subsequent name to the operation, and a name subsequent to that match to the specifier. (All simple string values.)

The naive n^2 version is much too slow for the amount of data involved.

Is there an efficient way to do this kind of matching?

Thanks! Graydon

Hi Christian --

Thank you, that was helpful! (And the faint squeaking noise of my brain expanding when I first read through it may stick, too. :)

The eventual business requirement wound up being something else entirely.

The overall goal is to alter some complex application files that happen to be expressed in XML, to avoid a (lengthy and expensive) need for someone to manually change all the files through the application. It's not an open file format. The actual alteration is being done with XSLT, but BaseX is being extremely useful when it comes to "failing fast" on possible approaches to figuring out what to change. Matching on composed names turned out not to work at all, but finding that out in a couple of days was much better than finding out after the XSLT produced gibberish.

Thanks! Graydon

On Mon, Apr 4, 2016 at 1:27 PM, Christian Grün christian.gruen@gmail.com wrote:

Hi Graydon,

...

I can't give you a real example because it's the client's health care

data,

No problem, your example looks fine.

...

let $found := //*[@name eq $match(1)][./descendant::*[@name eq $match(2)][./descendant::*[@name eq $match(3)]]]

Right. You could try to rewrite this for index access:

1. You’ll have to mark the generated arrays as string arrays:

   let $composedNames as array(xs:string) := for $x in $composed//composed return array { tokenize($x/string(),'.') }

2. You need to replace "eq" with "=", and you can simplify the

predicates a little:

let $found := //*[@name = $match(1)] [descendant::*/@name = $match(2)] [descendant::*/@name = $match(3)]

You indicated that you’ll have thousands of paths. How do they look like? Could you add some more examples (besides "class.operation.specifier")? Are some parts of the paths more specific than others? E.g...

A.A.A A.A.B A.A.C A.B.D A.B.E A.B.F ...

In this case, it could make sense to only look for the last path segment via the index. You could also try to group your results by the first segment, then do the search on the second segment, etc. See my attached query as example (I’m sure it needs to be revised to work properly, because I have only run it with your simple example file).

Does this help? Christian

...

This works, but it's going over the entire database for every three part class-operation-specifier compound name. I can't shake the feeling that there's a more efficient way to do this, but I can't see what it might

be.

...

Thanks! Graydon

On Fri, Apr 1, 2016 at 12:04 PM, Christian Grün <

christian.gruen@gmail.com>

...

wrote:

...

Hi Graydon,

Do you think there’d be a chance for us to get a minimized, self-contained example, which demonstrates the n^2 solution?

Thanks in advance, Christian

On Fri, Apr 1, 2016 at 5:24 PM, Graydon Saunders graydonish@gmail.com wrote:

...

Hello -

I've got a problem I'm not sure how to best approach.

I've got triplets of names -- class.operation.specifier -- that I need to match against much longer sequences of names. (Which are in attributes in an XML hierarchy; each sequence of names derives from a path to a leaf element.)

If there is a match (as there usually is not) one of the names in the sequence of names will match to the class, a subsequent name to the operation, and a name subsequent to that match to the specifier. (All simple string values.)

The naive n^2 version is much too slow for the amount of data

involved.

...

...

...

Is there an efficient way to do this kind of matching?

Thanks! Graydon