Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
*string-join*(*for **$name* *in **request:parameter-names*() *return **$name* || ': ' || *string-join*(*request:parameter*(*$name*), '; '), ' ')
In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed and the values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1] http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht... [2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
*string-join*(*for **$name* *in **request:parameter-names*() *return **$name* || ': ' || *string-join*(*request:parameter*(*$name*), '; '), ' ')
In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi France,
I'm wondering what you mean with tracking. Is it to gather diagnostics during development? Is it for adding logging or is it part of running production code? Do you want to add this to every function or only a few specific ones?
Maybe some kind tracing/profiling mechanism could provide this without altering your running code. But I'm not sure if that's what you are after.
If this tracking is a real feature of your code then it's different. I hope your request is not because your functions have a lot of arguments. That would be an indication that your code needs some refactoring :)
Could you tell us what you mean with tracking capabilities?
--Marc
On 18 aug. 2014, at 21:39, Andy Bunce bunce.andy@gmail.com wrote:
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed and the values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1] http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht... [2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com wrote: Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '), ' ')
In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Some time ago, Charles Foster had the idea that all bound variables should be added to the stack trace whenever an error occurs [1]; would that be helpful? Before implementing this, however, we would need to think of various issues related to performance (try/catch statements will get slower when variables are bound to the stack every time; large XML snippets that will not be garbage-colected when being bound to an error; database references are not available anymore when an error is raised, etc.).
[1] https://github.com/BaseXdb/basex/issues/831
On Mon, Aug 18, 2014 at 9:39 PM, Andy Bunce bunce.andy@gmail.com wrote:
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed and the values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1] http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht... [2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
If that means that I may get thousands of XML files in the trace when batch processing, it seems I would get so much information that what ever is relevant would get lost in the mess. It may be useful for other use cases. I'll let other people debate their case about adding that much info to the trace.
We would be concerned if it ended up having an impact on performance. We worked hard to improve on that front.
On Thu, Aug 21, 2014 at 6:16 AM, Christian Grün christian.gruen@gmail.com wrote:
Some time ago, Charles Foster had the idea that all bound variables should be added to the stack trace whenever an error occurs [1]; would that be helpful? Before implementing this, however, we would need to think of various issues related to performance (try/catch statements will get slower when variables are bound to the stack every time; large XML snippets that will not be garbage-colected when being bound to an error; database references are not available anymore when an error is raised, etc.).
[1] https://github.com/BaseXdb/basex/issues/831
On Mon, Aug 18, 2014 at 9:39 PM, Andy Bunce bunce.andy@gmail.com wrote:
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed and
the
values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1]
http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht...
[2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com
wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to
find a
function that would do the equivalent of what the code bellow does, but
for
internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
If that means that I may get thousands of XML files in the trace when batch processing, it seems I would get so much information that what ever is relevant would get lost in the mess.
It would mean that all variables would be bound to a map in an additional error variable (e.g. $err:variables), and you could freely choose the information that seems relevant to you.
We would be concerned if it ended up having an impact on performance. We worked hard to improve on that front.
Sure; we'd only add such a functionality if it won't have negative effects on performance.
Regarding your first mail in this thread, I'm probably still not sure what the extension you are asking for should do, and how it should look like. If you have a concrete idea how the function could be called, or could look like, feel free to propose it here.
Christian
On Thu, Aug 21, 2014 at 6:16 AM, Christian Grün christian.gruen@gmail.com wrote:
Some time ago, Charles Foster had the idea that all bound variables should be added to the stack trace whenever an error occurs [1]; would that be helpful? Before implementing this, however, we would need to think of various issues related to performance (try/catch statements will get slower when variables are bound to the stack every time; large XML snippets that will not be garbage-colected when being bound to an error; database references are not available anymore when an error is raised, etc.).
[1] https://github.com/BaseXdb/basex/issues/831
On Mon, Aug 18, 2014 at 9:39 PM, Andy Bunce bunce.andy@gmail.com wrote:
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed and the values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1]
http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht... [2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi France,
It has been a few days. I'm sorry if in my last response I jumped to conclusions about your code too fast and went into a kind of sermon on software development. I couldn't draw these conclusions from the code you shown. Only defense I have is that shortly after I fell ill and was off-line for a most of the week, so maybe I was already affected ;-)
I'm still not sure if what you're asking for is purely for debugging so you can fix the root cause in the code, or if this is an application issue/exception in the processed files that you want to handle within your normal application logic. Some of your comments suggest the latter whereas traceback stuff etc. is meant for the former.
Regarding the tracebacks: I think that if performance is too much affected by this trace with bound variables it should definitely be an option that you can switch off. Also wouldn't it help (or maybe it amounts to the same, I don't know) to have an option to do dumps of bound variables and possible traceback info to a file whenever something goes awry.
--Marc
On Thu, Aug 21, 2014 at 3:55 PM, Christian Grün christian.gruen@gmail.com wrote:
If that means that I may get thousands of XML files in the trace when batch processing, it seems I would get so much information that what ever is relevant would get lost in the mess.
It would mean that all variables would be bound to a map in an additional error variable (e.g. $err:variables), and you could freely choose the information that seems relevant to you.
We would be concerned if it ended up having an impact on performance. We worked hard to improve on that front.
Sure; we'd only add such a functionality if it won't have negative effects on performance.
Regarding your first mail in this thread, I'm probably still not sure what the extension you are asking for should do, and how it should look like. If you have a concrete idea how the function could be called, or could look like, feel free to propose it here.
Christian
On Thu, Aug 21, 2014 at 6:16 AM, Christian Grün christian.gruen@gmail.com wrote:
Some time ago, Charles Foster had the idea that all bound variables should be added to the stack trace whenever an error occurs [1]; would that be helpful? Before implementing this, however, we would need to think of various issues related to performance (try/catch statements will get slower when variables are bound to the stack every time; large XML snippets that will not be garbage-colected when being bound to an error; database references are not available anymore when an error is raised, etc.).
[1] https://github.com/BaseXdb/basex/issues/831
On Mon, Aug 18, 2014 at 9:39 PM, Andy Bunce bunce.andy@gmail.com wrote:
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed and the values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1]
http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht... [2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi,
No offense taken about the code. Feedback is how we improve.
The main purpose of tracking is actually to find what triggers the issues that are out of our hands. As our system grows more complex, it becomes very hard to build small examples for reporting bugs to the BaseX team. On the last few attempts, it took so long to figure out a repeatable pattern that we ended up not reporting the issues and getting fixes through upgrades anyway.
We identify the source of issues 2 ways: * Remove files, until we figure out the characteristics of the files that get corrupted. Past examples were: attribute at the root of a file or attribute whose namespace was indicated by a prefix, like xml:lang. * Remove processes until we no longer get the issue.
For multiple reasons different files are routed through different processes within the workflow. Being able to track that will help us go through the above process of building a small case much faster.
We have added a switch to turn tracking on/off, but for long term issue, we'd like to keep it on as often as possible. We got another corrupted DB this week and we can't know just from the latest operation how that happened. With all other language DB uncorrupted, if we had a trace of operations we could identify what was done to that DB that never happened to any of the non-corrupted others.
When corruption is identified, it's usually not the latest step that created the corruption, but a previous step that left something in a 'strange' state. We only identify the issue when we try to apply a subsequent update to a file in one of these 'strange' state. This is what we are trying to figure out with tracking, we want to get to the chain of events.
On Tue, Aug 26, 2014 at 3:08 AM, Marc van Grootel < marc.van.grootel@gmail.com> wrote:
Hi France,
It has been a few days. I'm sorry if in my last response I jumped to conclusions about your code too fast and went into a kind of sermon on software development. I couldn't draw these conclusions from the code you shown. Only defense I have is that shortly after I fell ill and was off-line for a most of the week, so maybe I was already affected ;-)
I'm still not sure if what you're asking for is purely for debugging so you can fix the root cause in the code, or if this is an application issue/exception in the processed files that you want to handle within your normal application logic. Some of your comments suggest the latter whereas traceback stuff etc. is meant for the former.
Regarding the tracebacks: I think that if performance is too much affected by this trace with bound variables it should definitely be an option that you can switch off. Also wouldn't it help (or maybe it amounts to the same, I don't know) to have an option to do dumps of bound variables and possible traceback info to a file whenever something goes awry.
--Marc
On Thu, Aug 21, 2014 at 3:55 PM, Christian Grün christian.gruen@gmail.com wrote:
If that means that I may get thousands of XML files in the trace when
batch processing, it seems I would get so much information that what ever is relevant would get lost in the mess.
It would mean that all variables would be bound to a map in an additional error variable (e.g. $err:variables), and you could freely choose the information that seems relevant to you.
We would be concerned if it ended up having an impact on performance. We worked hard to improve on that front.
Sure; we'd only add such a functionality if it won't have negative effects on performance.
Regarding your first mail in this thread, I'm probably still not sure what the extension you are asking for should do, and how it should look like. If you have a concrete idea how the function could be called, or could look like, feel free to propose it here.
Christian
On Thu, Aug 21, 2014 at 6:16 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Some time ago, Charles Foster had the idea that all bound variables should be added to the stack trace whenever an error occurs [1]; would that be helpful? Before implementing this, however, we would need to think of various issues related to performance (try/catch statements will get slower when variables are bound to the stack every time; large XML snippets that will not be garbage-colected when being bound to an error; database references are not available anymore when an error is raised, etc.).
[1] https://github.com/BaseXdb/basex/issues/831
On Mon, Aug 18, 2014 at 9:39 PM, Andy Bunce bunce.andy@gmail.com
wrote:
Hi France,
I think this is not possible, but I think it would be a good thing! I think it is related to Marc's query about call an anonymous function with a variable argument list [1]
XQuery provides no access to the stack. But..I can see that one could implement a new function maybe in the BaseX profiling module [2] that returns a map. The keys would be the argument names of the function being executed
and
the values those of the current arguments.
All that is required is to implement it :-)
/Andy
[1]
http://www.mail-archive.com/basex-talk%40mailman.uni-konstanz.de/msg04539.ht...
[2] http://docs.basex.org/wiki/Profiling_Module
On 18 August 2014 18:14, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does,
but
for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' ||
string-join(request:parameter($name),
'; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- --Marc
Hi France,
I would also be interested if you'd like a functionality as proposed by Andy, and if you are interested in improving debugging your XQuery code, or if you think that such a function could also be helpful in productive code? Do you have some more input?
Thanks, Christian
On Mon, Aug 18, 2014 at 7:14 PM, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi,
I am hoping to use this for debugging. Our project is fairly complex and I use a controller function to apply multiple operations to content (each of these may use one or more function). When I get a 500 error, it's very hard for me to pinpoint the source of the issue.
I can get the data from the form filled by the user and a few of the redirect, but that only gets me so far, if I could track both the function names and the parameters for sub-functions, it would save me a lot of time.
Below is an example of what I call a controller... I use those when I need content to be saved before I can query it again for the next updating operation. Each if the workflow step uses multiple sub functions that are not accessed through http and that are as far as I know untraceable.
(: Transforms content to xliff for each language and export packages for translation vendors :)
*declare* %rest:path('/translation/create-package') %rest:GET %rest:query-param('menu-id', '{$menu-id}', '') %rest:query-param('lang', '{$lang-in}', '') (: The step that we were at when the function was called :) %rest:query-param('step-id', '{$step-id}', '0') %rest:query-param('package-number-in', '{$package-number-in}', '') %output:method("html") %output:html-version("5.0") *updating* *function* *translation:create-package-controller*(*$menu-id* *as **xs:string*, *$lang-in* *as **xs:string**, *$package-number-in* *as **xs:string*, *$step-id* *as * *xs:string*){
...
*let* *$workflow* := <steps> <step *id*=*"1"* *name*=*"Add-ids"*>Add @ids for en-us</step> <step *id*=*"2"* *name*=*"Health-check"*>DB health check for en-us</step> <!--step id="3" name="Check-keywords">Check for keywords that may need to be replaced by prompts</step--> <step *id*=*"3"* *name*=*"Process-to-xliff"*>Process files to skeletons and xliff</step> <step *id*=*"4"* *name*=*"Register-packages"*>Register new packages in translation-management.xml... before deleting files with no new segments.</step> <step *id*=*"5"* *name*=*"Old-content-to-target"*>Process files in no-new-segment (could have different attributes or deleted content) to target lang.</step> <step *id*=*"6"* *name*=*"Delete-no-new-segment"*>Deletes no-new-segment xliffs that were processed in step Old-content-to-target. </step> <step *id*=*"7"* *name*=*"Set-target-status"*>Sets updatibility of target topic</step> <step *id*=*"8"* *name*=*"Export-and-zip"*>Export and zip files</step> </steps>
...
* switch* (*$workflow*//*step*[*@id*=*$this-step*]/*data*(*@name*)) *case *'Add-ids' *return *(:let $debug := file:append('debug-create-pack-crash.xml', ' In Add-ids '):) *let **$t* := '' *return *(*admin:add-ids*('en-us'), *db:output*( *$next-step-forward*))
...
*case *'Set-target-status' *return *(:let $debug := file:append('debug-create-pack-crash.xml', ' In Set-target-status '):) *let **$t* := ''
*return *(*for **$lg* *in * *$lang* *return **translation:toggle-target-status*(*$lg*), *db:output*( *$next-step-forward*))
....
*case *'Export-and-zip' *return *(*let **$zip* := *translation:export-and-zip*(*$menu-id*, *$lang*, *$package-number*) (:let $debug := file:append('debug-create-pack-crash.xml', ' In Export-and-zip '):) *return *((), *db:output*( *$next-step-forward*)) ) *default* *return *((), *db:output*( *admin:display-in-dashboard*(<div><div *class*=*"l2"* *style*=*"color:red;"*
An error has occured, you should have been redirected to the export
function before reaching this point. Default in switch steps.</div>{ *app:get-beep-div-for-dashboard*('error')}</div>)))
};
On Tue, Aug 19, 2014 at 10:26 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi France,
I would also be interested if you'd like a functionality as proposed by Andy, and if you are interested in improving debugging your XQuery code, or if you think that such a function could also be helpful in productive code? Do you have some more input?
Thanks, Christian
On Mon, Aug 18, 2014 at 7:14 PM, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find
a
function that would do the equivalent of what the code bellow does, but
for
internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
If the goal is debugging .. another approach might be to think about Aspect Oriented Programming something like http://www.mkyong.com/spring3/spring-aop-aspectj-annotation-example/
/Andy
On 19 August 2014 18:05, France Baril france.baril@architextus.com wrote:
Hi,
I am hoping to use this for debugging. Our project is fairly complex and I use a controller function to apply multiple operations to content (each of these may use one or more function). When I get a 500 error, it's very hard for me to pinpoint the source of the issue.
I can get the data from the form filled by the user and a few of the redirect, but that only gets me so far, if I could track both the function names and the parameters for sub-functions, it would save me a lot of time.
Below is an example of what I call a controller... I use those when I need content to be saved before I can query it again for the next updating operation. Each if the workflow step uses multiple sub functions that are not accessed through http and that are as far as I know untraceable.
(: Transforms content to xliff for each language and export packages for translation vendors :)
*declare* %rest:path('/translation/create-package') %rest:GET %rest:query-param('menu-id', '{$menu-id}', '') %rest:query-param('lang', '{$lang-in}', '') (: The step that we were at when the function was called :) %rest:query-param('step-id', '{$step-id}', '0') %rest:query-param('package-number-in', '{$package-number-in}', '') %output:method("html") %output:html-version("5.0") *updating* *function* *translation:create-package-controller*(*$menu-id* *as **xs:string*, *$lang-in* *as **xs:string**, *$package-number-in* *as **xs:string*, *$step-id* *as **xs:string*){
...
*let* *$workflow* := <steps> <step *id*=*"1"* *name*=*"Add-ids"*>Add @ids for en-us</step> <step *id*=*"2"* *name*=*"Health-check"*>DB health check for en-us</step> <!--step id="3" name="Check-keywords">Check for keywords that may need to be replaced by prompts</step--> <step *id*=*"3"* *name*=*"Process-to-xliff"*>Process files to skeletons and xliff</step> <step *id*=*"4"* *name*=*"Register-packages"*>Register new packages in translation-management.xml... before deleting files with no new segments.</step> <step *id*=*"5"* *name*=*"Old-content-to-target"*>Process files in no-new-segment (could have different attributes or deleted content) to target lang.</step> <step *id*=*"6"* *name*=*"Delete-no-new-segment"*>Deletes no-new-segment xliffs that were processed in step Old-content-to-target.
</step> <step *id*=*"7"* *name*=*"Set-target-status"*>Sets updatibility of target topic</step> <step *id*=*"8"* *name*=*"Export-and-zip"*>Export and zip files</step> </steps>
...
- switch* (*$workflow*//*step*[*@id*=*$this-step*]/*data*(*@name*)) *case *'Add-ids' *return *(:let $debug :=
file:append('debug-create-pack-crash.xml', ' In Add-ids '):) *let **$t* := '' *return *(*admin:add-ids*('en-us'), *db:output*( *$next-step-forward*))
... *case *'Set-target-status' *return *(:let $debug :=file:append('debug-create-pack-crash.xml', ' In Set-target-status '):) *let **$t* := ''
*return *(*for **$lg* *in**$lang* *return **translation:toggle-target-status*(*$lg*), *db:output*( *$next-step-forward*))
.... *case *'Export-and-zip' *return *(*let **$zip* :=*translation:export-and-zip*(*$menu-id*, *$lang*, *$package-number*) (:let $debug := file:append('debug-create-pack-crash.xml', ' In Export-and-zip '):) *return *((), *db:output*( *$next-step-forward*)) ) *default* *return *((), *db:output*( *admin:display-in-dashboard*(<div><div *class*=*"l2"* *style*= *"color:red;"*>An error has occured, you should have been redirected to the export function before reaching this point. Default in switch steps.
</div>{*app:get-beep-div-for-dashboard*('error')}</div>)))
};
On Tue, Aug 19, 2014 at 10:26 AM, Christian Grün < christian.gruen@gmail.com> wrote:
Hi France,
I would also be interested if you'd like a functionality as proposed by Andy, and if you are interested in improving debugging your XQuery code, or if you think that such a function could also be helpful in productive code? Do you have some more input?
Thanks, Christian
On Mon, Aug 18, 2014 at 7:14 PM, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to
find a
function that would do the equivalent of what the code bellow does, but
for
internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
Hi France,
Ok I get it now. No, really, because what your working on is very similar to what I'm regularly dealing with at work (translation, xliff etc.). I can also relate to your feeling that you need more control or insight in what's going on in your code. I have been through this journey too and sometimes I still find myself in a similar spot as you describe. Especially when deadlines are breathing in your neck ;-)
But I think that instead of looking for "technical" solutions to insert more logging to see what's going on what you need is to break up your code, bring more structure to it. There are many things going on in this small piece of code.For starters each "case" warrants a separate function. A function should be more or less about one thing (Single Responsibility Principle). Doing this will also make the code more testable. You can test each function separately. I admit this is not always easy but it is key to keep your code under control.
Also have a look at the Unit module which provides you with a way to test each individual function with all types of arguments. You can run it very easily from the commandline (e.g. basex -t test-files/). During development of code immediately write these tests. You can even adopt a bit of test driven design (TDD) where you write tests even before the code works. Then you code until your tests pass. Best of all, you will keep these tests for the lifetime of your code and running them and seeing all of them pass will give you confidence in your code (or find out that you broke something). These test cases are also great documentation because they remind you of what a function was supposed to do when you already forgot because you went on to other projects.
Another tip: when tackling something new and I'm not sure what combination of XQuery functions I need I always start the GUI, load the modules I'm working on and start experimenting. But once I have a better idea I start writing tests.
It's not easy but very rewarding (and writing tests after the fact can feel very boring that's why you should do it while you work on the code). Every time I neglect these principles, because, for example, a deadline is near, it comes back to bite me later. In many cases it resulted in throwing away the code and starting from scratch.
Sorry, no instant gratification by pointing you to a "magic" module. Still, I hope it helps you.
Cheers,
--Marc
On Tue, Aug 19, 2014 at 11:12 PM, Andy Bunce bunce.andy@gmail.com wrote:
If the goal is debugging .. another approach might be to think about Aspect Oriented Programming something like http://www.mkyong.com/spring3/spring-aop-aspectj-annotation-example/
/Andy
On 19 August 2014 18:05, France Baril france.baril@architextus.com wrote:
Hi,
I am hoping to use this for debugging. Our project is fairly complex and I use a controller function to apply multiple operations to content (each of these may use one or more function). When I get a 500 error, it's very hard for me to pinpoint the source of the issue.
I can get the data from the form filled by the user and a few of the redirect, but that only gets me so far, if I could track both the function names and the parameters for sub-functions, it would save me a lot of time.
Below is an example of what I call a controller... I use those when I need content to be saved before I can query it again for the next updating operation. Each if the workflow step uses multiple sub functions that are not accessed through http and that are as far as I know untraceable.
(: Transforms content to xliff for each language and export packages for translation vendors :)
declare %rest:path('/translation/create-package') %rest:GET %rest:query-param('menu-id', '{$menu-id}', '') %rest:query-param('lang', '{$lang-in}', '') (: The step that we were at when the function was called :) %rest:query-param('step-id', '{$step-id}', '0') %rest:query-param('package-number-in', '{$package-number-in}', '') %output:method("html") %output:html-version("5.0") updating function translation:create-package-controller($menu-id as xs:string, $lang-in as xs:string*, $package-number-in as xs:string, $step-id as xs:string){
...
let $workflow := <steps> <step id="1" name="Add-ids">Add @ids for en-us</step> <step id="2" name="Health-check">DB health check for en-us</step> <!--step id="3" name="Check-keywords">Check for keywords that may need to be replaced by prompts</step--> <step id="3" name="Process-to-xliff">Process files to skeletons and xliff</step> <step id="4" name="Register-packages">Register new packages in translation-management.xml... before deleting files with no new segments.</step> <step id="5" name="Old-content-to-target">Process files in no-new-segment (could have different attributes or deleted content) to target lang.</step> <step id="6" name="Delete-no-new-segment">Deletes no-new-segment xliffs that were processed in step Old-content-to-target.</step> <step id="7" name="Set-target-status">Sets updatibility of target topic</step> <step id="8" name="Export-and-zip">Export and zip files</step> </steps>
...
switch ($workflow//step[@id=$this-step]/data(@name)) case 'Add-ids' return (:let $debug := file:append('debug-create-pack-crash.xml', ' In Add-ids '):) let $t := '' return (admin:add-ids('en-us'),
db:output($next-step-forward))
... case 'Set-target-status' return (:let $debug :=file:append('debug-create-pack-crash.xml', ' In Set-target-status '):) let $t := ''
return (for $lg in $langreturn translation:toggle-target-status($lg), db:output($next-step-forward))
.... case 'Export-and-zip' return (let $zip :=translation:export-and-zip($menu-id, $lang, $package-number) (:let $debug := file:append('debug-create-pack-crash.xml', ' In Export-and-zip '):) return ((),
db:output($next-step-forward)) ) default return ((), db:output(admin:display-in-dashboard(<div><div class="l2" style="color:red;">An error has occured, you should have been redirected to the export function before reaching this point. Default in switch steps.</div>{app:get-beep-div-for-dashboard('error')}</div>)))
};
On Tue, Aug 19, 2014 at 10:26 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi France,
I would also be interested if you'd like a functionality as proposed by Andy, and if you are interested in improving debugging your XQuery code, or if you think that such a function could also be helpful in productive code? Do you have some more input?
Thanks, Christian
On Mon, Aug 18, 2014 at 7:14 PM, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does, but for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' || string-join(request:parameter($name), '; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
The code is separated in a way that creates as few redirects as possible. The large pieces that you see are split into much smaller units. The smaller units are the ones that I need to track. The large piece, I can track through the http requests.
The purpose of these large segments are to force BaseX to update when upcoming processes need to query the modified content.
Redirects have a direct impact on performance. Submitting only when necessary has improved our performance tremendously.
On Tue, Aug 19, 2014 at 6:05 PM, Marc van Grootel < marc.van.grootel@gmail.com> wrote:
Hi France,
Ok I get it now. No, really, because what your working on is very similar to what I'm regularly dealing with at work (translation, xliff etc.). I can also relate to your feeling that you need more control or insight in what's going on in your code. I have been through this journey too and sometimes I still find myself in a similar spot as you describe. Especially when deadlines are breathing in your neck ;-)
But I think that instead of looking for "technical" solutions to insert more logging to see what's going on what you need is to break up your code, bring more structure to it. There are many things going on in this small piece of code.For starters each "case" warrants a separate function. A function should be more or less about one thing (Single Responsibility Principle). Doing this will also make the code more testable. You can test each function separately. I admit this is not always easy but it is key to keep your code under control.
Also have a look at the Unit module which provides you with a way to test each individual function with all types of arguments. You can run it very easily from the commandline (e.g. basex -t test-files/). During development of code immediately write these tests. You can even adopt a bit of test driven design (TDD) where you write tests even before the code works. Then you code until your tests pass. Best of all, you will keep these tests for the lifetime of your code and running them and seeing all of them pass will give you confidence in your code (or find out that you broke something). These test cases are also great documentation because they remind you of what a function was supposed to do when you already forgot because you went on to other projects.
Another tip: when tackling something new and I'm not sure what combination of XQuery functions I need I always start the GUI, load the modules I'm working on and start experimenting. But once I have a better idea I start writing tests.
It's not easy but very rewarding (and writing tests after the fact can feel very boring that's why you should do it while you work on the code). Every time I neglect these principles, because, for example, a deadline is near, it comes back to bite me later. In many cases it resulted in throwing away the code and starting from scratch.
Sorry, no instant gratification by pointing you to a "magic" module. Still, I hope it helps you.
Cheers,
--Marc
On Tue, Aug 19, 2014 at 11:12 PM, Andy Bunce bunce.andy@gmail.com wrote:
If the goal is debugging .. another approach might be to think about Aspect Oriented Programming something like http://www.mkyong.com/spring3/spring-aop-aspectj-annotation-example/
/Andy
On 19 August 2014 18:05, France Baril france.baril@architextus.com
wrote:
Hi,
I am hoping to use this for debugging. Our project is fairly complex
and I
use a controller function to apply multiple operations to content (each
of
these may use one or more function). When I get a 500 error, it's very
hard
for me to pinpoint the source of the issue.
I can get the data from the form filled by the user and a few of the redirect, but that only gets me so far, if I could track both the
function
names and the parameters for sub-functions, it would save me a lot of
time.
Below is an example of what I call a controller... I use those when I
need
content to be saved before I can query it again for the next updating operation. Each if the workflow step uses multiple sub functions that
are
not accessed through http and that are as far as I know untraceable.
(: Transforms content to xliff for each language and export packages for translation vendors :)
declare %rest:path('/translation/create-package') %rest:GET %rest:query-param('menu-id', '{$menu-id}', '') %rest:query-param('lang', '{$lang-in}', '') (: The step that we were at when the function was called :) %rest:query-param('step-id', '{$step-id}', '0') %rest:query-param('package-number-in', '{$package-number-in}',
'')
%output:method("html") %output:html-version("5.0")updating function translation:create-package-controller($menu-id as xs:string, $lang-in as xs:string*, $package-number-in as xs:string, $step-id as xs:string){
...
let $workflow := <steps> <step id="1" name="Add-ids">Add @ids for en-us</step> <step id="2" name="Health-check">DB health check for en-us</step> <!--step id="3" name="Check-keywords">Check for keywords that may need to be replaced by prompts</step--> <step id="3" name="Process-to-xliff">Process
files
to skeletons and xliff</step> <step id="4" name="Register-packages">Register
new
packages in translation-management.xml... before deleting files with no
new
segments.</step> <step id="5"
name="Old-content-to-target">Process
files in no-new-segment (could have different attributes or deleted
content)
to target lang.</step> <step id="6"
name="Delete-no-new-segment">Deletes
no-new-segment xliffs that were processed in step Old-content-to-target.</step> <step id="7" name="Set-target-status">Sets updatibility of target topic</step> <step id="8" name="Export-and-zip">Export and
zip
files</step> </steps>
...
switch ($workflow//step[@id=$this-step]/data(@name)) case 'Add-ids' return (:let $debug := file:append('debug-create-pack-crash.xml', ' In Add-ids '):) let $t := '' return (admin:add-ids('en-us'),
db:output($next-step-forward))
... case 'Set-target-status' return (:let $debug :=file:append('debug-create-pack-crash.xml', ' In Set-target-status '):) let $t := ''
return (for $lg in $langreturn translation:toggle-target-status($lg),
db:output($next-step-forward))
.... case 'Export-and-zip' return (let $zip :=translation:export-and-zip($menu-id, $lang, $package-number) (:let $debug := file:append('debug-create-pack-crash.xml', ' In
Export-and-zip '):)
return ((),db:output($next-step-forward)) ) default return ((), db:output(admin:display-in-dashboard(<div><div class="l2" style="color:red;">An error has occured, you should have been
redirected to
the export function before reaching this point. Default in switch steps.</div>{app:get-beep-div-for-dashboard('error')}</div>)))
};
On Tue, Aug 19, 2014 at 10:26 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi France,
I would also be interested if you'd like a functionality as proposed by Andy, and if you are interested in improving debugging your XQuery code, or if you think that such a function could also be helpful in productive code? Do you have some more input?
Thanks, Christian
On Mon, Aug 18, 2014 at 7:14 PM, France Baril france.baril@architextus.com wrote:
Hi,
I'm working on improving our tracking capabilities. I was hoping to find a function that would do the equivalent of what the code bellow does,
but
for internal functions that are not called through the rest interface.
string-join(for $name in request:parameter-names() return $name || ': ' ||
string-join(request:parameter($name),
'; '),
' ')In short, I was wondering if there is way I can read the function's parameters and their values without listing each one explicitly.
I searched the documentation without success.
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com
-- --Marc
Also, we have a pretty slick system when it comes to identifying small issues. We are looking at preventing larger issues (that occur from domino effects), or at least we want to be able to study them when they occur. Here is an example of a domino effect issue we had:
Ex. 1. Add an attribute to the root of an element. 2. Run a few read operations with success. 3. Run a process that updates one of the files that has a new root attribute, get a corrupted DB.
Anything that can leave a trace to identify that file out of the few 100 I am processing is a win. Since our files go through different processes based on their state, I can eliminate a lot of files just by knowing which functions were called for which files. This helps me figure out the link between operation #1 and operation #3. In our last case, operation #3 was run a few weeks after operation #1. A trace would not have helped much, but in most cases, it could.
basex-talk@mailman.uni-konstanz.de
-
Andy Bunce -
Christian Grün -
France Baril -
Marc van Grootel