Hi,
- Our issue is with performance. - Performing all the transformations lead to 2 issues: - We run out of memory. - We sometimes need to query the content that has been transformed after it has gone through some of the transformations. - Web controller is jquery, but we redirect from xquery. jquery just says: translate these files in these languages. The restxq function handles the steps, and calls itself back with an incremented step number when a group of transformations that could be handle without committing content (without a need to query the saved files or without running out of memory) are completed.
On Wed, Feb 11, 2015 at 1:47 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi France,
I guess there is no simple answer to your question; it mostly depends on the architecture of your approach what would be the best solution and further steps. And I'm not quite sure what's the major challenge? Is it performance, is it technical restrictions, is it the overall concept?
As you were mentioning that you are working with a web interface, our approach would be to provide a RESTXQ function that triggers all the transformations whenever a user requests it. Have you thought about that? What language is your web controller built on?
Best, Christian
On Mon, Feb 9, 2015 at 8:12 PM, France Baril france.baril@architextus.com wrote:
Hi,
I have an item that I would like to bring to attention. We have
developed a
web controller to let users manages translation processes for BaseX
content.
Our process is something like this:
Users select content to translation (1 to 500 small files) + languages to translation to (1 to 32 selections). For each language, for each files:
The system transforms the content to xliff and sets the segments to translate="yes" if they have changed since the last translation for this language. Content is saved because we'll need to query it, we redirect to the next task. File without new segment to translate, content is processed and saved to
the
target languages (because attributes might have changed and/or segments might have been deleted). The xliff-file is deleted, we redirect to the
next
task because we'll need the new information for the next query. We query the server to offer the users stats on items to translate.
This is a simplification of the process, but it shows the logics.
Redirects
occur after each step for each language. We have grouped operations to
limit
commits/redirects to a minimum. We apply them:
After each language to avoid running out of memory. Before each operation that needs to query files based on the changes from the previous steps.
We have also split groups of tasks into smaller groups where too many
tasks
have led us to run out of memory in the past.
Our request would be for a way to force changes to commit without having
to
redirect. Refreshing the browser has a big impact on performance. Or
maybe
you have suggestions to improve batch processing when using a web
interface
for process management.
Thank you in advance for you input!
-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com