We may have found a first evidence for the slow down. Do your
documents use namespaces?
On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün
<christian.gruen@gmail.com> wrote:
> Hi France,
>
> sorry, I know that an answer to your e-mail is still pending. If you
> should manage to build a self-contained example, it could help us to
> find the culprit.
>
> Best,
> Christian
>
>
> On Wed, Mar 19, 2014 at 6:59 PM, France Baril
> <france.baril@architextus.com> wrote:
>> Hi,
>>
>> We run batch processes on our files in order to send them out to translation
>> in xliff format (xml format for translations).
>>
>> I run a piece of code that:
>> 1. applies transformations
>> 2. file:writes the resulting files (for debug purposes)
>> 3. and then db:replaces the resulting files.
>>
>> If I send 5 files into the process, it takes a few seconds before the files
>> appear on the file system. Yet full process with the db:replace takes 2
>> minutes to complete.
>>
>> When I send 60 files... the process never ends. All files are available on
>> the file system and are valid, which means I am fairly certain that the
>> freeze doesn't occur because of an error in one of the files.
>>
>> There is no error message in the terminal window. Is possible that something
>> was changed after 7.7 that would affect performance? On our production
>> server, which is on 7.7 we can run 100+ files in just a few minutes. Maybe
>> there are new server settings that we missed?
>>
>> For reference, here is the code. I understand that updates are all performed
>> at the end of the process, which is why we get the debug files first, but
>> the lag time between the time where we get all files on the file system and
>> the db:replace seems huge.
>>
>> declare updating function translation:transform-to-xliff($menu-id as
>> xs:string, $lang as xs:string*, $package-number as xs:string){
>>
>> let $menu := menu:xml-get-menu($menu-id, 'en-us')
>>
>> (: Need to translate menu and its topics without getting duplicated
>> topics twice:)
>> (: Always send the reusable strings from writers and from dev... doesn't
>> cost anything if no new string anyway :)
>> let $file-ids := ($menu-id, for $id in
>> distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1]
>> return $id,
>> 'stringreferences', 'stringreferencestransform')
>>
>> return (for $file-id in distinct-values($file-ids)
>> let $file := db:open('en-us')/*[@id=$file-id]
>> let $skl-href := 'skeletons/' || $package-number || '/' ||
>> substring-after($file/base-uri(), 'en-us/')
>> let $file-with-preserve :=
>> copy $copy := $file
>> modify(if (exists($copy/@xml:space))
>> then ()
>> else insert node attribute
>> {'xml:space'}{'preserve'} into $copy)
>> return $copy
>> let $all-trans-files :=
>> xliff-transformer:create-translation-files($file-with-preserve,
>> $package-number, $menu-id, $lang, $skl-href)
>>
>> let $skeleton := $all-trans-files[1]
>>
>> let $xliffs := for $xliff in $all-trans-files[position() != 1]
>> return $xliff
>>
>> let $debug := file:write('Debug-xliff/skls/' ||
>> tokenize($skl-href, '/')[last()], $skeleton)
>>
>>
>> return (db:replace('Translation-management', $skl-href, $skeleton),
>> for $xliff in $xliffs
>> let $target-lang :=
>> $xliff/descendant::*[name()='file']/data(@target-language)
>> let $lang-pack-name := concat($target-lang,'_',
>> $package-number)
>> let $file-loc-end := $menu-id || '/' ||
>> $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/')
>> (:let $debug := file:append('debug-ids.xml', ' '
>> || $file-loc-end):)
>> let $file-loc := if
>> (exists($xliff/descendant::*[@translate='yes']))
>> then 'xliff-to-translate/' ||
>> $file-loc-end
>> else 'xliff-no-new-segment/' ||
>> $file-loc-end
>> let $debug := file:write('Debug-xliff/' ||
>> tokenize($file-loc, '/')[last()], $xliff)
>> return db:replace('Translation-management', $file-loc,
>> $xliff)
>> )
>> )
>> };
>>
>>
>>
>> --
>> France Baril
>> Architecte documentaire / Documentation architect
>> france.baril@architextus.com
>> (514) 572-0341
>>
>> _______________________________________________
>> BaseX-Talk mailing list
>> BaseX-Talk@mailman.uni-konstanz.de
>> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
>>