Re: [basex-talk] Multiple db:replace() = slower updates with 7.8

Hi France,

sorry, I know that an answer to your e-mail is still pending. If you should manage to build a self-contained example, it could help us to find the culprit.

Best, Christian

On Wed, Mar 19, 2014 at 6:59 PM, France Baril france.baril@architextus.com wrote:

Hi,

We run batch processes on our files in order to send them out to translation in xliff format (xml format for translations).

I run a piece of code that:

1. applies transformations
2. file:writes the resulting files (for debug purposes)
3. and then db:replaces the resulting files.

If I send 5 files into the process, it takes a few seconds before the files appear on the file system. Yet full process with the db:replace takes 2 minutes to complete.

When I send 60 files... the process never ends. All files are available on the file system and are valid, which means I am fairly certain that the freeze doesn't occur because of an error in one of the files.

There is no error message in the terminal window. Is possible that something was changed after 7.7 that would affect performance? On our production server, which is on 7.7 we can run 100+ files in just a few minutes. Maybe there are new server settings that we missed?

For reference, here is the code. I understand that updates are all performed at the end of the process, which is why we get the debug files first, but the lag time between the time where we get all files on the file system and the db:replace seems huge.

declare updating function translation:transform-to-xliff($menu-id as xs:string, $lang as xs:string*, $package-number as xs:string){

let $menu := menu:xml-get-menu($menu-id, 'en-us')

(: Need to translate menu and its topics without getting duplicated

topics twice:) (: Always send the reusable strings from writers and from dev... doesn't cost anything if no new string anyway :) let $file-ids := ($menu-id, for $id in distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] return $id, 'stringreferences', 'stringreferencestransform')

return  (for $file-id in distinct-values($file-ids)
         let $file := db:open('en-us')/*[@id=$file-id]
         let $skl-href := 'skeletons/' || $package-number || '/' ||

substring-after($file/base-uri(), 'en-us/') let $file-with-preserve := copy $copy := $file modify(if (exists($copy/@xml:space)) then () else insert node attribute {'xml:space'}{'preserve'} into $copy) return $copy let $all-trans-files := xliff-transformer:create-translation-files($file-with-preserve, $package-number, $menu-id, $lang, $skl-href)

         let $skeleton := $all-trans-files[1]

         let $xliffs := for $xliff in $all-trans-files[position() != 1]
                        return $xliff

         let $debug := file:write('Debug-xliff/skls/' ||

tokenize($skl-href, '/')[last()], $skeleton)

return (db:replace('Translation-management', $skl-href, $skeleton), for $xliff in $xliffs let $target-lang := $xliff/descendant::*[name()='file']/data(@target-language) let $lang-pack-name := concat($target-lang,'_', $package-number) let $file-loc-end := $menu-id || '/' || $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') (:let $debug := file:append('debug-ids.xml', '
' || $file-loc-end):) let $file-loc := if (exists($xliff/descendant::*[@translate='yes'])) then 'xliff-to-translate/' || $file-loc-end else 'xliff-no-new-segment/' || $file-loc-end let $debug := file:write('Debug-xliff/' || tokenize($file-loc, '/')[last()], $xliff) return db:replace('Translation-management', $file-loc, $xliff) ) ) };

-- France Baril Architecte documentaire / Documentation architect france.baril@architextus.com (514) 572-0341

---

BaseX-Talk mailing list BaseX-Talk@mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk