Hello,
I am having noticeable performance degradation for updating existing database in this function with Basex 81. I can say this function never finish with 81. But when I replace nodes call with file:append (writing result to a file), I can see it is slowness.
But Basex_79 is much faster than Basex81 with regard to this update and finish this function quickly without any issue.
I like to upgrade to 81 but because of this issue I could not.
Can you help me?
declare %updating function local:addCRMSegmentsIntoProducts () {
let $allProds := fn:doc("AccumulatedProducts") let $segments := ("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") let $productsMap := map:new( for $rec in $allProds//doc let $prodIDField := $rec/field[@name eq "ProductID"] let $pid := $prodIDField/text() return map:entry($pid,$prodIDField))
for tumbling window $prodGroup in fn:doc("CRMSegments")//record start $first next $second when fn:true() end $last next $beyond when $last/PRODUCT_ID ne $beyond/PRODUCT_ID
let $pid := $prodGroup[1]/PRODUCT_ID return let $prodEntry := map:get($productsMap,$pid)/.. return if (fn:empty($prodEntry)) then () else let $segData := for $segGroup in $prodGroup let $sid := $segGroup/SON_SEGMENT/text() let $sid := fn:concat("10",fn:index-of($segments,$sid)) group by $sid return let $segAmount := sum($segGroup//AMOUNT/text()) let $segOrderCount := sum($segGroup//ORDER_COUNT) let $segFields := (<field name="SegAmount_{$sid}">{$segAmount}</field>, <field name="SegOrderCount_{$sid}">{$segOrderCount}</field>) return $segFields
return insert nodes ($segData) as last into $prodEntry
};
Hi Erol,
Could you possibly provide us with an sscce [1]?
Thanks in advance, Christian
On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu eakarsu@gmail.com wrote:
Hello,
I am having noticeable performance degradation for updating existing database in this function with Basex 81. I can say this function never finish with 81. But when I replace nodes call with file:append (writing result to a file), I can see it is slowness.
But Basex_79 is much faster than Basex81 with regard to this update and finish this function quickly without any issue.
I like to upgrade to 81 but because of this issue I could not.
Can you help me?
declare %updating function local:addCRMSegmentsIntoProducts () {
let $allProds := fn:doc("AccumulatedProducts") let $segments :=("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") let $productsMap := map:new( for $rec in $allProds//doc let $prodIDField := $rec/field[@name eq "ProductID"] let $pid := $prodIDField/text() return map:entry($pid,$prodIDField))
for tumbling window $prodGroup in fn:doc("CRMSegments")//record start $first next $second when fn:true() end $last next $beyond when $last/PRODUCT_ID ne $beyond/PRODUCT_ID let $pid := $prodGroup[1]/PRODUCT_ID return let $prodEntry := map:get($productsMap,$pid)/.. return if (fn:empty($prodEntry)) then () else let $segData := for $segGroup in $prodGroup let $sid := $segGroup/SON_SEGMENT/text() let $sid :=fn:concat("10",fn:index-of($segments,$sid)) group by $sid return let $segAmount := sum($segGroup//AMOUNT/text()) let $segOrderCount := sum($segGroup//ORDER_COUNT) let $segFields := (<field name="SegAmount_{$sid}">{$segAmount}</field>, <field name="SegOrderCount_{$sid}">{$segOrderCount}</field>) return $segFields
return insert nodes ($segData) as last into $prodEntry};
Christian,
I will try to bring in this information for you.
Today, I am trying to export a Bade DB about 2.5G but getting error with JDK 8.1
With Basex 79 with JDK 7 was fine.
export AccumulatedProducts
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.8.0_25 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException at org.basex.io.random.DataAccess.readBytes(DataAccess.java:216) at org.basex.io.random.DataAccess.readToken(DataAccess.java:194) at org.basex.io.random.DataAccess.readToken(DataAccess.java:185) at org.basex.data.DiskData.txt(DiskData.java:267) at org.basex.data.DiskData.text(DiskData.java:235) at org.basex.io.serial.Serializer.serialize(Serializer.java:459) at org.basex.io.serial.Serializer.serialize(Serializer.java:335) at org.basex.io.serial.Serializer.serialize(Serializer.java:135) at org.basex.io.serial.Serializer.serialize(Serializer.java:115) at org.basex.io.serial.Serializer.serialize(Serializer.java:105) at org.basex.core.cmd.Export.export(Export.java:122) at org.basex.core.cmd.Export.export(Export.java:66) at org.basex.core.cmd.Export.run(Export.java:44) at org.basex.core.Command.run(Command.java:329) at org.basex.core.Command.execute(Command.java:94) at org.basex.server.ClientListener.run(ClientListener.java:145)
On Wed, Apr 15, 2015 at 3:06 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Erol,
Could you possibly provide us with an sscce [1]?
Thanks in advance, Christian
On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu eakarsu@gmail.com wrote:
Hello,
I am having noticeable performance degradation for updating existing database in this function with Basex 81. I can say this function never finish with 81. But when I replace nodes call with file:append (writing result to a file), I can see it is slowness.
But Basex_79 is much faster than Basex81 with regard to this update and finish this function quickly without any issue.
I like to upgrade to 81 but because of this issue I could not.
Can you help me?
declare %updating function local:addCRMSegmentsIntoProducts () {
let $allProds := fn:doc("AccumulatedProducts") let $segments :=("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik")
let $productsMap := map:new( for $rec in $allProds//doc let $prodIDField := $rec/field[@name eq "ProductID"] let $pid := $prodIDField/text() return map:entry($pid,$prodIDField)) for tumbling window $prodGroup in fn:doc("CRMSegments")//record start $first next $second when fn:true() end $last next $beyond when $last/PRODUCT_ID ne$beyond/PRODUCT_ID
let $pid := $prodGroup[1]/PRODUCT_ID return let $prodEntry := map:get($productsMap,$pid)/.. return if (fn:empty($prodEntry)) then () else let $segData := for $segGroup in $prodGroup let $sid := $segGroup/SON_SEGMENT/text() let $sid :=fn:concat("10",fn:index-of($segments,$sid)) group by $sid return let $segAmount := sum($segGroup//AMOUNT/text()) let $segOrderCount := sum($segGroup//ORDER_COUNT) let $segFields := (<field name="SegAmount_{$sid}">{$segAmount}</field>, <field name="SegOrderCount_{$sid}">{$segOrderCount}</field>) return $segFields
return insert nodes ($segData) as last into $prodEntry};
Today, I am trying to export a Bade DB about 2.5G but getting error with JDK 8.1
With Basex 79 with JDK 7 was fine.
So you mean you can export the same database with JDK 7, but not with JDK 8? This indicates you should probably stick with Java 7 and wait for newer versions of the JDK.
What happens with JRE 8? Does JDK 8.1 exist at all? Afaik, the latest version is 8u45.
C.
export AccumulatedProducts
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.8.0_25 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException at org.basex.io.random.DataAccess.readBytes(DataAccess.java:216) at org.basex.io.random.DataAccess.readToken(DataAccess.java:194) at org.basex.io.random.DataAccess.readToken(DataAccess.java:185) at org.basex.data.DiskData.txt(DiskData.java:267) at org.basex.data.DiskData.text(DiskData.java:235) at org.basex.io.serial.Serializer.serialize(Serializer.java:459) at org.basex.io.serial.Serializer.serialize(Serializer.java:335) at org.basex.io.serial.Serializer.serialize(Serializer.java:135) at org.basex.io.serial.Serializer.serialize(Serializer.java:115) at org.basex.io.serial.Serializer.serialize(Serializer.java:105) at org.basex.core.cmd.Export.export(Export.java:122) at org.basex.core.cmd.Export.export(Export.java:66) at org.basex.core.cmd.Export.run(Export.java:44) at org.basex.core.Command.run(Command.java:329) at org.basex.core.Command.execute(Command.java:94) at org.basex.server.ClientListener.run(ClientListener.java:145)
On Wed, Apr 15, 2015 at 3:06 AM, Christian Grün christian.gruen@gmail.com wrote:
Hi Erol,
Could you possibly provide us with an sscce [1]?
Thanks in advance, Christian
On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu eakarsu@gmail.com wrote:
Hello,
I am having noticeable performance degradation for updating existing database in this function with Basex 81. I can say this function never finish with 81. But when I replace nodes call with file:append (writing result to a file), I can see it is slowness.
But Basex_79 is much faster than Basex81 with regard to this update and finish this function quickly without any issue.
I like to upgrade to 81 but because of this issue I could not.
Can you help me?
declare %updating function local:addCRMSegmentsIntoProducts () {
let $allProds := fn:doc("AccumulatedProducts") let $segments :=("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik") let $productsMap := map:new( for $rec in $allProds//doc let $prodIDField := $rec/field[@name eq "ProductID"] let $pid := $prodIDField/text() return map:entry($pid,$prodIDField))
for tumbling window $prodGroup in fn:doc("CRMSegments")//record start $first next $second when fn:true() end $last next $beyond when $last/PRODUCT_ID ne$beyond/PRODUCT_ID
let $pid := $prodGroup[1]/PRODUCT_ID return let $prodEntry := map:get($productsMap,$pid)/.. return if (fn:empty($prodEntry)) then () else let $segData := for $segGroup in $prodGroup let $sid := $segGroup/SON_SEGMENT/text() let $sid :=fn:concat("10",fn:index-of($segments,$sid)) group by $sid return let $segAmount := sum($segGroup//AMOUNT/text()) let $segOrderCount := sum($segGroup//ORDER_COUNT) let $segFields := (<field name="SegAmount_{$sid}">{$segAmount}</field>, <field name="SegOrderCount_{$sid}">{$segOrderCount}</field>) return $segFields
return insert nodes ($segData) as last into $prodEntry};
I am sorry for mistyping. It was jdk1.8.0_25. I am not sure why I got that exception. Maybe, it is disk I/O issue? After I got above exception, my working folder had tons of files with names consisting of digits. This may explain disk issue. What do you think?
Erol Akarsu
On Wed, Apr 15, 2015 at 2:40 PM, Christian Grün christian.gruen@gmail.com wrote:
Today, I am trying to export a Bade DB about 2.5G but getting error with
JDK
8.1
With Basex 79 with JDK 7 was fine.
So you mean you can export the same database with JDK 7, but not with JDK 8? This indicates you should probably stick with Java 7 and wait for newer versions of the JDK.
What happens with JRE 8? Does JDK 8.1 exist at all? Afaik, the latest version is 8u45.
C.
export AccumulatedProducts
Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk@mailman.uni-konstanz.de Version: BaseX 7.9 Java: Oracle Corporation, 1.8.0_25 OS: Linux, amd64 Stack Trace: java.lang.NegativeArraySizeException at org.basex.io.random.DataAccess.readBytes(DataAccess.java:216) at org.basex.io.random.DataAccess.readToken(DataAccess.java:194) at org.basex.io.random.DataAccess.readToken(DataAccess.java:185) at org.basex.data.DiskData.txt(DiskData.java:267) at org.basex.data.DiskData.text(DiskData.java:235) at org.basex.io.serial.Serializer.serialize(Serializer.java:459) at org.basex.io.serial.Serializer.serialize(Serializer.java:335) at org.basex.io.serial.Serializer.serialize(Serializer.java:135) at org.basex.io.serial.Serializer.serialize(Serializer.java:115) at org.basex.io.serial.Serializer.serialize(Serializer.java:105) at org.basex.core.cmd.Export.export(Export.java:122) at org.basex.core.cmd.Export.export(Export.java:66) at org.basex.core.cmd.Export.run(Export.java:44) at org.basex.core.Command.run(Command.java:329) at org.basex.core.Command.execute(Command.java:94) at org.basex.server.ClientListener.run(ClientListener.java:145)
On Wed, Apr 15, 2015 at 3:06 AM, Christian Grün <
christian.gruen@gmail.com>
wrote:
Hi Erol,
Could you possibly provide us with an sscce [1]?
Thanks in advance, Christian
On Wed, Apr 15, 2015 at 4:11 AM, Erol Akarsu eakarsu@gmail.com wrote:
Hello,
I am having noticeable performance degradation for updating existing database in this function with Basex 81. I can say this function
never
finish with 81. But when I replace nodes call with file:append
(writing
result to a file), I can see it is slowness.
But Basex_79 is much faster than Basex81 with regard to this update
and
finish this function quickly without any issue.
I like to upgrade to 81 but because of this issue I could not.
Can you help me?
declare %updating function local:addCRMSegmentsIntoProducts () {
let $allProds := fn:doc("AccumulatedProducts") let $segments :=("Aburcubur","Ãay_Kahve","İçecek","Karma_Az","Meyve_Sebze","Saç_Bakım","Süt_Su-Madeaze_Tüketim","Temizlik")
let $productsMap := map:new( for $rec in $allProds//doc let $prodIDField := $rec/field[@name eq "ProductID"] let $pid := $prodIDField/text() return map:entry($pid,$prodIDField)) for tumbling window $prodGroup in fn:doc("CRMSegments")//record start $first next $second when fn:true() end $last next $beyond when $last/PRODUCT_ID ne$beyond/PRODUCT_ID
let $pid := $prodGroup[1]/PRODUCT_ID return let $prodEntry := map:get($productsMap,$pid)/.. return if (fn:empty($prodEntry)) then () else let $segData := for $segGroup in $prodGroup let $sid := $segGroup/SON_SEGMENT/text() let $sid :=fn:concat("10",fn:index-of($segments,$sid)) group by $sid return let $segAmount :=
sum($segGroup//AMOUNT/text())
let $segOrderCount :=sum($segGroup//ORDER_COUNT) let $segFields := (<field name="SegAmount_{$sid}">{$segAmount}</field>, <field name="SegOrderCount_{$sid}">{$segOrderCount}</field>) return $segFields
return insert nodes ($segData) as last into $prodEntry};
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Erol Akarsu