Possible bug in database size (bytes) after database optimize from BaseX Database Administration
I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10.
Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
Hi Geoff,
Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"?
Best, Christian
Geoff Alexander gdlxn@us.ibm.com schrieb am Do., 16. Juli 2020, 18:06:
I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10.
Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
On the BaseX Database Administration's Database page at https://localhost:10443/BaseX/dba/databases, I selected a database I knew was unoptimized and pressed the Optimize button. The database's COUNT column decreased to the number entries in the database as expected. However, the database's BYTES column did not change, even after I logged off and back on to BaseX Database Administration.
Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
From: "Christian Grün" christian.gruen@gmail.com To: Geoff Alexander gdlxn@us.ibm.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Date: 07/16/2020 01:10 PM Subject: [EXTERNAL] Re: [basex-talk] Possible bug in database size (bytes) after database optimize from BaseX Database Administration
Hi Geoff,
Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"?
Best, Christian
Geoff Alexander gdlxn@us.ibm.com schrieb am Do., 16. Juli 2020, 18:06: I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10.
Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
It's surprising that the count value changed, as it should represent the number of resources (documents, binary files) in your database – and this value shouldn't change if your data is optimized. Feel free to provide us with a little reproducible example.
The size of the database may stay the same, though. The DBA provides no way to trigger a full optimization, but you can e.g. use the query panel for that.
Geoff Alexander gdlxn@us.ibm.com schrieb am Do., 16. Juli 2020, 20:04:
On the BaseX Database Administration's Database page at https://localhost:10443/BaseX/dba/databases, I selected a database I knew was unoptimized and pressed the Optimize button. The database's COUNT column decreased to the number entries in the database as expected. However, the database's BYTES column did not change, even after I logged off and back on to BaseX Database Administration.
Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
[image: Inactive hide details for "Christian Grün" ---07/16/2020 01:10:38 PM---Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., t]"Christian Grün" ---07/16/2020 01:10:38 PM---Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by
From: "Christian Grün" christian.gruen@gmail.com To: Geoff Alexander gdlxn@us.ibm.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Date: 07/16/2020 01:10 PM Subject: [EXTERNAL] Re: [basex-talk] Possible bug in database size (bytes) after database optimize from BaseX Database Administration
Hi Geoff,
Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"?
Best, Christian
Geoff Alexander <*gdlxn@us.ibm.com* gdlxn@us.ibm.com> schrieb am Do., 16. Juli 2020, 18:06:
I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10.
Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
Here are steps to recreate the problem:
(1) Add one or more entries to an new (empty) database. One the BaseX Database Administration's Database page, you'll find that the database's COUNT column shows the number of entries added and that the database's BYTES column shows the database size.
(2) Update one or of the database's entries. After refreshing BaseX Database Administration's Database page, you should find that the database's COUNT and BYTES columns both increased.
(3) On the BaseX Database Administration's Database page, select the database and press the Optimize button. You should find that the database's COUNT column decreases back to the number of entries in the database. However, the database's BYTES column doesn't decrease to reflect a reduction in the database size.
Maybe I have a misundertanding in what the Optimize button on the BaseX Database Administration's Database page actually does.
Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
From: "Christian Grün" christian.gruen@gmail.com To: Geoff Alexander gdlxn@us.ibm.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Date: 07/16/2020 02:11 PM Subject: [EXTERNAL] Re: Possible bug in database size (bytes) after database optimize from BaseX Database Administration
It's surprising that the count value changed, as it should represent the number of resources (documents, binary files) in your database – and this value shouldn't change if your data is optimized. Feel free to provide us with a little reproducible example.
The size of the database may stay the same, though. The DBA provides no way to trigger a full optimization, but you can e.g. use the query panel for that.
Geoff Alexander gdlxn@us.ibm.com schrieb am Do., 16. Juli 2020, 20:04: On the BaseX Database Administration's Database page at https://localhost:10443/BaseX/dba/databases, I selected a database I knew was unoptimized and pressed the Optimize button. The database's COUNT column decreased to the number entries in the database as expected. However, the database's BYTES column did not change, even after I logged off and back on to BaseX Database Administration.
Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
"Christian Grün" ---07/16/2020 01:10:38 PM---Hi Geoff, Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by
From: "Christian Grün" christian.gruen@gmail.com To: Geoff Alexander gdlxn@us.ibm.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Date: 07/16/2020 01:10 PM Subject: [EXTERNAL] Re: [basex-talk] Possible bug in database size (bytes) after database optimize from BaseX Database Administration
Hi Geoff,
Did you run OPTIMIZE ALL or db:optimize(..., true()) ? What do you mean by "record count"?
Best, Christian
Geoff Alexander gdlxn@us.ibm.com schrieb am Do., 16. Juli 2020, 18:06: I've found that when I perform a database optimize on an unoptimized database that while the record count decreases as expected, the database size (bytes column) stay the same. Is this a bug in reporting the database size (less severe problem) or a bug in the database not reducing on optimize (more severe problem)? I've encountered this with both BaseX 8.6.7 and 9.3.3 running on Windows 10.
Thanks, Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
[attachment "graycol.gif" deleted by Geoff Alexander/Raleigh/IBM]
(2) Update one or of the database's entries.
If I replace the document via the DBA, or if I run an update expression via the Queries Panel, the count always reflects the number of resources, it doesn’t change.
How did you update the document? Is it an XML document or a binary file you updated?
Maybe I have a misundertanding in what the Optimize button on the BaseX Database Administration's Database page actually does.
Feel free to have a look into our documentation [1].
We use the BaseX REST API from a Java problem to add and update documents in BaseX.
Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
From: "Christian Grün" christian.gruen@gmail.com To: Geoff Alexander gdlxn@us.ibm.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Date: 07/16/2020 03:07 PM Subject: [EXTERNAL] Re: Possible bug in database size (bytes) after database optimize from BaseX Database Administration
(2) Update one or of the database's entries.
If I replace the document via the DBA, or if I run an update expression via the Queries Panel, the count always reflects the number of resources, it doesn’t change.
How did you update the document? Is it an XML document or a binary file you updated? Maybe I have a misundertanding in what the Optimize button on the BaseX Database Administration's Database page actually does.
Feel free to have a look into our documentation [1].
I would think so. Let me see if I can create a small example using the BaseX WEB API that recreates the problem.
Geoff Alexander, Ph.D. Software Engineer, Corporate Tools Development IBM Corporation Charlotte, NC
From: "Christian Grün" christian.gruen@gmail.com To: Geoff Alexander gdlxn@us.ibm.com Cc: BaseX basex-talk@mailman.uni-konstanz.de Date: 07/16/2020 03:10 PM Subject: [EXTERNAL] Re: Possible bug in database size (bytes) after database optimize from BaseX Database Administration
We use the BaseX REST API from a Java problem to add and update documents in BaseX.
Do you think it’s reproducible for us?
basex-talk@mailman.uni-konstanz.de
-
Christian Grün -
Geoff Alexander