Hi James,
I'm glad to tell you that I have now implemented the projected optimizations:
1. While a database is open, freed slots in the index heap file will now be remembered and refilled with new texts. If this approach proves to be successful, we might make this free slot structure persistent such that it will also be available after closing a database.
2. The operations of the REPLACE command, which is also used by the REST PUT method, have been rewritten to take advantage of various existing low-level optimizations. Before, a document was deleted and inserted, and now it may be directly replaced (overwritten) in the storage.
I was also having a think and wondered if, assuming a full fix is
difficult, a special optimising function would be fast and easy. Instead of rebuilding the index content by reading the database just rebuild the files eliminating the free space - rather like a disk defragmenter.
This in an interesting idea. The problem in practice is that we could not find free space that simply in the past . Solution 1 might already solve the discussed problem, at least partially.
Do let me know if I can help testing any snapshots or similar.
I have uploaded the latest snapshot [1]; your testing feedback is more than welcome.
Christian