[MapDB Persistence] Add periodic database compaction

[nbrealey]

The mapdb persistence database grows in size and fills the disk if the everyUpdate strategy is used.

Expected Behavior

The database should not grow in size if the keys do not change.
The database should be compacted automatically if there is a lot of wasted space.

Current Behavior

The mapdb persistence database grows in size and fills the disk.
Restarting the openhab2 service causes the database to be compacted.
I saw the problem with String items.

Steps to Reproduce

1. Configure mapdb persistence with to save some String items using the everyUpdate strategy.
2. Watch the database grow in size: du -h -s /var/lib/openhab2/persistence/mapdb (location of database may vary)

Context

The virtual disk on the VM running openHAB became full.
I was using openHAB to implement multiple heating schedules on my Drayton Wiser system when the manufacturer only provided a single shedule. See https://community.openhab.org/t/drayton-wiser-thermostat-binding/35640/320

Your Environment

• Version used:

openHAB 2.5.0 Build #1557 and Build #1558 (openHAB 2.5.0-SNAPSHOT version) from yum repository.

MapDB Persistence: persistence-mapdb - 1.14.0.SNAPSHOT

• Environment name and version:

zulu-8 8.36.0.1-1 from yum repository:
java -version
openjdk version "1.8.0_202"
OpenJDK Runtime Environment (Zulu 8.36.0.1-CA-linux64) (build 1.8.0_202-b05)
OpenJDK 64-Bit Server VM (Zulu 8.36.0.1-CA-linux64) (build 25.202-b05, mixed mode)

• Operating System and version:

CentOS 7.6 on VirtualBox (5.2.256) VM on XigmaNAS 11.2.04.6536 (FreeBSD 11.2-RELEASE-P9).

[davidgraeff]

I cannot flag this as "bug" as it really is no bug :/ The openHAB persistence layer is not yet prepared to constrain persistence data to a specific quota.

The same with log files btw.

[5iver]

Since you are using the 1.x version of the binding, you should post your issue here... https://github.com/openhab/openhab1-addons/issues. There is a 2.x version of mapdb that was available in ESH, and I see it in the repo, but I don't see it yet in Paper UI (using S1557).

[davidgraeff]

@openhab-5iver All persistence services are 1.x bundles. The map 2.x bundle was never used and is not part of the karaf feature file.

But Kai expressed the will to move them over to this repo soonish, so it is the correct repository to report such an issue.

[5iver]

But the reported issue is for mapdb 1.14, so how would that be resolved in this repo?

[davidgraeff]

I cannot yet be resolved that is true. But the issue is existing and where do we keep track of it if not here? Nobody will watch the OH1 repo for such an issue, I assume? Especially not if the files have been moved over.

[5iver]

There are 1.x maintainers and it hasn't been moved yet... https://github.com/openhab/openhab1-addons/tree/master/bundles/persistence/org.openhab.persistence.mapdb. 😄 And the issue may not exist in the 2.x version...

[davidgraeff]

Of course it does, the 2.x version is the 1.x version with renamed package names. But we can of course hope that OH 1 maintainers implement a fix although I don't know how, because the core would need to be changed. I try to transfer the issue now.

Edit: I'm not allowed to because I have no write access to OH1-addons.

[davidgraeff]

@kaikreuzer Can you move this Issue to the OH1 repo?

[kaikreuzer]

Moved.

[9037568]

So the easy fix is, don't use everyUpdate if your use case doesn't make it meaningful, as here.

Compacting the DB occasionally is an interesting feature request, though, so I've relabelled this as a request for that.

[kaikreuzer]

Not using everyUpdate isn't really a good option as this is the main usecase of the mapdb database.

Note that MapDB isn't a normal database, which would store time series. Instead, it only holds the very last value for an item. With that in mind, it is indeed a bit weird if the database size is growing over time.

[9037568]

Ok, that's a wrinkle.

@nbrealey can you get a trace log for this and attach it?