Performance for large archives

[rotdrop]

Trying to mount iso images or large tar files I find that the performance in these cases is just not sufficient. The problem is that the current implementation scans the entire archive on each actual access to the storage, and this is really slow for large archives.

To improve this we could

• Check the interaction with the file-cache. I already have set file_system_check_changes to 0 in the mount-point but I have the impression that Nextcloud still rescans the storage. The situation has to be analysed.
  • Avoid scanning files in the constructor. This is probably the primary source of the problem.
  • Fix Storage::hasUpdated() -- trigger update if the archive file has been changed
• Optionally move the mount-operation to a background job. Looking at the points above it should -- in principle -- enough as the cloud should only re-read the archive when the archive file has changed.
• [] Check whether it was enough to resolve the points mentioned above.

[hampoelz]

The performance seems to be quite good, I was able to mount and browse a 16GB zip archive without any issues. however, since the mounting takes some time in this case, a message should be displayed that the archive is being prepared/mounted.

And instead of the following log entry, a corresponding message should be displayed when a user tries to mount the archive again if it is currently being prepared/mounted in the background.

[files_archive] Error:
/var/www/html/custom_apps/files_archive/lib/Controller/MountController.php:241:
OCA\FilesArchive\Controller\MountController::mount: Caught an Exception

[rotdrop]

The performance seems to be quite good, I was able to mount and browse a 16GB zip archive without any issues. however, since the mounting takes some time in this case, a message should be displayed that the archive is being prepared/mounted.

And instead of the following log entry, a corresponding message should be displayed when a user tries to mount the archive again if it is currently being prepared/mounted in the background.

[files_archive] Error:
/var/www/html/custom_apps/files_archive/lib/Controller/MountController.php:241:
OCA\FilesArchive\Controller\MountController::mount: Caught an Exception

Here we have to decide what should happen:

• first possibility: prohibit direct mounts while the archive is being prepared for mount in the background
• second possibility: cancel the background job if mounting in the frontend succeeded, the user got the desired result and everything is well.

So in this case I would vote that a frontend mount just should cancel the background job. If the user tries to file a second background job then the user should be informed that one is already pending.

What is also missing would be a control to cancel a pending job. One could use the mount-point list in the sidebar tab and add an entry for a pending mounts, not only for successful mounts. I think that this should be enough. Pending mounts would have a cancel button. Successful mounts already have a "Disconnect storage" button.