Add local caching for crawled pages to enhance development efficiency

[matecsaj]

It would be a huge time saver during development if the website only needed to be crawled once. While iterating on my data extraction code, I wish I could load the pages from my local disk instead of the Internet.

During development, repeatedly crawling the website to test data extraction code is time-consuming and inefficient. A caching mechanism that stores previously downloaded pages on the local disk would be a huge time saver. Instead of fetching the pages from the Internet, the development environment could simply load them from local storage.

This approach would not only speed up the iterative development process but also reduce load on the website's server. Such a feature would be particularly useful for debugging and refining data extraction scripts.

[vdusek]

Hi, this sounds reasonable, while it is not currently on our roadmap. We will revise it in the future.