Data caching / optimisation for web apps from client side. Using Test Driven Development.
Cache Optimisation depends on multiple levels. It can be a simpler thing like reducing seek time from network to disk to RAM. It can be challenging like anticipating future data use, especially for naturally random data needs.
Done better, Cache improves performance (cache-hit vs cache-miss). Client-Server apps, especially with huge read/write requests on shared data poses such an opportunity e.g. social networking apps, stock trading, multi-player online games etc This opportunity of optimising high volume read/write using advanced caching will be focus of this project.
- Reduce seek time by caching network files to 1. RAM and 2. disk
- Cache Priority Index: anticipate future usage by observing usage history to generate ‘cache priority index’, calculated using multiple indicators like access count, last accessed, first accessed, etc.
- Dictionary will provide quick access to cached items. Can be later optimized to file with saved indexes.
- Test Driven Development needs a way to demonstrate cache performance: use mocking, etc.
- Pluggable design can help clean seperation between app and this library e.g. adapter on network & file access classes.
- Cache performance % can be measured by: cache-hit/total-requests or cache-miss/total-requests
- Priority Heap as data structure, could help optimise dynamic re-shuffling of items based on ‘cache priority index’. this optimisation, later, can be possible due to TDD.
- cache is i/o bound thus concurrency is all welcome, to me implies, request queue (producer) and thread pool (consumers).
make m repeat requests for n files, for each of n files, measure c1, c2, c3…cn, Where ci = cache-hit/total-requests. Then overall average cache performance C = (c1 + c2 + c3 … cn) / n
make n (say 1000) requests, measure time, should typically be n/100, atleast greater than n/c (where c is number of cores)