How does cache get refresh?

[HeidXC]

Dear maintainers,

As these are not described in docs, and I'm wandering how cache for a single trader get refreshed?
Here is my scenario:in Binance I retrieve last 300 hours kline for all tickers at 00:00 every day, and filter out 20 tickers based on those klines, and subscribe trades for these 20 tickers.
I'm wondering how does these klines and trades stored in cache and how and when they get refresh? I hope these klines to be one time usage, and trades to be last for 1 hour, or is there anyway I can control the cache? or I need to store it in my own mem db?

Thanks in advance.

[HeidXC]

I also see there is flush_db in cache, does that mean I can wire a db (redis or mongo) behind the cache system?

[cjdsellers]

Hey @VeraLyu

The cache is rather simple, for each data type there is just a deque per instrument (or bar type for bars) - with a default length of 10_000 (configurable).

When you make a request for historical ticks (or bars), the deque is replaced by them on arrival, otherwise as each tick (or bar) hits the DataEngine the tick is added to the Cache, which appends to the left side of the queue, and will eventually fall off the right end as more are added.

I think for a simple use case where you just want to maintain a hot cache of the most recent data, then what's built in will work for you. Notably though, the data is not persisted in the cache (generally only execution data is persisted, as well as currencies and instruments).

To complicate the matter there is a message bus which can be backed by Redis - which will write any publishable message to a Redis stream, acting as a makeshift durable message broker. This can be autotrimmed to a specific lookback window, there's some write up on this in the docs.

https://docs.nautilustrader.io/concepts/message_bus.html

I hope that helps!

[HeidXC]

Thanks a lot! I'll take a closer look