[Unity][nn.module] Rolling KVCache override method

[davidpissarra]

Part of the effort in bringing Mistral to the compilation flow with nn.Module on mlc-llm. Adding support for Rolling KVCache override method in the nn.module frontend.

cc @CharlieFRuan

[tqchen]

cc @junrushao @cyx-6

[junrushao]

I think we should move all KVCache logic to MLC LLM instead, given we expect plenty of variants of KVCache implementation. @tqchen @davidpissarra what do you guys think?

[tqchen]

Yes, i think moving to mlc is OK, we can move back if there is a point of stablization.

[davidpissarra]

I think we should move all KVCache logic to MLC LLM instead, given we expect plenty of variants of KVCache implementation. @tqchen @davidpissarra what do you guys think?

Thanks @junrushao! Makes sense to move KVCache to MLC LLM