Prompting v2.0.0 - streaming feedback

[steffencruz]

As we continue development on SN1, we have our sights on new and exciting ideas. A part of our efforts has been dedicated to incorporating streaming into the response pipeline, and we are finally ready to bring this to the community. Streaming, for those unaware, will give miners the ability to send chunks of data back to the validator during inference, allowing for miners running slower machines / larger models more time to acquire a partial response before a timeout condition.

In light of this change, we will be depreciating all support for non-stream mining. Therefore, every miner must use the streaming communication protocol.

Streaming fundamentally changes the way communication is done between the miner and the validator, and we expect the community will need time to implement changes to their miners for those running custom implementations. Next week, we will be releasing information regarding a new testnet environment that will ONLY support streaming. Here, miners running custom mining implementations can test out their systems and ensure that they work. If you are in need of test tao, please reach out to @mccrindlebrian or @bkb2135 for some.

Streaming is planning to be released in version 2.0 of Prompting, and we will give the community sufficient time to become acquainted with this new paradigm. Reach out if you have any questions or major concerns.

We would love to hear your feedback and ideas regarding the transition to streaming SN1.

This will be a 2 week risk-free experiment for all participants, which is featured on testnet at netuid 102. In order to run a streaming miner you must be running on pre-staging. The testnet experiment is planned to stay live until March 25th, after which we are planning to launch the changes to mainnet.