Skip to content

Commit

Permalink
add zmq radix cache
Browse files Browse the repository at this point in the history
  • Loading branch information
josephydu committed Sep 23, 2024
1 parent b3bf7be commit d1b1035
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions python/sglang/srt/mem_cache/radix_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ def loop_for_send_tree_cache(self):
with self.change_cnt_lock:
if self.change_cnt != 0:
self.change_cnt -= 1
self.send_prefix_tree()
time.sleep(1)
self.send_prefix_tree()
time.sleep(0.5)

def send_prefix_tree(self):
# t1 = time.time()
Expand Down

0 comments on commit d1b1035

Please sign in to comment.