Skip to content

Commit

Permalink
Update vllm/model_executor/models/glm.py
Browse files Browse the repository at this point in the history
Co-authored-by: Cyrus Leung <[email protected]>
  • Loading branch information
sixsixcoder and DarkLight1337 authored Nov 28, 2024
1 parent 7e57962 commit cb127dc
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions vllm/model_executor/models/glm.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
# 2. There is no bias for o_proj in attention
for layer in self.model.layers:
if not isinstance(layer, PPMissingLayer):
layer.self_attn.rotary_emb.rotary_dim = (
layer.self_attn.rotary_emb.rotary_dim // 2)
layer.self_attn.rotary_emb.rotary_dim //= 2
layer.self_attn.rotary_emb.is_neox_style = False
layer.self_attn.o_proj.bias = None
layer.self_attn.o_proj.skip_bias_add = True

0 comments on commit cb127dc

Please sign in to comment.