How to use trained model_epoch_best.pth

[kimmich0615]

Hi author, i have trained the model with sam2.1_hiera_large.pt and get the model_epoch_best.pth . However, when i applied the pth file into the SAM2 model, it went wrong.

File "/home/user8/anaconda3/envs/zwr_sam2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2584, in load_state_dict
raise RuntimeError(
RuntimeError: Error(s) in loading state_dict for SAM2Base:
Missing key(s) in state_dict: "maskmem_tpos_enc", "no_mem_embed", "no_mem_pos_enc", "no_obj_ptr", "no_obj_embed_spatial", "mask_downsample.weight", "mask_downsample.bias", "memory_attention.layers.0.self_attn.q_proj.weight", "memory_attention.layers.0.self_attn.q_proj.bias", "memory_attention.layers.0.self_attn.k_proj.weight", "memory_attention.layers.0.self_attn.k_proj.bias", "memory_attention.layers.0.self_attn.v_proj.weight", "memory_attention.layers.0.self_attn.v_proj.bias", "memory_attention.layers.0.self_attn.out_proj.weight", "memory_attention.layers.0.self_attn.out_proj.bias", "memory_attention.layers.0.cross_attn_image.q_proj.weight", "memory_attention.layers.0.cross_attn_image.q_proj.bias", "memory_attention.layers.0.cross_attn_image.k_proj.weight", "memory_attention.layers.0.cross_attn_image.k_proj.bias", "memory_attention.layers.0.cross_attn_image.v_proj.weight", "memory_attention.layers.0.cross_attn_image.v_proj.bias", "memory_attention.layers.0.cross_attn_image.out_proj.weight", "memory_attention.layers.0.cross_attn_image.out_proj.bias", "memory_attention.layers.0.linear1.weight", "memory_attention.layers.0.linear1.bias", "memory_attention.layers.0.linear2.weight", "memory_attention.layers.0.linear2.bias", "memory_attention.layers.0.norm1.weight", "memory_attention.layers.0.norm1.bias", "memory_attention.layers.0.norm2.weight", "memory_attention.layers.0.norm2.bias", "memory_attention.layers.0.norm3.weight", "memory_attention.layers.0.norm3.bias", "memory_attention.layers.1.self_attn.q_proj.weight", "memory_attention.layers.1.self_attn.q_proj.bias", "memory_attention.layers.1.self_attn.k_proj.weight", "memory_attention.layers.1.self_attn.k_proj.bias", "memory_attention.layers.1.self_attn.v_proj.weight", "memory_attention.layers.1.self_attn.v_proj.bias", "memory_attention.layers.1.self_attn.out_proj.weight", "memory_attention.layers.1.self_attn.out_proj.bias", "memory_attention.layers.1.cross_attn_image.q_proj.weight", "memory_attention.layers.1.cross_attn_image.q_proj.bias", "memory_attention.layers.1.cross_attn_image.k_proj.weight", "memory_attention.layers.1.cross_attn_image.k_proj.bias", "memory_attention.layers.1.cross_attn_image.v_proj.weight", "memory_attention.layers.1.cross_attn_image.v_proj.bias", "memory_attention.layers.1.cross_attn_image.out_proj.weight", "memory_attention.layers.1.cross_attn_image.out_proj.bias", "memory_attention.layers.1.linear1.weight", "memory_attention.layers.1.linear1.bias", "memory_attention.layers.1.linear2.weight", "memory_attention.layers.1.linear2.bias", "memory_attention.layers.1.norm1.weight", "memory_attention.layers.1.norm1.bias", "memory_attention.layers.1.norm2.weight", "memory_attention.layers.1.norm2.bias", "memory_attention.layers.1.norm3.weight", "memory_attention.layers.1.norm3.bias", "memory_attention.layers.2.self_attn.q_proj.weight", "memory_attention.layers.2.self_attn.q_proj.bias", "memory_attention.layers.2.self_attn.k_proj.weight", "memory_attention.layers.2.self_attn.k_proj.bias", "memory_attention.layers.2.self_attn.v_proj.weight", "memory_attention.layers.2.self_attn.v_proj.bias", "memory_attention.layers.2.self_attn.out_proj.weight", "memory_attention.layers.2.self_attn.out_proj.bias", "memory_attention.layers.2.cross_attn_image.q_proj.weight", "memory_attention.layers.2.cross_attn_image.q_proj.bias", "memory_attention.layers.2.cross_attn_image.k_proj.weight", "memory_attention.layers.2.cross_attn_image.k_proj.bias", "memory_attention.layers.2.cross_attn_image.v_proj.weight", "memory_attention.layers.2.cross_attn_image.v_proj.bias", "memory_attention.layers.2.cross_attn_image.out_proj.weight", "memory_attention.layers.2.cross_attn_image.out_proj.bias", "memory_attention.layers.2.linear1.weight", "memory_attention.layers.2.linear1.bias", "memory_attention.layers.2.linear2.weight", "memory_attention.layers.2.linear2.bias", "memory_attention.layers.2.norm1.weight", "memory_attention.layers.2.norm1.bias", "memory_attention.layers.2.norm2.weight", "memory_attention.layers.2.norm2.bias", "memory_attention.layers.2.norm3.weight", "memory_attention.layers.2.norm3.bias", "memory_attention.layers.3.self_attn.q_proj.weight", "memory_attention.layers.3.self_attn.q_proj.bias", "memory_attention.layers.3.self_attn.k_proj.weight", "memory_attention.layers.3.self_attn.k_proj.bias", "memory_attention.layers.3.self_attn.v_proj.weight", "memory_attention.layers.3.self_attn.v_proj.bias", "memory_attention.layers.3.self_attn.out_proj.weight", "memory_attention.layers.3.self_attn.out_proj.bias", "memory_attention.layers.3.cross_attn_image.q_proj.weight", "memory_attention.layers.3.cross_attn_image.q_proj.bias", "memory_attention.layers.3.cross_attn_image.k_proj.weight", "memory_attention.layers.3.cross_attn_image.k_proj.bias", "memory_attention.layers.3.cross_attn_image.v_proj.weight", "memory_attention.layers.3.cross_attn_image.v_proj.bias", "memory_attention.layers.3.cross_attn_image.out_proj.weight", "memory_attention.layers.3.cross_attn_image.out_proj.bias", "memory_attention.layers.3.linear1.weight", "memory_attention.layers.3.linear1.bias", "memory_attention.layers.3.linear2.weight", "memory_attention.layers.3.linear2.bias", "memory_attention.layers.3.norm1.weight", "memory_attention.layers.3.norm1.bias", "memory_attention.layers.3.norm2.weight", "memory_attention.layers.3.norm2.bias", "memory_attention.layers.3.norm3.weight", "memory_attention.layers.3.norm3.bias", "memory_attention.norm.weight", "memory_attention.norm.bias", "memory_encoder.mask_downsampler.encoder.0.weight", "memory_encoder.mask_downsampler.encoder.0.bias", "memory_encoder.mask_downsampler.encoder.1.weight", "memory_encoder.mask_downsampler.encoder.1.bias", "memory_encoder.mask_downsampler.encoder.3.weight", "memory_encoder.mask_downsampler.encoder.3.bias", "memory_encoder.mask_downsampler.encoder.4.weight", "memory_encoder.mask_downsampler.encoder.4.bias", "memory_encoder.mask_downsampler.encoder.6.weight", "memory_encoder.mask_downsampler.encoder.6.bias", "memory_encoder.mask_downsampler.encoder.7.weight", "memory_encoder.mask_downsampler.encoder.7.bias", "memory_encoder.mask_downsampler.encoder.9.weight", "memory_encoder.mask_downsampler.encoder.9.bias", "memory_encoder.mask_downsampler.encoder.10.weight", "memory_encoder.mask_downsampler.encoder.10.bias", "memory_encoder.mask_downsampler.encoder.12.weight", "memory_encoder.mask_downsampler.encoder.12.bias", "memory_encoder.pix_feat_proj.weight", "memory_encoder.pix_feat_proj.bias", "memory_encoder.fuser.layers.0.gamma", "memory_encoder.fuser.layers.0.dwconv.weight", "memory_encoder.fuser.layers.0.dwconv.bias", "memory_encoder.fuser.layers.0.norm.weight", "memory_encoder.fuser.layers.0.norm.bias", "memory_encoder.fuser.layers.0.pwconv1.weight", "memory_encoder.fuser.layers.0.pwconv1.bias", "memory_encoder.fuser.layers.0.pwconv2.weight", "memory_encoder.fuser.layers.0.pwconv2.bias", "memory_encoder.fuser.layers.1.gamma", "memory_encoder.fuser.layers.1.dwconv.weight", "memory_encoder.fuser.layers.1.dwconv.bias", "memory_encoder.fuser.layers.1.norm.weight", "memory_encoder.fuser.layers.1.norm.bias", "memory_encoder.fuser.layers.1.pwconv1.weight", "memory_encoder.fuser.layers.1.pwconv1.bias", "memory_encoder.fuser.layers.1.pwconv2.weight", "memory_encoder.fuser.layers.1.pwconv2.bias", "memory_encoder.out_proj.weight", "memory_encoder.out_proj.bias", "sam_prompt_encoder.pe_layer.positional_encoding_gaussian_matrix", "sam_prompt_encoder.point_embeddings.0.weight", "sam_prompt_encoder.point_embeddings.1.weight", "sam_prompt_encoder.point_embeddings.2.weight", "sam_prompt_encoder.point_embeddings.3.weight", "sam_prompt_encoder.not_a_point_embed.weight", "sam_prompt_encoder.mask_downscaling.0.weight", "sam_prompt_encoder.mask_downscaling.0.bias", "sam_prompt_encoder.mask_downscaling.1.weight", "sam_prompt_encoder.mask_downscaling.1.bias", "sam_prompt_encoder.mask_downscaling.3.weight", "sam_prompt_encoder.mask_downscaling.3.bias", "sam_prompt_encoder.mask_downscaling.4.weight", "sam_prompt_encoder.mask_downscaling.4.bias", "sam_prompt_encoder.mask_downscaling.6.weight", "sam_prompt_encoder.mask_downscaling.6.bias", "sam_prompt_encoder.no_mask_embed.weight", "sam_mask_decoder.transformer.layers.0.self_attn.q_proj.weight", "sam_mask_decoder.transformer.layers.0.self_attn.q_proj.bias", "sam_mask_decoder.transformer.layers.0.self_attn.k_proj.weight", "sam_mask_decoder.transformer.layers.0.self_attn.k_proj.bias", "sam_mask_decoder.transformer.layers.0.self_attn.v_proj.weight", "sam_mask_decoder.transformer.layers.0.self_attn.v_proj.bias", "sam_mask_decoder.transformer.layers.0.self_attn.out_proj.weight", "sam_mask_decoder.transformer.layers.0.self_attn.out_proj.bias", "sam_mask_decoder.transformer.layers.0.norm1.weight", "sam_mask_decoder.transformer.layers.0.norm1.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.q_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.q_proj.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.k_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.k_proj.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.v_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.v_proj.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.out_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_token_to_image.out_proj.bias", "sam_mask_decoder.transformer.layers.0.norm2.weight", "sam_mask_decoder.transformer.layers.0.norm2.bias", "sam_mask_decoder.transformer.layers.0.mlp.layers.0.weight", "sam_mask_decoder.transformer.layers.0.mlp.layers.0.bias", "sam_mask_decoder.transformer.layers.0.mlp.layers.1.weight", "sam_mask_decoder.transformer.layers.0.mlp.layers.1.bias", "sam_mask_decoder.transformer.layers.0.norm3.weight", "sam_mask_decoder.transformer.layers.0.norm3.bias", "sam_mask_decoder.transformer.layers.0.norm4.weight", "sam_mask_decoder.transformer.layers.0.norm4.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.q_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.q_proj.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.k_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.k_proj.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.v_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.v_proj.bias", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.out_proj.weight", "sam_mask_decoder.transformer.layers.0.cross_attn_image_to_token.out_proj.bias", "sam_mask_decoder.transformer.layers.1.self_attn.q_proj.weight", "sam_mask_decoder.transformer.layers.1.self_attn.q_proj.bias", "sam_mask_decoder.transformer.layers.1.self_attn.k_proj.weight", "sam_mask_decoder.transformer.layers.1.self_attn.k_proj.bias", "sam_mask_decoder.transformer.layers.1.self_attn.v_proj.weight", "sam_mask_decoder.transformer.layers.1.self_attn.v_proj.bias", "sam_mask_decoder.transformer.layers.1.self_attn.out_proj.weight", "sam_mask_decoder.transformer.layers.1.self_attn.out_proj.bias", "sam_mask_decoder.transformer.layers.1.norm1.weight", "sam_mask_decoder.transformer.layers.1.norm1.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.q_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.q_proj.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.k_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.k_proj.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.v_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.v_proj.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.out_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_token_to_image.out_proj.bias", "sam_mask_decoder.transformer.layers.1.norm2.weight", "sam_mask_decoder.transformer.layers.1.norm2.bias", "sam_mask_decoder.transformer.layers.1.mlp.layers.0.weight", "sam_mask_decoder.transformer.layers.1.mlp.layers.0.bias", "sam_mask_decoder.transformer.layers.1.mlp.layers.1.weight", "sam_mask_decoder.transformer.layers.1.mlp.layers.1.bias", "sam_mask_decoder.transformer.layers.1.norm3.weight", "sam_mask_decoder.transformer.layers.1.norm3.bias", "sam_mask_decoder.transformer.layers.1.norm4.weight", "sam_mask_decoder.transformer.layers.1.norm4.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.q_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.q_proj.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.k_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.k_proj.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.v_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.v_proj.bias", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.out_proj.weight", "sam_mask_decoder.transformer.layers.1.cross_attn_image_to_token.out_proj.bias", "sam_mask_decoder.transformer.final_attn_token_to_image.q_proj.weight", "sam_mask_decoder.transformer.final_attn_token_to_image.q_proj.bias", "sam_mask_decoder.transformer.final_attn_token_to_image.k_proj.weight", "sam_mask_decoder.transformer.final_attn_token_to_image.k_proj.bias", "sam_mask_decoder.transformer.final_attn_token_to_image.v_proj.weight", "sam_mask_decoder.transformer.final_attn_token_to_image.v_proj.bias", "sam_mask_decoder.transformer.final_attn_token_to_image.out_proj.weight", "sam_mask_decoder.transformer.final_attn_token_to_image.out_proj.bias", "sam_mask_decoder.transformer.norm_final_attn.weight", "sam_mask_decoder.transformer.norm_final_attn.bias", "sam_mask_decoder.iou_token.weight", "sam_mask_decoder.mask_tokens.weight", "sam_mask_decoder.obj_score_token.weight", "sam_mask_decoder.output_upscaling.0.weight", "sam_mask_decoder.output_upscaling.0.bias", "sam_mask_decoder.output_upscaling.1.weight", "sam_mask_decoder.output_upscaling.1.bias", "sam_mask_decoder.output_upscaling.3.weight", "sam_mask_decoder.output_upscaling.3.bias", "sam_mask_decoder.conv_s0.weight", "sam_mask_decoder.conv_s0.bias", "sam_mask_decoder.conv_s1.weight", "sam_mask_decoder.conv_s1.bias", "sam_mask_decoder.output_hypernetworks_mlps.0.layers.0.weight", "sam_mask_decoder.output_hypernetworks_mlps.0.layers.0.bias", "sam_mask_decoder.output_hypernetworks_mlps.0.layers.1.weight", "sam_mask_decoder.output_hypernetworks_mlps.0.layers.1.bias", "sam_mask_decoder.output_hypernetworks_mlps.0.layers.2.weight", "sam_mask_decoder.output_hypernetworks_mlps.0.layers.2.bias", "sam_mask_decoder.output_hypernetworks_mlps.1.layers.0.weight", "sam_mask_decoder.output_hypernetworks_mlps.1.layers.0.bias", "sam_mask_decoder.output_hypernetworks_mlps.1.layers.1.weight", "sam_mask_decoder.output_hypernetworks_mlps.1.layers.1.bias", "sam_mask_decoder.output_hypernetworks_mlps.1.layers.2.weight", "sam_mask_decoder.output_hypernetworks_mlps.1.layers.2.bias", "sam_mask_decoder.output_hypernetworks_mlps.2.layers.0.weight", "sam_mask_decoder.output_hypernetworks_mlps.2.layers.0.bias", "sam_mask_decoder.output_hypernetworks_mlps.2.layers.1.weight", "sam_mask_decoder.output_hypernetworks_mlps.2.layers.1.bias", "sam_mask_decoder.output_hypernetworks_mlps.2.layers.2.weight", "sam_mask_decoder.output_hypernetworks_mlps.2.layers.2.bias", "sam_mask_decoder.output_hypernetworks_mlps.3.layers.0.weight", "sam_mask_decoder.output_hypernetworks_mlps.3.layers.0.bias", "sam_mask_decoder.output_hypernetworks_mlps.3.layers.1.weight", "sam_mask_decoder.output_hypernetworks_mlps.3.layers.1.bias", "sam_mask_decoder.output_hypernetworks_mlps.3.layers.2.weight", "sam_mask_decoder.output_hypernetworks_mlps.3.layers.2.bias", "sam_mask_decoder.iou_prediction_head.layers.0.weight", "sam_mask_decoder.iou_prediction_head.layers.0.bias", "sam_mask_decoder.iou_prediction_head.layers.1.weight", "sam_mask_decoder.iou_prediction_head.layers.1.bias", "sam_mask_decoder.iou_prediction_head.layers.2.weight", "sam_mask_decoder.iou_prediction_head.layers.2.bias", "sam_mask_decoder.pred_obj_score_head.layers.0.weight", "sam_mask_decoder.pred_obj_score_head.layers.0.bias", "sam_mask_decoder.pred_obj_score_head.layers.1.weight", "sam_mask_decoder.pred_obj_score_head.layers.1.bias", "sam_mask_decoder.pred_obj_score_head.layers.2.weight", "sam_mask_decoder.pred_obj_score_head.layers.2.bias", "obj_ptr_proj.layers.0.weight", "obj_ptr_proj.layers.0.bias", "obj_ptr_proj.layers.1.weight", "obj_ptr_proj.layers.1.bias", "obj_ptr_proj.layers.2.weight", "obj_ptr_proj.layers.2.bias", "obj_ptr_tpos_proj.weight", "obj_ptr_tpos_proj.bias".
Unexpected key(s) in state_dict: "mask_decoder.transformer.layers.0.self_attn.q_proj.weight", "mask_decoder.transformer.layers.0.self_attn.q_proj.bias", "mask_decoder.transformer.layers.0.self_attn.k_proj.weight", "mask_decoder.transformer.layers.0.self_attn.k_proj.bias", "mask_decoder.transformer.layers.0.self_attn.v_proj.weight", "mask_decoder.transformer.layers.0.self_attn.v_proj.bias", "mask_decoder.transformer.layers.0.self_attn.out_proj.weight", "mask_decoder.transformer.layers.0.self_attn.out_proj.bias", "mask_decoder.transformer.layers.0.norm1.weight", "mask_decoder.transformer.layers.0.norm1.bias", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.q_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.q_proj.bias", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.k_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.k_proj.bias", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.v_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.v_proj.bias", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.out_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_token_to_image.out_proj.bias", "mask_decoder.transformer.layers.0.norm2.weight", "mask_decoder.transformer.layers.0.norm2.bias", "mask_decoder.transformer.layers.0.mlp.lin1.weight", "mask_decoder.transformer.layers.0.mlp.lin1.bias", "mask_decoder.transformer.layers.0.mlp.lin2.weight", "mask_decoder.transformer.layers.0.mlp.lin2.bias", "mask_decoder.transformer.layers.0.norm3.weight", "mask_decoder.transformer.layers.0.norm3.bias", "mask_decoder.transformer.layers.0.norm4.weight", "mask_decoder.transformer.layers.0.norm4.bias", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.q_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.q_proj.bias", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.k_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.k_proj.bias", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.v_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.v_proj.bias", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.out_proj.weight", "mask_decoder.transformer.layers.0.cross_attn_image_to_token.out_proj.bias", "mask_decoder.transformer.layers.1.self_attn.q_proj.weight", "mask_decoder.transformer.layers.1.self_attn.q_proj.bias", "mask_decoder.transformer.layers.1.self_attn.k_proj.weight", "mask_decoder.transformer.layers.1.self_attn.k_proj.bias", "mask_decoder.transformer.layers.1.self_attn.v_proj.weight", "mask_decoder.transformer.layers.1.self_attn.v_proj.bias", "mask_decoder.transformer.layers.1.self_attn.out_proj.weight", "mask_decoder.transformer.layers.1.self_attn.out_proj.bias", "mask_decoder.transformer.layers.1.norm1.weight", "mask_decoder.transformer.layers.1.norm1.bias", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.q_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.q_proj.bias", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.k_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.k_proj.bias", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.v_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.v_proj.bias", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.out_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_token_to_image.out_proj.bias", "mask_decoder.transformer.layers.1.norm2.weight", "mask_decoder.transformer.layers.1.norm2.bias", "mask_decoder.transformer.layers.1.mlp.lin1.weight", "mask_decoder.transformer.layers.1.mlp.lin1.bias", "mask_decoder.transformer.layers.1.mlp.lin2.weight", "mask_decoder.transformer.layers.1.mlp.lin2.bias", "mask_decoder.transformer.layers.1.norm3.weight", "mask_decoder.transformer.layers.1.norm3.bias", "mask_decoder.transformer.layers.1.norm4.weight", "mask_decoder.transformer.layers.1.norm4.bias", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.q_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.q_proj.bias", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.k_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.k_proj.bias", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.v_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.v_proj.bias", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.out_proj.weight", "mask_decoder.transformer.layers.1.cross_attn_image_to_token.out_proj.bias", "mask_decoder.transformer.final_attn_token_to_image.q_proj.weight", "mask_decoder.transformer.final_attn_token_to_image.q_proj.bias", "mask_decoder.transformer.final_attn_token_to_image.k_proj.weight", "mask_decoder.transformer.final_attn_token_to_image.k_proj.bias", "mask_decoder.transformer.final_attn_token_to_image.v_proj.weight", "mask_decoder.transformer.final_attn_token_to_image.v_proj.bias", "mask_decoder.transformer.final_attn_token_to_image.out_proj.weight", "mask_decoder.transformer.final_attn_token_to_image.out_proj.bias", "mask_decoder.transformer.norm_final_attn.weight", "mask_decoder.transformer.norm_final_attn.bias", "mask_decoder.iou_token.weight", "mask_decoder.mask_tokens.weight", "mask_decoder.obj_score_token.weight", "mask_decoder.output_upscaling.0.weight", "mask_decoder.output_upscaling.0.bias", "mask_decoder.output_upscaling.1.weight", "mask_decoder.output_upscaling.1.bias", "mask_decoder.output_upscaling.3.weight", "mask_decoder.output_upscaling.3.bias", "mask_decoder.conv_s0.weight", "mask_decoder.conv_s0.bias", "mask_decoder.conv_s1.weight", "mask_decoder.conv_s1.bias", "mask_decoder.output_hypernetworks_mlps.0.layers.0.weight", "mask_decoder.output_hypernetworks_mlps.0.layers.0.bias", "mask_decoder.output_hypernetworks_mlps.0.layers.1.weight", "mask_decoder.output_hypernetworks_mlps.0.layers.1.bias", "mask_decoder.output_hypernetworks_mlps.0.layers.2.weight", "mask_decoder.output_hypernetworks_mlps.0.layers.2.bias", "mask_decoder.output_hypernetworks_mlps.1.layers.0.weight", "mask_decoder.output_hypernetworks_mlps.1.layers.0.bias", "mask_decoder.output_hypernetworks_mlps.1.layers.1.weight", "mask_decoder.output_hypernetworks_mlps.1.layers.1.bias", "mask_decoder.output_hypernetworks_mlps.1.layers.2.weight", "mask_decoder.output_hypernetworks_mlps.1.layers.2.bias", "mask_decoder.iou_prediction_head.layers.0.weight", "mask_decoder.iou_prediction_head.layers.0.bias", "mask_decoder.iou_prediction_head.layers.1.weight", "mask_decoder.iou_prediction_head.layers.1.bias", "mask_decoder.iou_prediction_head.layers.2.weight", "mask_decoder.iou_prediction_head.layers.2.bias", "mask_decoder.pred_obj_score_head.weight", "mask_decoder.pred_obj_score_head.bias", "pe_layer.positional_encoding_gaussian_matrix", "no_mask_embed.weight", "image_encoder.trunk.patch_embed1.proj.weight", "image_encoder.trunk.patch_embed1.proj.bias", "image_encoder.trunk.patch_embed1.norm.weight", "image_encoder.trunk.patch_embed1.norm.bias", "image_encoder.trunk.patch_embed2.proj.weight", "image_encoder.trunk.patch_embed2.proj.bias", "image_encoder.trunk.patch_embed2.norm.weight", "image_encoder.trunk.patch_embed2.norm.bias", "image_encoder.trunk.patch_embed3.proj.weight", "image_encoder.trunk.patch_embed3.proj.bias", "image_encoder.trunk.patch_embed3.norm.weight", "image_encoder.trunk.patch_embed3.norm.bias", "image_encoder.trunk.patch_embed4.proj.weight", "image_encoder.trunk.patch_embed4.proj.bias", "image_encoder.trunk.patch_embed4.norm.weight", "image_encoder.trunk.patch_embed4.norm.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator1.proj.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator1.proj.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator1.norm.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator1.norm.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator2.proj.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator2.proj.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator2.norm.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator2.norm.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator3.proj.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator3.proj.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator3.norm.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator3.norm.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator4.proj.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator4.proj.bias", "image_encoder.trunk.prompt_generator.handcrafted_generator4.norm.weight", "image_encoder.trunk.prompt_generator.handcrafted_generator4.norm.bias", "image_encoder.trunk.prompt_generator.embedding_generator1.weight", "image_encoder.trunk.prompt_generator.embedding_generator1.bias", "image_encoder.trunk.prompt_generator.embedding_generator2.weight", "image_encoder.trunk.prompt_generator.embedding_generator2.bias", "image_encoder.trunk.prompt_generator.embedding_generator3.weight", "image_encoder.trunk.prompt_generator.embedding_generator3.bias", "image_encoder.trunk.prompt_generator.embedding_generator4.weight", "image_encoder.trunk.prompt_generator.embedding_generator4.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp1_0.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp1_0.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp1_1.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp1_1.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp1_2.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp1_2.0.bias", "image_encoder.trunk.prompt_generator.shared_mlp1.weight", "image_encoder.trunk.prompt_generator.shared_mlp1.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_0.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_0.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_1.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_1.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_2.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_2.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_3.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_3.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_4.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_4.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_5.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_5.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp2_6.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp2_6.0.bias", "image_encoder.trunk.prompt_generator.shared_mlp2.weight", "image_encoder.trunk.prompt_generator.shared_mlp2.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_0.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_0.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_1.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_1.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_2.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_2.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_3.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_3.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_4.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_4.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_5.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_5.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_6.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_6.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_7.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_7.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_8.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_8.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_9.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_9.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_10.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_10.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_11.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_11.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_12.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_12.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_13.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_13.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_14.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_14.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_15.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_15.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_16.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_16.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_17.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_17.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_18.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_18.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_19.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_19.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_20.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_20.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_21.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_21.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_22.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_22.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_23.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_23.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_24.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_24.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_25.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_25.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_26.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_26.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_27.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_27.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_28.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_28.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_29.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_29.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_30.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_30.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_31.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_31.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_32.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_32.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_33.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_33.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_34.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_34.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_35.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_35.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp3_36.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp3_36.0.bias", "image_encoder.trunk.prompt_generator.shared_mlp3.weight", "image_encoder.trunk.prompt_generator.shared_mlp3.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp4_0.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp4_0.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp4_1.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp4_1.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp4_2.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp4_2.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp4_3.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp4_3.0.bias", "image_encoder.trunk.prompt_generator.lightweight_mlp4_4.0.weight", "image_encoder.trunk.prompt_generator.lightweight_mlp4_4.0.bias", "image_encoder.trunk.prompt_generator.shared_mlp4.weight", "image_encoder.trunk.prompt_generator.shared_mlp4.bias".

[tianrun-chen]

Kindly ask that whether you use the inference code we provided or the inference code from SAM2? (these two are not compatible)