Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

在进行测试的时候发现生成的预测图与文本提示内容无关 #5

Open
fanjuchen opened this issue Sep 7, 2023 · 0 comments

Comments

@fanjuchen
Copy link

我在训练好的r2cnet.pth进行测试的时候,使用test文件下的一张图片进行测试,提示文本使用CLIP去生成然后转为输出尺寸,发现输出的预测图与输入的文本提示无关,以下是我的代码,请问我的操作是否有误?
`import argparse
import os

import imageio
import numpy as np
from tqdm import tqdm
from torchvision import transforms
from PIL import Image
from torch import nn
import torch
import torch.nn.functional as F
from models.utils import BasicConv2d
from models.r2cnet import Network
from data import get_dataloader
import utils.metrics as Measure
from utils.utils import load_model_params
import clip

if name == 'main':
image_size = 352
device = "cuda" if torch.cuda.is_available() else "cpu"
net = Network().to(device)
net.load_state_dict(torch.load('./r2cnet.pth')['state_dict'])
net.eval()
img_transform = transforms.Compose([
transforms.Resize((image_size, image_size)),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])
pic = img_transform(Image.open('./COD10K-CAM-3-Flying-51-Bee-2981.jpg')).unsqueeze(0).to(device)
print(pic.shape)
model, preprocess = clip.load("ViT-B/32", device=device)

text = clip.tokenize(["yellow flower"]).to(device)
text_features = model.encode_text(text).unsqueeze(-1).unsqueeze(-1).to(torch.float32)

conv1x1 = nn.Conv2d(in_channels=512, out_channels=2048, kernel_size=1).to(device)
text_features = conv1x1(text_features).to(device)
res, _ = net(x=pic, ref_x=text_features)
res = F.interpolate(res, size=(1023, 736), mode='bilinear', align_corners=False)
res = res.sigmoid().data.cpu().numpy().squeeze()
res = (res - res.min()) / (res.max() - res.min() + 1e-8)
imageio.imwrite('./test.png', res)

`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant