New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nsfw_data_scraper 图像url99%失效 #4

Open

zhangbanxian123 opened this issue Mar 3, 2020 · 6 comments

zhangbanxian123 commented Mar 3, 2020

nsfw_data_scraper 中给出来的所有url，失效的数据过多，不知道楼主有没有下载好的数据，可以提供网盘吗？另一份数据集，类别过多，我仅仅只想训练一个5种类别的模型，，且没有明确定义的pron类别。

Owner

yangbisheng2009 commented Mar 4, 2020

去这个路径下 nsfw_data_scraper-master/raw_data：找到每个类别对应的 url文件。自己写个python脚本，循环下载url对应的jpg文件即可。别用它官方的下载脚本，那个太麻烦。这样就不会各种失败了

Author

zhangbanxian123 commented Mar 4, 2020

嗯，就是自己写的脚本，失效过多。所以才想向你申请数据^-^

Owner

yangbisheng2009 commented Mar 4, 2020

哭。。。我的MEGA数据过期了，目前没法下载
不过用 nsfw_data_scraper 这个数据，即使有失败的，也是能够满足测试要求的
如果你想真实解决线上问题，这份数据是远远不够的，需要根据你的适用场景单独标数据

Author

zhangbanxian123 commented Mar 5, 2020

请问一下您下载数据之后是不是对各个类别数据进行整理过后再训练的？因为数据中有很多脏数据

Owner

yangbisheng2009 commented Mar 5, 2020

是的，我对下载的图片做了些筛选，然后又额外标注了一些数据，才能达到不错的效果

yangbisheng2009 closed this as completed

Owner

yangbisheng2009 commented Mar 30, 2020

我记起来了，打不开的原因，可能是你需要翻墙。国内当然打不开

yangbisheng2009 reopened this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment