Skip to content

sagekobe/scrapy_whaley

Repository files navigation

Scrapy_Whaley

1、The subject for these codes is crawling the website of the Whaley's projector in Tmall to obtain its items.

2、The items of the related projectors include the products' name, price, quantities of the comments and sales volume.

3、There are a few of the products about the Whaley's projectors in Tmall, so the final table only contains 16 records. More deteils of the table in databases could see the picture(whaleyinformation.png)

4、The methods used here mainly include 'Scrapy + Sqlalchemy + Mysql'.

5、There is a file named 'whayelwt_comments.py'. This file is used to crawl the products' comments for text analysis.It's a independent file that don't include into the Scrapy structure.

6、Maybe it's a simplest frame structure, no special function inluded. The next steps are to built the method to break the anti-crawling , follow up the next pages and so on.

 

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages