Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Follow links to next page #1

Open
Mappboy opened this issue Jun 10, 2016 · 1 comment
Open

Follow links to next page #1

Mappboy opened this issue Jun 10, 2016 · 1 comment
Labels
enhancement

Comments

@Mappboy
Copy link
Contributor

Mappboy commented Jun 10, 2016

Currently only processing the front page till we are sure everything is sweet. However I can't seem to find the link to go to the next page. Might be able to just use QualitemContentRollupPage={page_num}& but then we have to keep track of page numbers boooo.
@keithamoss
Copy link
Contributor

That's what I ended up doing for the maps scraper :) Not elegant, but it works!

Can't scrapy find the with the text "Next"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@keithamoss @Mappboy