limit of crawling

[adamrag]

please what is the limit of crawling using advanced search,when we do we need to use proxy to avoid block our request from twitter?
thanks.

[JayMehtaUK]

What I do is to surround (tweets = got.manager.TweetManager.getTweets(tweetCriteria)) in a try except. And if you get blocked then make your program sleep for a bit and retry getting the tweets again. It will take longer but at least your program wont stop if you get blocked and can retry at a specified interval (I did 30 seconds).

Alternatively you can use a proxy as shown in this commit: #49