Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variable number of reviews returned #676

Open
gianlucascoccia opened this issue Apr 3, 2024 · 2 comments
Open

Variable number of reviews returned #676

gianlucascoccia opened this issue Apr 3, 2024 · 2 comments

Comments

@gianlucascoccia
Copy link

The reviews methods seems to return a different number of reviews on each call.
I slightly modified the example in the documentation to reproduce the issue:

import gplay from "google-play-scraper";

gplay.reviews({
  appId: 'com.facebook.katana',
  num: 3000
}).then(function(apps){
  console.log(apps.data.length);
}, console.log);

When called multiple times the output is :

300
300
150
150
150
150
1200

This used to work in the past, getting hundreds of thousands of reviews in a single call

@gianlucascoccia
Copy link
Author

In this sister project they have encountered the same issue.
Here I provided a workaround around the issue that appears to be working.
I would submit a PR to fix the issue but my JavaScript skills are fairly limited.

@petskratt
Copy link
Contributor

petskratt commented Apr 6, 2024

comment edited - pagination is built-in, just implement cookieJar

Variance in response seems to come from Google's loadbalancer picking different backend that doesn't know about our nextPaginationToken and responds with error. Providing NID cookie (that stores user prefs & ad data) gives me constant results.

PR on the way, in a meanwhile you can test with my fork.

petskratt pushed a commit to petskratt/google-play-scraper that referenced this issue Apr 7, 2024
petskratt pushed a commit to petskratt/google-play-scraper that referenced this issue Apr 8, 2024
For detection of facundoolano#676 (less than actual number of requests returned, different response count on each request).

Using app with large number of reviews sorted by helpfulness (assuming this set does not change as frequently as just reviews), num: 1500 is reasonably fast and we get 2 x 10 paged requests (of 150 results) that ought to prove paging works.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants