You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, with dynamic websites, when page number N is fetched the scraper fetches the initial url and then "clicks" on the next-page button N times. So page 0 means fetch and click 0 times, page 1 fetch and click 1 time, page 2 fetch and click 2 times. Since there is a short delay between each click, fetching many pages can take quite some time.
This can probably be improved if we store the intermediate state somehow. Then for each page we'd only need to "click" once on the next-page button.
The text was updated successfully, but these errors were encountered:
Currently, with dynamic websites, when page number N is fetched the scraper fetches the initial url and then "clicks" on the next-page button N times. So page 0 means fetch and click 0 times, page 1 fetch and click 1 time, page 2 fetch and click 2 times. Since there is a short delay between each click, fetching many pages can take quite some time.
This can probably be improved if we store the intermediate state somehow. Then for each page we'd only need to "click" once on the next-page button.
The text was updated successfully, but these errors were encountered: