Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process pooling / re-using the same Puppeteer instance for multiple screenshots? #284

Open
olalonde opened this issue Oct 27, 2016 · 8 comments

Comments

@olalonde
Copy link

Is there any way to re-use the same phantomjs instance for multiple screenshots if the list of URLs aren't known in advance?

@SamVerschueren
Copy link
Contributor

SamVerschueren commented Oct 27, 2016

Duplicate of #242 and #282.

@olalonde
Copy link
Author

It's not really, unless the answer is that "no" this won't be implemented. What I'm looking for is to have a "single" phantomjs across all screenshots and this is not possible at the moment.

@SamVerschueren
Copy link
Contributor

It was requested here though #282 (comment).

As @kevva points out, we don't think it would be possible to change the URL while keeping the process alive.

@olalonde
Copy link
Author

I've actually just started working on that using node-phantomjs: https://gist.github.com/a1f2d6f56db7d8bbf0f8616ab5018346 (WIP!)

It re-uses the same phatomjs instance (process) for all screenshots within the same Node.js process. It would require a pretty big refactoring of pageres as far as I can tell, so I wrote it from scratch. I'd be happy to contribute to pageres but the way it's designed doesn't make it easy.

@SamVerschueren
Copy link
Contributor

SamVerschueren commented Oct 27, 2016

I don't think it's worth the effort right now. Once headless chrome lands (not sure when it will happen though), we will switch to that one. So then we could take a look at how it could be done in headless chrome.

Unless @kevva or @sindresorhus have another opinion on this one.

@olalonde
Copy link
Author

olalonde commented Oct 27, 2016

Ok, sure. FWIW, after switching from pageres to the code I just pasted, I got a huge performance boost (possibly 10x) in my script:

(each migration generates a screenshot from a url)

Before:

info: Migrating 5eVMdE
info: Migrated 5eVMdE in 8.70s
info: Migrating mep2bM
info: Migrated mep2bM in 8.60s
info: Migrating 7ax9by
info: Migrated 7ax9by in 8.47s
info: Migrating YaOpbx
info: Migrated YaOpbx in 7.52s
info: Migrating RdG7dD
info: Migrated RdG7dD in 8.56s
info: Migrating QdJ2dO
info: Migrated QdJ2dO in 8.94s
info: Migrating 1aKReQ

After:

info: Migrating RdG7dD
info: Migrated RdG7dD in 5.94s
info: Migrating QdJ2dO
info: Migrated QdJ2dO in 1.19s
info: Migrating mbk5ez
info: Migrated mbk5ez in 1.21s
info: Migrating 7ax9by
info: Migrated 7ax9by in 1.13s
info: Migrating 1aKReQ
info: Migrated 1aKReQ in 1.19s
info: Migrating wdLwbj
info: Migrated wdLwbj in 1.20s
info: Migrating 1aMAaW

Notice how the first screenshot is slower (it creates the phantomjs process) and others are much faster because they re-use that process.

So I think it's worth considering in the future, for use cases where performance matters.

@olalonde
Copy link
Author

Phantom resource pool: https://gist.github.com/88c0d6cfcd410d655d53eaa787f6b317

I might end up open sourcing this, haha 👍 feel free to re-use

@olalonde
Copy link
Author

Released this under https://github.com/blockai/phantom-pool

@sindresorhus sindresorhus changed the title Process pooling / re-using the same phantomjs process for multiple screenshots? Process pooling / re-using the same Puppeteer instance for multiple screenshots? Feb 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants