Process pooling / re-using the same Puppeteer instance for multiple screenshots?

[olalonde]

Is there any way to re-use the same phantomjs instance for multiple screenshots if the list of URLs aren't known in advance?

[SamVerschueren]

Duplicate of #242 and #282.

[olalonde]

It's not really, unless the answer is that "no" this won't be implemented. What I'm looking for is to have a "single" phantomjs across all screenshots and this is not possible at the moment.

[SamVerschueren]

It was requested here though #282 (comment).

As @kevva points out, we don't think it would be possible to change the URL while keeping the process alive.

[olalonde]

I've actually just started working on that using node-phantomjs: https://gist.github.com/a1f2d6f56db7d8bbf0f8616ab5018346 (WIP!)

It re-uses the same phatomjs instance (process) for all screenshots within the same Node.js process. It would require a pretty big refactoring of pageres as far as I can tell, so I wrote it from scratch. I'd be happy to contribute to pageres but the way it's designed doesn't make it easy.

[SamVerschueren]

I don't think it's worth the effort right now. Once headless chrome lands (not sure when it will happen though), we will switch to that one. So then we could take a look at how it could be done in headless chrome.

Unless @kevva or @sindresorhus have another opinion on this one.

[olalonde]

Ok, sure. FWIW, after switching from pageres to the code I just pasted, I got a huge performance boost (possibly 10x) in my script:

(each migration generates a screenshot from a url)

Before:

info: Migrating 5eVMdE
info: Migrated 5eVMdE in 8.70s
info: Migrating mep2bM
info: Migrated mep2bM in 8.60s
info: Migrating 7ax9by
info: Migrated 7ax9by in 8.47s
info: Migrating YaOpbx
info: Migrated YaOpbx in 7.52s
info: Migrating RdG7dD
info: Migrated RdG7dD in 8.56s
info: Migrating QdJ2dO
info: Migrated QdJ2dO in 8.94s
info: Migrating 1aKReQ

After:

info: Migrating RdG7dD
info: Migrated RdG7dD in 5.94s
info: Migrating QdJ2dO
info: Migrated QdJ2dO in 1.19s
info: Migrating mbk5ez
info: Migrated mbk5ez in 1.21s
info: Migrating 7ax9by
info: Migrated 7ax9by in 1.13s
info: Migrating 1aKReQ
info: Migrated 1aKReQ in 1.19s
info: Migrating wdLwbj
info: Migrated wdLwbj in 1.20s
info: Migrating 1aMAaW

Notice how the first screenshot is slower (it creates the phantomjs process) and others are much faster because they re-use that process.

So I think it's worth considering in the future, for use cases where performance matters.

[olalonde]

Phantom resource pool: https://gist.github.com/88c0d6cfcd410d655d53eaa787f6b317

I might end up open sourcing this, haha 👍 feel free to re-use

[olalonde]

Released this under https://github.com/blockai/phantom-pool