You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
is it possible to share the libpostal memory between two different nodejs processes calling require('node-postal') on the same machine?
calling require('node-postal') uses a fair chunk of RAM so having two different services on the same machine using node-postal consumes more RAM than maybe it needs to?
I'm assuming that the memory is read-only so would it be possible to share that memory between processes?
The text was updated successfully, but these errors were encountered:
Not without re-architecting a few parts of libpostal. require('node-postal') reads the model files from disk and creates the necessary in-memory data structures for libpostal using malloc. To share memory between processes in UNIX you're almost always talking about mmap. Parts of the said model files like the arrays/matrices should be mmap-able, but because of the way we store numeric types on disk (endian-agnostic, floats/doubles stored as their uint32/uint64 representations and converted on read), and the fact that mmap would only provide access to the raw bytes, it would require quite a bit of refactoring apart from just the I/O functions.
In any case, reducing memory should be relatively easy to accomplish at the application level by designating one process per machine to load libpostal and communicate with other processes through a named pipe, a TCP socket, something like ZeroMQ, Redis, etc.
is it possible to share the libpostal memory between two different nodejs processes calling
require('node-postal')
on the same machine?calling
require('node-postal')
uses a fair chunk of RAM so having two different services on the same machine using node-postal consumes more RAM than maybe it needs to?I'm assuming that the memory is read-only so would it be possible to share that memory between processes?
The text was updated successfully, but these errors were encountered: