Skip to content

Commit

Permalink
Update Readme
Browse files Browse the repository at this point in the history
  • Loading branch information
akash-akya committed Dec 15, 2023
1 parent 8d9b443 commit d9aec2a
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 8 deletions.
47 changes: 39 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,45 @@
[![Hex.pm](https://img.shields.io/hexpm/v/exile.svg)](https://hex.pm/packages/exile)
[![docs](https://img.shields.io/badge/docs-hexpm-blue.svg)](https://hexdocs.pm/exile/)

Exile is an alternative to [ports](https://hexdocs.pm/elixir/Port.html) for running external programs. It provides back-pressure, non-blocking io, and tries to fix ports issues.

Exile is built around the idea of having demand-driven, asynchronous
interaction with external process. Think of streaming a video through
`ffmpeg` to serve a web request. Exile internally uses NIF. See
[Rationale](#rationale) for details. It also provides stream
abstraction for interacting with an external program. For example,
getting audio out of a stream is as simple as
Exile is an alternative to
[ports](https://hexdocs.pm/elixir/Port.html) for running external
programs. It let you stream input and output to external program with
back-pressure, non-blocking IO. Also, it fixes other port related
issues such as selectively closing stdin.

### Port IO Issue

With [Port](https://hexdocs.pm/elixir/Port.html) if you run external
program which generates lot of output to stdout. Something like
streaming video using `ffmpeg` to serve a web request. If you try
this with port then quickly going to run out of memory. Because port
IO is not not demand driven. It consumes output from stdout as soon as
it is available and `send` it to process mailbox. And as you know beam
process mailbox is unbounded, it sits there waiting to be `receive`d.

#### Lets take an example.

Memory consumption with Port

```elixir
Port.open({:spawn_executable, "/bin/cat"}, [{:args, ["/dev/random"]}, {:line, 10}, :binary, :use_stdio])
```

![Port memory consumption](./images/port.png)

#### Memory consumption with Exile

```elixir
Exile.stream!(~w(cat /dev/random))
|> Enum.each(fn data ->
IO.puts(IO.iodata_length(data))
end)
```

![Exile memory consumption](./images/exile.png)

Exile achieves this by implementing demand-driven, asynchronous IO mechanism with external process using NIF.
See [Rationale](#rationale) for details. For example, getting audio out of a stream is as simple as

``` elixir
Exile.stream!(~w(ffmpeg -i pipe:0 -f mp3 pipe:1), input: File.stream!("music_video.mkv", [], 65_535))
Expand Down
Binary file added images/exile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/port.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit d9aec2a

Please sign in to comment.