Producer shutdown configuration

[petersonfs]

We are working on a project where closing the producer connection after 5s is causing some problems. Is there any reason for producer shutdown connection after 5s?

producer_spec(ClientPid, Topic, Partition, Config0) ->
  {Config, DelaySecs} =
    take_delay_secs(Config0, partition_restart_delay_seconds,
                    ?DEFAULT_PRODUCER_RESTART_DELAY),
  Args      = [ClientPid, Topic, Partition, Config],
  { _Id       = Partition
  , _Start    = {brod_producer, start_link, Args}
  , _Restart  = {permanent, DelaySecs}
  , _Shutdown = 5000
  , _Type     = worker
  , _Module   = [brod_producer]
  }.

[zmstone]

it's to force kill if sys:terminate failed to stop the process in 5s.
this is an attempt to gracefully stop producer process, but no good reason for 5s.
what's the issue ?

[petersonfs]

In our use case (instant payment) we need to settle the transactions in few seconds. But when we stop to receive traffic for minutes for example the producer gets down, and it takes around 2 seconds to warm up each of them because we are using the auto-start feature, and we have several topics to complete the process this 5s killing us.

As a workaround, we create a heartbeat that publish messages in each topic each 2s. But we are wondering if a PR will be welcome with producer shutdown parameter instead 5s hard-coded?

[zmstone]

But when we stop to receive traffic for minutes for example the producer gets down

Why does the producer go down ?

As a workaround, we create a heartbeat that publish messages in each topic each 2s.

It sound like what you need is TCP keep-alive.

Try provide connection config with options like this:

{extra_sock_opts, [
  {keepalive, true},
  {raw, 6, 4, <<Idle:32/native>>},
  {raw, 6, 5, <<Interval:32/native>>},
  {raw, 6, 6, <<Probes:32/native>>}
]}