Vault::ConnectionPool::PoolShuttingDownError

[PTcrusher]

Hi,

We are noticing a lot of Vault::ConnectionPool::PoolShuttingDownError lately.
After looking at the stack traces we saw the exception is raised in the pop method whenever the shutdown_block is set (which by looking at the code would only occur if we are in a middle of a client.shutdown call).

@mutex.synchronize do
  loop do
    raise ConnectionPool::PoolShuttingDownError if @shutdown_block
    (...)
    @resource.wait(@mutex, to_wait)
  end
end

Given so, could this mean that the resource is release before the connections are properly closed (waking up all treads waiting for the resource)?

  def shutdown(&block)
    raise ArgumentError, "shutdown must receive a block" unless block_given?
    @mutex.synchronize do
      @shutdown_block = block
      @resource.broadcast
      shutdown_connections
    end
  end

Shouldn't we make sure that all the connections are shutdown through the shutdown_connections method before the @resource.broadcast can actually be called?

Could the order of this two lines be the reason for the Vault::ConnectionPool::PoolShuttingDownError errors?

STACK_TRACE.log
CALLER_JOIN.log