Net.ReadTimeout is not a network-level property

[leonidboykov]

Description

The description of the Net struct says that inner properties correspond to network-level configuration. However, Net.ReadTimeout is affected by some other timeouts, i.e. if Consumer.Group.Rebalance.Timeout is greater than Net.ReadTimeout the rebalance fails with i/o timeout error. Perhaps, Kafka does not reply immediately to the client.

At the moment, the only way to make other timeouts work is to set Net.ReadTimeout as the longest timeout. However, this may lead to problems with actual network issues because there is no actual network-level timeout.

Versions

Sarama	Kafka	Go
v1.41.3	3.1.0	1.21

Configuration

conf := sarama.NewConfig()
conf.Version = sarama.V3_3_1_0
conf.Consumer.Return.Errors = true
conf.Consumer.Offsets.Initial = sarama.OffsetOldest
conf.Consumer.MaxProcessingTime = 1 * time.Minute
conf.Consumer.Group.Rebalance.Timeout = 10 * time.Minute
conf.Consumer.Group.Session.Timeout = 10 * time.Minute
conf.Consumer.Group.Heartbeat.Interval = 5 * time.Second
conf.Net.ReadTimeout = 11 * time.Minute // Consumer.Group.Rebalance.Timeout with an extra 1 minute.
}

Additional Context

I've done some research, and it seems, other Sarama users are dealing with the same problem: #1422 (comment).

I'm not sure, but perhaps, this parameter wasn't tested properly. While default settings for sarama.Config set Consumer.Group.Rebalance.Timeout (60s) greater than Net.ReadTimeout (30s), the value Consumer.Group.Rebalance.Timeout is set to only 10 seconds in the corresponding test:

sarama/functional_consumer_group_test.go

Lines 396 to 405 in 3e385a6

	func defaultConfig(clientID string) *Config {
	config := NewFunctionalTestConfig()
	config.ClientID = clientID
	config.Consumer.Return.Errors = true
	config.Consumer.Offsets.Initial = OffsetOldest
	config.Consumer.Group.Rebalance.Timeout = 10 * time.Second
	config.Metadata.Full = false
	config.Metadata.RefreshFrequency = 5 * time.Second
	return config
	}