Why is ISchedulerLongRunning used by default in Delay() and ObserveOn()

[rose-a]

Hi,

I've tripped over the so called "optimization" to schedule tasks as LongRunning by default multiple times now, especially since there is no documentation regarding this aspect of scheduling anywhere (or at least its not easy to find).

My most recent incident was a server application getting really slow after running for a week or two to the point where it would become unusable, which after taking a dotTrace snapshot shows it has apparently started lots of threads through the use of Observable.Delay().

I've already created a ShortRunningTaskPoolScheduler helper class which returns a TaskPoolScheduler.Default.DisableOptimizations(typeof(ISchedulerLongRunning))), which migitates the issue when I stuff it in every call to ObserveOn() or Delay().

My intention behind using ObserveOn() can basically be described as "Get the subscribed work off the calling thread and just queue it into the task pool", rather than "The subscribed method runs veery long and therefore should use its own 'private' thread".

I find it rather unintuitive that ObserveOn() and Delay() don't actually use the threads in the ThreadPool, especially if I explicitely use a TaskPoolScheduler.Default.

Could anyone please explain the reasoning behind this behaviour?

Maybe I'm getting something totally wrong, but if not, please document this behavior i.e. on IntroToRx (which is an awesome resource by the way) so that people don't have to find out the hard way!

[idg10]

I don't know why this decision was originally made (having only taken control of the project fairly recently), so unless @bartdesmet happens to look in, we might not be able to get a definitive answer.

However, I would guess it's because Delay is inherently a high-overhead mechanism that is attempting to do something really quite difficult. It's attempting to replicate the exact timing of its source, but time shifted. This gets harder to do properly the fast the underlying source runs, and eventually, the precision with which it works will be limited by the rate at which it can process items. Therefore, an implementation of Delay that can minimize the per-item overhead will be able to cope with a higher rate of items before becoming overwhelmed than one that does more work.

Going via an IScheduler that dispatches every single item individually through the ThreadPool is going to have much, much higher overheads than a loop (or recursive equivalent) that doesn't go through any Rx or OS scheduling for each item. To avoid OS scheduling, you need a dedicated thread. So the bottom line is that the long running approach is going to be able to go a lot faster before becoming overwhelmed than the implementation you were expecting.

I try to avoid Delay whenever possible because what it does is inherently expensive.

What are you doing that requires Delay?

As for ObserveOn, I thought that although it used long-running scheduling, it shuts down once it has drained the queue. Are you seeing high thread usage due to ObserveOn? Or just Delay?

[neuecc]

Task.Delay uses the ThreadPool, so saying that Delay should be avoided is not really an essential answer.
This, like System.Threading.Timer and similar mechanisms, internally uses TimerQueueTimer (which is ultimately TimeProvider.System's Timer)
and is optimized for efficient execution on the dotnet/runtime side.
While we should consider that Rx's implementation predates these .NET improvements,
it's undoubtedly true that a per-Thread implementation is quite problematic.

[rose-a]

@idg10 Thanks for picking up on this.

Are you seeing high thread usage due to ObserveOn? Or just Delay?

Happens with both methods, the screenshot only shows threads left over by Delay because the ObserveOn calls have been caught and migitated in a previous iteration. (It's a ASP.NET Core application running a kestrel web service on Windows 10 LTSC, if that makes any difference)

The usage of Delay() is propably questionable and can be described as "code smell" in most cases, and often there surely is a better solution to the problem that it's trying to solve, but that's not the point I'm trying to make.

As @neuecc pointed out, Task.Delay uses the task pool, so intuitively that's what I expect Observable.Delay to do (and using it in production code is just as questionable as using Observable.Delay).

If I put ObserveOn(TaskPoolScheduler.Default) in my code, I expect the result to be that the observable queues each work item onto the task pool (because "TaskPoolScheduler"). If I use the NewThreadScheduler, I expect it to start a new thread for each work item.

The ISchedulerLongRunning optimization totally breaks this intuitively expected behavior. This comes around as a total surprise if you didn't work yourself all the way through the source code of this library (which, because of lots of abstractions, is not easy for an outsider).

Since it's not documented anywhere, this is a trap that many people might run into unknowingly, and it can result in weird application behavior which might vary on different platforms and is hard to debug (at least that happened to me).

I suggest the following measures:

1. Document the default scheduler each observable method uses and if it makes use of the ISchedulerLongRunning optimization
2. Challenge the decision to implement the ISchedulerLongRunning optimization, and propably make it "opt-in" instead of "opt-out"