Nodes with no inputs has empty events stream

[haixuanTao]

If a node does not specify any inputs, it will have no event stream. And the event stream is always returning None.

This is however misleading as we might want to check the event stream for other event than INPUT event such as Stopping event.

Quick example

use std::time::Duration;

use dora_node_api::{self, dora_core::config::DataId, DoraNode, Event, IntoArrow};

fn main() -> eyre::Result<()> {
    println!("hello");

    let output = DataId::from("random".to_owned());

    let (mut node, mut events) = DoraNode::init_from_env()?;

    if events.recv_timeout(Duration::from_secs(100)).is_none() {
        eprintln!("Evemt is None");
    }
    Ok(())
}

With an empty input dataflow description, will return immediately instead of timing out on 100s.

[phil-opp]

Our current behavior is that an event stream is closed as soon as all of its inputs are closed. This way, we enable dataflows to finish "naturally" without requiring a manual stop.

The "stop" event is only sent to signal a premature exit caused by a manual stop operation.

When a node has no inputs, it will never receive any input events. So there is no way to prematurely stop the input stream, as it's already finished from the start. In other words, a node with no inputs always finishes "naturally" even if a manual stop command is sent because the manual stop always happens after the last input has already been closed.

---

We can of course discuss whether we want to change this behavior. However, if we always wait for a manual stop command, it's no longer possible to have dataflows that finish "naturally".

Could you give more details on your use case for a node without inputs?

[haixuanTao]

Sure thing. Basically, I have some source nodes that generates output without any input.

I guess, the thing is I wish to make them stop when I click on Ctrl+C but I don't have any easy way to do that at the moment.

We don't have to fix this just now.

[phil-opp]

Some ideas how we could support this use case:

• Add a some sort of dummy input to the source node that never fires. We could e.g. provide a built-in dora/never input for this use case.
• Add a config key to allow nodes to wait for manual stop, e.g. via the YAML file or as argument to init_from_env.
• Add a new EventChannelClosed event that is sent immediately before the stream returns None. This event could contain a manual stop channel so that it is possible to wait for manual stop commands even after the event stream returned None.

[oortlieb]

We're also using a number of input-only nodes, and this issue makes them somewhat difficult to shut down cleanly. Our current workaround is to use a dora/timer with a long timeout and ignoring those messages when they are sent.