diff --git a/docs/wiki/_Sidebar.md b/docs/wiki/_Sidebar.md index 8d1545e8b..2b0488b92 100644 --- a/docs/wiki/_Sidebar.md +++ b/docs/wiki/_Sidebar.md @@ -22,13 +22,13 @@ Notes for editors: - [Historical Buffers](Historical-Buffers) - [Execution Modes](Execution-Modes) - [Adapters](Adapters) +- [Feedback and Delayed Edge](Feedback-and-Delayed-Edge) - [Common Mistakes](Common-Mistakes) **How-to guides** - [Use Statistical Nodes](Use-Statistical-Nodes) - Use Adapters (coming soon) -- [Add Cycles in Graphs](Add-Cycles-in-Graphs) - [Create Dynamic Baskets](Create-Dynamic-Baskets) - Write Adapters: - [Write Historical Input Adapters](Write-Historical-Input-Adapters) diff --git a/docs/wiki/concepts/Feedback-and-Delayed-Edge.md b/docs/wiki/concepts/Feedback-and-Delayed-Edge.md new file mode 100644 index 000000000..62a296004 --- /dev/null +++ b/docs/wiki/concepts/Feedback-and-Delayed-Edge.md @@ -0,0 +1,97 @@ +## Table of Contents + +- [Table of Contents](#table-of-contents) +- [`csp.feedback`](#cspfeedback) +- [`csp.DelayedEdge`](#cspdelayededge) + +CSP has two methods of late binding a time series to an edge. The first, `csp.feedback`, is used to create connections from downstream nodes to upstream nodes in a graph without introducing a cycle. +The second, `csp.DelayedEdge`, is used when a single edge may come from many possible sources and will be bound after its passed as an argument to some other node. + +## `csp.feedback` + +CSP graphs are always *directed acyclic graphs* (DAGs); however, there are some occasions where you may want to feed back the output of a downstream node into a node at a prior [rank](CSP-Graph#Graph-Propagation-and-Single-Dispatch). + +This is usually the case when a node is making a decision that depends on the result of its previous action. +For example, consider a `csp.graph` that simulates population dynamics. We have a system of [wolves and elk](https://www.yellowstonepark.com/things-to-do/wildlife/wolf-reintroduction-changes-ecosystem/) with W wolves and E elk. +We have a node `change_wolf_pop` which simulates introducing or removing wolves from the ecosystem. Then we have a node `compute_elk_pop` which computes the new expected elk population based on the number of wolves *and* other factors. +The output of `compute_elk_pop`, being the number of expected elks, will **feed back** into the `change_wolf_pop` node. This means the population dynamics model can be wholly separate from our intervention decision. + +For use cases like this, the `csp.feedback` construct exists. It allows us to pass the output of a downstream node to one upstream so that a recomputation is triggered on the next [engine cycle](CSP-Graph#Graph-Propagation-and-Single-Dispatch). +Using `csp.feedback`, one can wire a feedback as an input to a node, and effectively bind the actual edge that feeds it later in the graph. + +> \[!IMPORTANT\] +> A graph containing one or more `csp.feedback` edges is still acyclic. The feedback connection will trigger a recomputation of the upstream node on the next engine cycle, which will be at the same engine time as the current cycle. Internally `csp.feedback` creates a pair of input and output adapters that are bound together. + +- **`csp.feedback(ts_type)`**: `ts_type` is the type of the timeseries (ie int, str). + This returns an instance of a feedback object, which will *later* be bound to a downstream edge. + - **`out()`**: this method returns the timeseries edge which can be passed as an input to your node + - **`bind(ts)`**: this method is called to bind a downstream edge as the source of the feedback + +Let us demonstrate the usage of `csp.feedback` using our wolves and elk example above. The graph code would look something like this: + +```python +import csp +from csp import ts + +@csp.node +def change_wolf_pop(elks: ts[int], wolf_only_factors: ts[float]) -> ts[int]: + # External factors could be anything here that affects the wolf population + # but not the elk population directly + + # compute the desired wolf population here... + + return 0 + +@csp.node +def compute_elk_pop(wolves: ts[int], elk_only_factors: ts[float]) -> ts[int]: + # Similarly external factors here only directly affect the elk population + + # compute the new expected elk population here... + + return 0 + +@csp.graph +def population_dynamics(): + # create the feedback first so that we can refer to it later + elk_pop_fb = csp.feedback(int) + + # update the wolf population, passing feedback out() which isn't bound yet + wolves = change_wolf_pop(elk_pop_fb.out(), csp.const(0.0)) + + # get elks output from compute_elk_pop + elks = compute_elk_pop(wolves, csp.const(0.0)) + + # now bind the elk population to the feedback, finishing the "loop" + elk_pop_fb.bind(elks) +``` + +We can visualize the graph using `csp.show_graph`. We see that it remains acyclic, but since the `FeedbackOutputDef` is bound to the `FeedbackInputDef` any output tick will loop back in at the next engine cycle. + +![Output generated by show_graph](images/feedback-graph.png) + +## `csp.DelayedEdge` + +The delayed edge is similar to `csp.feedback` in the sense that it's a time series which is bound after its declared. Delayed edges must be bound *exactly* once and will raise an error during graph building if unbound. +Delayed edges can also not be used to create a cycle; if the edge is being bound to a downstream output, `csp.feedback` must be used instead. Any cycle will be detected by the CSP engine and raise a runtime error. + +Delayed edges are useful when the exact input source needed is not known until graph-time; for example, you may want to subscribe to a list of data feeds which will only be known when you construct the graph. +They are also used by some advanced `csp.baselib` utilities like `DelayedCollect` and `DelayedDemultiplex` which help with input and output data processing. + +An example usage of `csp.DelayedEdge` is below: + +```python +import csp + +@csp.graph +def delayed_edge(): + delayed = csp.DelayedEdge(csp.ts[int]) + three = csp.const(2) + delayed + delayed.bind(csp.const(1)) + csp.print('three', three) +``` + +Executing this graph will give: + +``` +2020-01-01 00:00:00 three:3 +``` diff --git a/docs/wiki/how-tos/Add-Cycles-in-Graphs.md b/docs/wiki/how-tos/Add-Cycles-in-Graphs.md deleted file mode 100644 index d8fba4312..000000000 --- a/docs/wiki/how-tos/Add-Cycles-in-Graphs.md +++ /dev/null @@ -1,52 +0,0 @@ -By definition of the graph building code, CSP graphs can only produce acyclical graphs. -However, there are many occasions where a cycle may be required. -For example, lets say you want part of your graph to simulate an exchange. -That part of the graph would need to accept new orders and return acks and executions. -However, the acks / executions would likely need to *feedback* into the same part of the graph that generated the orders. -For this reason, the `csp.feedback` construct exists. -Using `csp.feedback` one can wire a feedback as an input to a node, and effectively bind the actual edge that feeds it later in the graph. -Note that internally the graph is still acyclical. -Internally `csp.feedback` creates a pair of output and input adapters that are bound together. -When a timeseries that is bound to a feedback ticks, it is fed to the feedback which then schedules the tick on its bound input to be executed on the **next engine cycle**. -The next engine cycle will execute with the same engine time as the cycle that generated it, but it will be evaluated in a subsequent cycle. - -- **`csp.feedback(ts_type)`**: `ts_type` is the type of the timeseries (ie int, str). - This returns an instance of a feedback object - - **`out()`**: this method returns the timeseries edge which can be passed as an input to your node - - **`bind(ts)`**: this method is called to bind an edge as the source of the feedback after the fact - -A simple example should help demonstrate a possible usage. -Lets say we want to simulate acking orders that are generated from a node called `my_algo`. -In addition to generating the orders, `my_algo` also wants needs to receive the execution reports (this is demonstrated in example `e_13_feedback.py`) - -The graph code would look something like this: - -```python -# Simulate acking an order -@csp.node -def my_exchange(order:ts[Order]) -> ts[ExecReport]: - # ... impl details ... - -@csp.node -def my_algo(exec_report:ts[ExecReport]) -> ts[Order]: - # .. impl details ... - -@csp.graph -def my_graph(): - # create the feedback first so that we can refer to it later - exec_report_fb = csp.feedback(ExecReport) - - # generate orders, passing feedback out() which isn't bound yet - orders = my_algo(exec_report_fb.out()) - - # get exec_reports from "simulator" - exec_report = my_exchange(orders) - - # now bind the exec reports to the feedback, finishing the "loop" - exec_report_fb.bind(exec_report) -``` - -The graph would end up looking like this. -It remains acyclical, but the `FeedbackOutputDef` is bound to the `FeedbackInputDef` here, any tick to out will push the tick to in on the next cycle: - -![366521848](https://github.com/Point72/csp/assets/3105306/c4f920ff-49f9-4a52-8404-7c1989768da7) diff --git a/docs/wiki/images/feedback-graph.png b/docs/wiki/images/feedback-graph.png new file mode 100644 index 000000000..31b21af4d Binary files /dev/null and b/docs/wiki/images/feedback-graph.png differ