Making cross-sectional statistics and scalar outputs

[poldpold]

Thank you for making csp, it's great! I have been reading through the examples for a few days now, and trying to do a proof of concept. It consists of generating 10 standard Brownian motions that tick each second. Then at each time step I would like to calculate the difference between the maximum and the minimum across the 10 current values, and finally calculate the maximum of this max-min difference across the time series, and the single final max-min value.

In pandas I would do something like this:

import numpy as np
import pandas as pd
np.random.seed(42)
number_of_products = 10

index = pd.date_range('2020-01-01', '2020-01-02', freq='min')
prices = pd.DataFrame(data=np.random.normal(size=(len(index), number_of_products)), index=index).cumsum()
print((prices.max(axis=1)-prices.min(axis=1)).max(), (prices.max(axis=1)-prices.min(axis=1)).iloc[-1])

I have made some progress with csp and arrived to this point:

import numpy as np
import csp
from datetime import timedelta, datetime
from csp.random import brownian_motion

number_of_products = 10

@csp.graph
def graph():
    trigger = csp.timer(timedelta(minutes=1), 1)
    prices = [csp.random.brownian_motion_1d(trigger=trigger, drift=csp.const(0.0), variance=csp.const(1.0/60**.5), seed=p) for p in range(number_of_products)]
    prices = csp.stats.list_to_numpy(prices)
    csp.add_graph_output("prices", prices)
    min_max_np = np.max(prices) - np.min(prices)
    csp.add_graph_output("min_max_np", min_max_np)

results = csp.run(graph, starttime=datetime(2020, 1, 1), endtime=datetime(2020, 1, 2))

I am stuck with two problems. First, I must be using the csp-numpy glue wrong, because the above raises an error. What would be the correct syntax for my cross-sectional operation, which would calculate minimum and maximum across my list of 10 time series? Second, assuming I manage to assign max minus min to a new time series, how do I only retain two single scalar statistics, the max of max minus min and the final value of max minus min? I would like to avoid keeping the whole series in memory, if all I need are simple statistics that can be generated while doing the forward pass.

[AdamGlustein]

Here is the corrected version of the code with comments as to the changes.

import numpy as np
import csp
from datetime import timedelta, datetime
from csp.random import brownian_motion

number_of_products = 10

@csp.graph
def graph():
    trigger = csp.timer(timedelta(minutes=1), 1)
    prices = [csp.random.brownian_motion_1d(trigger=trigger, drift=csp.const(0.0), variance=csp.const(1.0/60**.5), seed=p) for p in range(number_of_products)]
    prices = csp.stats.list_to_numpy(prices)
    
    # Use csp.apply to "apply" the np.max/np.min functions to each individual value in "prices"
    # Note that np.min and np.max cannot be called directly since 'prices' is an Edge here
    # We want to apply the min/max functions to the values (at runtime), not the graph-time Edge
    max_minus_min = csp.apply(prices, np.max, float) - csp.apply(prices, np.min, float)

    # We can use stats.max to compute the current max value of the ts
    max_of_max_minus_min = csp.stats.max(max_minus_min)

    # We can use tick_count=1 to avoid storing a full history
    csp.add_graph_output("final_value_of_max_minus_min", max_minus_min, tick_count=1)
    csp.add_graph_output("max_of_max_minus_min", max_of_max_minus_min, tick_count=1)

results = csp.run(graph, starttime=datetime(2020, 1, 1), endtime=datetime(2020, 1, 2))
print(results)

[poldpold]

Thank you so much @AdamGlustein! I can’t wait to build something more general on this logic.