Indicator Series vs Increments

[DaveSkender]

I’ve noticed that some users are supplying a minimum amount of historical quotes to produce indicators. I suspect they are attempting to calculate a single incremental value, rather than a whole series is historical indicator results. The current intended use case is to produce a whole series in one call.

For many indicator types, like EMA and RSI, this incremental approach can be problematic because they require a warmup period where accuracy improves through numerical convergence. This is why we often recommend 100-250 preceding historical quotes in those cases and not just the bare minimum.

With that said, I’m considering adding parallel implementations of methods, such as GetEmaSeries and GetEmaIncrement in version 2.0, to better address those different use cases. I can see the increment approach as being useful in streaming models where you're adding one period quote at a time to an existing dataset.

Thoughts?

[Fredrik-C]

I have a crazy idea that I've tried to supress from my mind, but it goes like this:

I want to to get a "series" of some sort (RSI, MACD, etc) with a based on a certain resolution (24h, 1h...).
Then I want to add incremets at a lower lower resolution and the library will try to estimate the next "bucket".

Let me explain:

Given I have 300 samples of 24h end-of-day. During the current day I want to "increment" 1h samples to my series and based on the previous day the library will try to predict the current bucket.
estimatedBucket.Volume = increments.Average(x => x.Volume) * 8 //if 1h increments to 1Day resolution
estimatedBucket.Close= increments.Average(x => x.Close) * 8

This way I might be able to get a "peek" of the future at the end of the day if I work with 24/1Day indicators.

Maybe to crazy.. but you have to think out of the box sometimes

[DaveSkender]

I have mixed feelings about forecasting. There are classic methods, like extending your linear regressions or other line fitting techniques; and then there's the whole universe of machine learning, which can be misleading pseudo-science if you're not careful. Check out ML.net.

[DaveSkender]

In thinking about this a bit, I'm thinking the GetIncrement approach will be two parts 1) initialization and then 2) update increment.

using Skender.Stock.Indicators;

[..]  // prerequisite: get baseline quote history from your own source

// initialize the series
IncrementalResults results = Indicator.GetSmaIncrement(history,20);

Then you'd handle event based updates:

[..] // acquire incremental Quote

// update results with new info
results.Update(Quote);

Where IncrementalResults looks something like:

public class IncrementalResults
{
    // indicator results and data needed for next period calcs
    public Guid Id { get; set; }
    public IEnumerable<IResult> IndicatorValues {get; set;}. // e.g. IEnumerable<SmaResult>
    internal IEnumerable<IQuote> History {get; set;}
    internal string IntermediateValuesJson { get; set; }
}

There's still a lot of thinking to do on this one to optimize it for both performance and resource consumption.

[DaveSkender]

var quotes = [..];
MacdBase macdBase = quotes.InitMacd(26,12,9);
IEnumerable<MacdResult> results = macdBase.Results();

// to increment
macdBase.Add(quote);

And use an interface design pattern for the base. Base would need the .Add(quote) part and results would just update automatically without need for new extraction.

[tiger2014]

Do you think the lookback period should be default or set by user, when we use incremental function?
I think default is better, for every indicator's incremental function's look back period might be very diffrent.

[DaveSkender]

You’d provide parameters when you initialize the streaming indicator, as usual, and it will remember it when you add incremental updates. Many indicators do have default values, so you could use those if desired by not specifying parameters.

[tiger2014]

Thanks

[DaveSkender]

I’m working up a preview in #824 to test this out, and will release it for comment

[DaveSkender]

Follow or continue this conversation in our discussion on streaming indicators for progress and information about pre-release package versions to support live and incremental quote sources.