Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stats/incr): add incremental weighted standard deviation calculation #3088

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 145 additions & 0 deletions lib/node_modules/@stdlib/stats/incr/wstdev/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
<!--
@license Apache-2.0

Copyright (c) 2024 The Stdlib Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# incrwstdev

> Compute an incremental [weighted standard deviation][weighted-standard-deviation].

## Introduction

The [weighted standard deviation][weighted-standard-deviation] is defined as:

```math
\sqrt{ \frac{\displaystyle\sum_{i=0}^{n-1} w_{i} \left( x_i - \bar{x} \right)^2 }{ \displaystyle\sum_{i=0}^{n-1} w_{i} } }
```

This function provides an incremental approach to calculating the weighted standard deviation. It computes the value iteratively, updating the result with each new data point and weight, which is useful for scenarios where data is streamed or updated over time.

<!-- /.intro -->
<section class="usage>

## Usage

```javascript
var incrwstdev = require( '@stdlib/stats/incr/wstdev' );
```

### incrwstdev()

Returns an accumulator `function` that incrementally computes a [weighted standard deviation][weighted-standard-deviation].

```javascript
var accumulator = incrwstdev();
```

#### accumulator( [x,w] )

If provided an input value `x` and a weight `w`, the accumulator function returns an updated weighted standard deviation. If not provided any input values, the accumulator function returns the current weighted standard deviation.

```javascript
var accumulator = incrwstdev();

var s = accumulator( 2.0, 1.0 );
// returns 0.0

s = accumulator( 1.0, 0.5 );
// returns <updated weighted standard deviation>

s = accumulator();
// returns the current weighted standard deviation
```
</section>

<!-- /.usage -->

<section class="notes">

## Notes

- Input values are **not** type checked. If provided `NaN` or a value that results in `NaN` in computations, the accumulated value will be `NaN` for **all** future invocations. If non-numeric inputs are possible, you should type-check and handle accordingly **before** passing the value to the accumulator function.

</section>


<!-- /.notes -->

<section class="examples">

## Examples

<!-- eslint no-undef: "error" -->

```javascript
var randu = require( '@stdlib/random/base/randu' );
var incrwstdev = require( '@stdlib/stats/incr/wstdev' );

var accumulator;
var s;
var i;

// Initialize an accumulator:
accumulator = incrwstdev();

// For each simulated datum, update the weighted standard deviation...
for ( i = 0; i < 100; i++ ) {
var value = randu() * 100.0;
var weight = randu();
accumulator( value, weight );
}
console.log( accumulator() );
```

</section>

<!-- /.examples -->

<!-- Section for related `stdlib` packages. Do not manually edit this section, as it is automatically populated. -->

<section class="related">

* * *

## See Also

- <span class="package-name">[`@stdlib/stats/incr/ewstdev`][@stdlib/stats/incr/ewstdev]</span><span class="delimiter">: </span><span class="description">compute an exponentially weighted standard deviation incrementally.</span>
- <span class="package-name">[`@stdlib/stats/incr/stdev`][@stdlib/stats/incr/stdev]</span><span class="delimiter">: </span><span class="description">compute an unweighted standard deviation incrementally.</span>
- <span class="package-name">[`@stdlib/stats/incr/mstdev`][@stdlib/stats/incr/mstdev]</span><span class="delimiter">: </span><span class="description">compute a moving standard deviation incrementally.</span>

</section>

<!-- /.related -->

<!-- Section for all links. Make sure to keep an empty line after the `section` element and another before the `/section` close. -->


<section class="links">

[weighted-standard-deviation]:https://www.itl.nist.gov/div898/software/dataplot/refman2/ch2/weightsd.pdf

<!-- <related-links> -->

[@stdlib/stats/incr/mstdev]:https://github.com/stdlib-js/stdlib/tree/develop/lib/node_modules/%40stdlib/stats/incr/mstdev

[@stdlib/stats/incr/ewstdev]:https://github.com/stdlib-js/stdlib/tree/develop/lib/node_modules/%40stdlib/stats/incr/ewstdev

[@stdlib/stats/incr/mmeanstdev]:https://github.com/stdlib-js/stdlib/tree/develop/lib/node_modules/%40stdlib/stats/incr/mmeanstdev

<!-- </related-links> -->

</section>
<!-- /.links> -->
92 changes: 92 additions & 0 deletions lib/node_modules/@stdlib/stats/incr/wstdev/benchmark/benchmark.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/**
* @license Apache-2.0
*
* Copyright (c) 2018 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

'use strict';

// MODULES //

var bench = require( '@stdlib/bench' );
var randu = require( '@stdlib/random/base/randu' );
var pkg = require( './../package.json' ).name;
var incrwstdev = require( './../lib' );


// MAIN //

bench( pkg, function benchmark( b ) {
var f;
var i;
b.tic();
for ( i = 0; i < b.iterations; i++ ) {
f = incrwstdev();
if ( typeof f !== 'function' ) {
b.fail( 'should return a function' );
}
}
b.toc();
if ( typeof f !== 'function' ) {
b.fail( 'should return a function' );
}
b.pass( 'benchmark finished' );
b.end();
});


bench( pkg+'::accumulator', function benchmark( b ) {
var acc;
var v;
var i;

acc = incrwstdev();

b.tic();
for ( i = 0; i < b.iterations; i++ ) {
v = acc( randu(), randu() ); // passing random value and weight
if ( v !== v ) {
b.fail( 'should not return NaN' );
}
}
b.toc();
if ( v !== v ) {
b.fail( 'should not return NaN' );
}
b.pass( 'benchmark finished' );
b.end();
});

bench( pkg+'::accumulator,known_mean', function benchmark( b ) {
var acc;
var v;
var i;

acc = incrwstdev( 3.14 ); // using a known mean

b.tic();
for ( i = 0; i < b.iterations; i++ ) {
v = acc( randu(), randu() ); // passing random value and weight
if ( v !== v ) {
b.fail( 'should not return NaN' );
}
}
b.toc();
if ( v !== v ) {
b.fail( 'should not return NaN' );
}
b.pass( 'benchmark finished' );
b.end();
});
31 changes: 31 additions & 0 deletions lib/node_modules/@stdlib/stats/incr/wstdev/docs/repl.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{{alias}}( [mean] )
Returns an accumulator function which incrementally computes a weighted standard deviation.

If provided a value and its associated weight, the accumulator function updates and returns the current weighted standard deviation. If not provided with arguments, the accumulator function returns the current weighted standard deviation.

If provided `NaN` or if a calculation results in `NaN`, the accumulated value will be `NaN` for all future invocations.

Parameters
----------
mean: number (optional)
Known mean, if already computed.

Returns
-------
acc: Function
Accumulator function.

Examples
--------
> var accumulator = {{alias}}();
> var s = accumulator()
null
> s = accumulator( 2.0, 1.5 )
0.0
> s = accumulator( -5.0, 2.0 )
updated weighted standard deviation
> s = accumulator()
current weighted standard deviation

See Also
--------
62 changes: 62 additions & 0 deletions lib/node_modules/@stdlib/stats/incr/wstdev/docs/types/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/*
* @license Apache-2.0
*
* Copyright (c) 2019 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

// TypeScript Version: 4.1

/// <reference types="@stdlib/types"/>

/**
* If provided a value and a weight, returns an updated weighted standard deviation; otherwise, returns the current weighted standard deviation.
*
* ## Notes
*
* - If provided `NaN` or a value which, when used in computations, results in `NaN`, the accumulated value is `NaN` for all future invocations.
*
* @param x - value
* @param weight - weight associated with the value
* @returns weighted standard deviation
*/
type accumulator = ( x?: number, weight?: number ) => number | null;

/**
* Returns an accumulator function which incrementally computes a weighted standard deviation.
*
* @param mu - known mean
* @returns accumulator function
*
* @example
* var accumulator = incrwstdev();
*
* var s = accumulator();
* // returns null
*
* s = accumulator( 2.0, 1.5 );
* // returns 0.0
*
* s = accumulator( -5.0, 2.0 );
* // returns updated weighted standard deviation
*
* s = accumulator();
* // returns current weighted standard deviation
*/
declare function incrwstdev( mu?: number ): accumulator;


// EXPORTS //

export = incrwstdev;
62 changes: 62 additions & 0 deletions lib/node_modules/@stdlib/stats/incr/wstdev/docs/types/test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
/*
* @license Apache-2.0
*
* Copyright (c) 2019 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import incrwstdev = require( './index' );


// TESTS //

// The function returns an accumulator function...
{
incrwstdev(); // $ExpectType accumulator
incrwstdev( 0.0 ); // $ExpectType accumulator
}

// The compiler throws an error if the function is provided invalid arguments...
{
incrwstdev( '5' ); // $ExpectError
incrwstdev( true ); // $ExpectError
incrwstdev( false ); // $ExpectError
incrwstdev( null ); // $ExpectError
incrwstdev( [] ); // $ExpectError
incrwstdev( {} ); // $ExpectError
incrwstdev( ( x: number ): number => x ); // $ExpectError
}

// The function returns an accumulator function which returns an accumulated result...
{
const acc = incrwstdev();

acc(); // $ExpectType number | null
acc( 3.14, 1.5 ); // $ExpectType number | null
}

// The compiler throws an error if the returned accumulator function is provided invalid arguments...
{
const acc = incrwstdev();

acc( '5', 1.5 ); // $ExpectError
acc( 3.14, '1.5' ); // $ExpectError
acc( true, 1.5 ); // $ExpectError
acc( 3.14, false ); // $ExpectError
acc( null, 1.5 ); // $ExpectError
acc( 3.14, null ); // $ExpectError
acc( [], 1.5 ); // $ExpectError
acc( 3.14, {} ); // $ExpectError
acc( ( x: number ): number => x, 1.5 ); // $ExpectError
}
Loading
Loading