statstream.exact.streaming_mean_and_var¶
- statstream.exact.streaming_mean_and_var(X, steps=None)¶
Mean and variance of a streaming dataset.
Computes the mean and variance of a dataset from a stream of batches of samples. The data has to be provided by an iterator yielding batches of samples. Either a number of steps can be specified, or the iterator is assumed to be emptied in a finite number of steps. In the first case only the given number of batches is extracted from the iterator and used for the mean and variance calculation, even if the iterator could yield more data.
Samples are given along the first axis. The mean and variance have the same shape as the remaining axes, e.g. batches of shape
[batch_size, d1, ..., dN]
will produce a mean and variance of shape[d1, ..., dN]
.This function consumes an iterator, thus finite iterators will be empty after a call to this function, unless
steps
is set to a smaller number than batches in the iterator.- Parameters:
- Returns:
See also
streaming_mean
get only the mean in a single pass.
streaming_var
get only the variance in a single pass.
Notes
The streamed variances are calculated as described in [1].
References
[1]Tony F. Chan & Gene H. Golub & Randall J. LeVeque, “Updating formulae and a pairwise algorithm for computing sample variances”, 1979.