pyspark.pandas.window.Expanding.sem#

Expanding.sem(ddof=1)[source]#

Calculate the expanding standard error of the mean.

Added in version 4.3.0.

Note

the current implementation of this API uses Spark’s Window without specifying partition specification. This leads to move all data into single partition in single machine and could cause serious performance degradation. Avoid this method against very large dataset.

Parameters:
ddofint, default 1

Delta Degrees of Freedom. The divisor used in calculations is N - ddof, where N represents the number of elements.

Returns:
Series or DataFrame

Returns the same object type as the caller of the expanding calculation.

See also

pyspark.pandas.Series.expanding

Calling object with Series data.

pyspark.pandas.DataFrame.expanding

Calling object with DataFrames.

pyspark.pandas.Series.sem

Equivalent method for Series.

pyspark.pandas.DataFrame.sem

Equivalent method for DataFrame.

Examples

>>> s = ps.Series([5, 5, 6, 7, 5, 5, 5])
>>> s.expanding(3).sem()
0         NaN
1         NaN
2    0.408248
3    0.552771
4    0.447214
5    0.374166
6    0.321208
dtype: float64

For DataFrame, each expanding standard error of the mean is computed column-wise.

>>> df = ps.DataFrame({"A": s.to_numpy(), "B": s.to_numpy() ** 2})
>>> df.expanding(2).sem()
          A         B
0       NaN       NaN
1  0.000000  0.000000
2  0.408248  4.490731
3  0.552771  6.589132
4  0.447214  5.315073
5  0.374166  4.439970
6  0.321208  3.807887