visualdynamics.core.averaging¶
averaging
¶
How a time history is cut into frames before it is averaged.
A spectral average is set by a few independent choices: where to start, how long each frame is, how far the frames overlap, which window shapes them (with its parameter, for the windows that take one), whether each frame is detrended, and how many frames there are. Everything else, such as the hop between frames, where the analysis ends and how long it runs, follows from those and is derived here instead of stored, so nothing can disagree.
The frame count is stored, not the stop time. Dragging the end of the analysis in the GUI adds or removes whole frames, because half a frame is not an average.
What an average is worth follows from the same settings. An
autospectrum averaged over n independent frames is a chi-square
variable with 2n statistical degrees of freedom (statistical_dof):
two per frame, the real and the imaginary part of each line. Frames
that overlap are not independent, so the equivalent count is lower, by
how much the window lets two overlapped frames share
(frame_correlation). From the equivalent count follows the
confidence band on the estimate (chi_square_factors,
confidence_bands): the range the true spectrum lies in, at a stated
confidence, as a factor below and a factor above the averaged value.
The window also spreads each line's power into its neighbors, so the
estimates at adjacent lines vary together (line_correlation), and a
sum of adjacent lines is worth less than their counts added.
These degrees of freedom are statistical. They have nothing to do with
the measurement DOFs (node and direction) a record is taken at.
References
These frames feed Welch's averaged periodogram, and the window choices are Harris's.
- Welch, P. D. (1967). "The Use of Fast Fourier Transform for the Estimation of Power Spectra: A Method Based on Time Averaging Over Short, Modified Periodograms." IEEE Transactions on Audio and Electroacoustics, 15(2), 70-73. doi:10.1109/TAU.1967.1161901
- Harris, F. J. (1978). "On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform." Proceedings of the IEEE, 66(1), 51-83. doi:10.1109/PROC.1978.10837 Window shapes, equivalent noise bandwidths and overlap correlation.
- Bendat, J. S., & Piersol, A. G. (2010). Random Data: Analysis and Measurement Procedures, 4th ed. Wiley. The random error of an averaged estimate against the number of frames, which is what the frame count sets, and the chi-square confidence interval on an autospectral density estimate.
Classes:
| Name | Description |
|---|---|
Averaging |
The independent settings of an average; everything else is |
StatisticalDof |
How many statistical degrees of freedom an averaged autospectrum |
ConfidenceBand |
Where the true spectrum lies about an averaged estimate, at one |
Functions:
| Name | Description |
|---|---|
normalize_detrend |
The key for a detrend named any of the ways it might be. |
normalize_window |
The key for a window named any of the ways a file might name it. |
from_span |
The averaging a dragged span means, in whole frames. |
window_shape |
The window |
frame_correlation |
How much two frames |
statistical_dof |
The statistical degrees of freedom of an autospectrum averaged over |
line_correlation |
How much the averaged power at two lines varies together, at each |
chi_square_factors |
(lower, upper): the factors the true spectrum lies between, as |
db_text |
One edge of a band in decibels, signed, to a tenth of a decibel: |
dof_text |
A count of statistical degrees of freedom as it is shown: four |
confidence_bands |
The chi-square confidence band on an averaged autospectrum, one |
Classes¶
Averaging
dataclass
¶
Averaging(frame_length: int, overlap: float = 0.0, window: str = 'hann', frames: int = 1, start: float = 0.0, detrend: str = 'none', window_parameter: float | None = None, start_set: bool = False)
The independent settings of an average; everything else is derived from them.
frame_length is in samples, at least two. overlap is a fraction
of a frame, from 0 up to but not including 1, so 0.5 is the usual
half overlap. window is any name normalize_window accepts. Each
frame is leveled by detrend ('none', 'mean' or 'linear') before
it is windowed. frames is how many frames are averaged per record,
at least one. start is in seconds from the beginning of the
record. window_parameter is the Tukey or Kaiser window's number,
filled in with its default when None and refused for a window that
takes none. start_set says the start was chosen (by the detector,
a drag or a typed number) instead of left at the record's
beginning; it is not part of equality.
A history that already holds one frame per record, such as a
burst-random run saved as 20 captures, is described by a frame the
length of a record with frames=1, which is what for_records
builds and what averaging with no parameters does.
There is no zero-padding setting: interpolated lines resolve nothing new.
Methods:
| Name | Description |
|---|---|
for_records |
An averaging with each record as its own frame, which is what |
shape |
This averaging's window, as the values each frame is |
start_sample |
The index of the sample the analysis starts on, at |
stop |
When the analysis ends, in seconds on the record's clock; |
frame_bounds |
[(start, stop)] in seconds on the record's clock, one per |
fits |
Whether a record of |
most_frames |
How many whole frames a record of |
levels |
(level per frame, how many levels), the rows the frames are |
rail |
Where the flat rail's slots sit, in fractions of the visible |
statistical_dof |
The statistical degrees of freedom of a spectrum averaged |
line_correlation |
How the averaged power at lines one, two, ... apart varies |
overlap_text |
The overlap as every summary writes it, a percentage of a |
window_text |
The window as every summary writes it, with its parameter |
describe |
The settings in one phrase, the one statement of them a |
filled |
This averaging with as many frames as fit in a record of |
Attributes:
| Name | Type | Description |
|---|---|---|
hop |
int
|
Samples from the start of one frame to the start of the next. |
span |
int
|
Samples the whole analysis covers, first frame to last. |
Attributes¶
Methods:¶
for_records
classmethod
¶
for_records(samples: int, window: str = 'boxcar') -> Averaging
An averaging with each record as its own frame, which is what a stored capture is.
Everything but the window follows from the record: the frame
starts where the record starts, runs its whole length
(samples), and has no second frame to overlap with. The window
is still the user's choice (boxcar by default), so a saved
capture can be averaged again under Hann without the frames
moving.
Source code in src/visualdynamics/core/averaging.py
shape
¶
This averaging's window, as the values each frame is
multiplied by: window_shape with this frame length and this
parameter, so a caller cannot pair them wrongly.
Returns:
| Type | Description |
|---|---|
ndarray
|
One value per sample of a frame. |
Source code in src/visualdynamics/core/averaging.py
start_sample
¶
stop
¶
When the analysis ends, in seconds on the record's clock; derived, never stored.
origin is the record's first instant. start counts from the
record's beginning, in effect a sample count, whatever the clock
says. A record whose clock does not begin at zero (a truncation
keeps its measured times, an import window reads the run's) is
drawn and dragged on that clock, so every reading in seconds
adds the origin here.
Source code in src/visualdynamics/core/averaging.py
frame_bounds
¶
[(start, stop)] in seconds on the record's clock, one per
frame, overlaps included; the time history plot shades these.
origin is as in stop.
Source code in src/visualdynamics/core/averaging.py
fits
¶
Whether a record of samples samples has room for every
frame asked for.
most_frames
¶
How many whole frames a record of samples samples can hold
from start, 0 when not even one fits.
The drag handle snaps to this: a frame is added only when there is room for all of it.
Source code in src/visualdynamics/core/averaging.py
levels
¶
(level per frame, how many levels), the rows the frames are drawn on.
Frames that overlap cannot share a row without their windows running through each other, so each takes the lowest row whose last frame has already ended. With no overlap they all sit on one row; at half overlap they alternate between two. This is here, not in the overlay, because the report draws the same marks, and a figure that stacked them differently from the app would be describing a different analysis.
Source code in src/visualdynamics/core/averaging.py
rail
¶
Where the flat rail's slots sit, in fractions of the visible height: the 2-D overlay's geometry and the report's, from one set of numbers.
It is here for the same reason as levels: the app draws this
rail over a live plot and the report draws it in a canvas, and a
figure whose rail sat differently from the app's would be
describing a different analysis.
Returns a dict: levels (each frame's level), rows (how many
levels), slot (one slot's share), glyph (the window's height
inside its slot), cap (the tick under each end of a window),
baselines (each frame's baseline), band_alpha and
glyph_alpha (the fill opacities), and foot (the share of the
height the trace keeps below the rail, at least a quarter).
Source code in src/visualdynamics/core/averaging.py
statistical_dof
¶
statistical_dof(records: int = 1) -> StatisticalDof
The statistical degrees of freedom of a spectrum averaged
over these frames: statistical_dof with this averaging's
frame count, window and hop, so a caller cannot pair them
wrongly.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
int
|
How many records each channel holds, all cut the same way and pooled, as a capture saved one record per average is. Separate records share no samples, so each adds its frames' degrees of freedom in full. |
1
|
Returns:
| Type | Description |
|---|---|
StatisticalDof
|
The frames averaged, the nominal 2n and the equivalent count discounted for overlap. |
Source code in src/visualdynamics/core/averaging.py
line_correlation
¶
How the averaged power at lines one, two, ... apart varies
together: line_correlation with this averaging's window, hop
and frame count.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
most
|
int
|
The farthest separation, in lines, worked out. |
64
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The correlation at separations 0, 1, 2, ..., starting at 1. |
Source code in src/visualdynamics/core/averaging.py
overlap_text
¶
The overlap as every summary writes it, a percentage of a frame to three significant figures: '50%', '66.7%', '0%'.
window_text
¶
The window as every summary writes it, with its parameter where it takes one: 'hann', 'tukey (alpha 0.5)'.
Source code in src/visualdynamics/core/averaging.py
describe
¶
The settings in one phrase, the one statement of them a caption, a label or a summary uses, so no two say the same averaging two ways: 'hann, 50% overlap, frames of 1024 samples', with the window's parameter and the detrend when there are ('tukey (alpha 0.5), 66.7% overlap, frames of 2048 samples, mean removed from each frame').
Returns:
| Type | Description |
|---|---|
str
|
|
Source code in src/visualdynamics/core/averaging.py
StatisticalDof
dataclass
¶
How many statistical degrees of freedom an averaged autospectrum has.
frames is how many frames were averaged, every record's together.
nominal is two per frame, 2n, which is the count when no two
frames share a sample. equivalent is the count discounted for
overlap, Welch's: overlapped frames carry part of the same record,
so they vary together and an average of them steadies less than
the same number of independent frames would. With no overlap the
two are equal.
ConfidenceBand
dataclass
¶
ConfidenceBand(level: float, dof: float, lower_factor: float, upper_factor: float, lower: ndarray, upper: ndarray)
Where the true spectrum lies about an averaged estimate, at one confidence level.
level is the confidence, such as 0.9. dof is the statistical
degrees of freedom the band was worked out at. lower_factor and
upper_factor are the multiples of the estimate the band runs
between (chi_square_factors), and lower and upper are the
band itself: the estimate times each factor, the same shape and in
the same units as the estimate it was given.
Methods:
| Name | Description |
|---|---|
describe |
The band in words, as the app and the plots state it: the |
edges_text |
The two edges in decibels from the estimate, as every view |
Attributes:
| Name | Type | Description |
|---|---|---|
lower_db |
float
|
The band's lower edge in decibels from the estimate, a |
upper_db |
float
|
The band's upper edge in decibels from the estimate, a |
Attributes¶
lower_db
property
¶
The band's lower edge in decibels from the estimate, a negative number (ten times the log of the factor, as for any power quantity).
upper_db
property
¶
The band's upper edge in decibels from the estimate, a positive number.
Methods:¶
describe
¶
The band in words, as the app and the plots state it: the
confidence, the degrees of freedom it was worked out at
(dof_text) and its two edges in decibels from the estimate
(edges_text), such as '90 % band at 225.9 DOF: -0.8 / +0.9
dB'.
Source code in src/visualdynamics/core/averaging.py
edges_text
¶
The two edges in decibels from the estimate, as every view
shows them: to a tenth of a decibel, such as '-0.8 / +0.9 dB'
(db_text gives one edge).
Source code in src/visualdynamics/core/averaging.py
Functions:¶
normalize_detrend
¶
The key for a detrend named any of the ways it might be.
'none', 'mean' or 'linear', case-insensitive, or the aliases 'constant' and 'off'; None means 'none'. An unknown name raises ValueError.
Source code in src/visualdynamics/core/averaging.py
normalize_window
¶
The key for a window named any of the ways a file might name it.
One of scipy's names in WINDOWS, case-insensitive, or an alias
such as 'rectangle', 'uniform' or 'hanning'; None means 'boxcar'.
An unknown name raises ValueError.
Source code in src/visualdynamics/core/averaging.py
from_span
¶
from_span(averaging: Averaging, low: float, high: float, sample_rate: float, samples: int, origin: float = 0.0) -> Averaging
The averaging a dragged span means, in whole frames.
low and high are the span's edges in seconds on the record's
clock, which starts at origin, and samples is the record's
length. The frame length and the window stay as they are; a drag
sets only where the analysis starts and how many frames it holds.
The start is kept inside the record, and the frame count is at
least one and no more than fit. The 2-D region and the 3-D stage's
handles both commit through this, so they agree on what a span
means.
Source code in src/visualdynamics/core/averaging.py
window_shape
¶
The window name for a frame of length samples, as the values
each frame is multiplied by.
This is scipy's get_window, periodic (fftbins=True), the
convention for spectral work, where a frame is one period of a
signal assumed to repeat. parameter is the Tukey window's alpha or
the Kaiser window's beta, the default in WINDOW_PARAMETERS when
None; other windows ignore it. A length under one is refused.
Source code in src/visualdynamics/core/averaging.py
frame_correlation
¶
How much two frames apart hops from each other share, as their
window sees it: 0 for frames that do not overlap, 1 for a frame and
itself.
shape is the window, one value per sample of a frame, and hop
is the samples from one frame's start to the next
(Averaging.hop). The value is the window's overlap correlation,
the sum over the shared samples of one frame's weight times the
other's, over the sum of the weights squared. It is the number
Harris tabulates for each window: a Hann window at half overlap
gives 1/6, about 0.167, a rectangle at half overlap 0.5.
Source code in src/visualdynamics/core/averaging.py
statistical_dof
¶
statistical_dof(frames: int, shape: ArrayLike, hop: int, records: int = 1) -> StatisticalDof
The statistical degrees of freedom of an autospectrum averaged over
frames frames of the window shape, each hop samples after the
last, in each of records records.
The nominal count is two per frame. The equivalent count is Welch's for overlapped frames: the variance of the average is the variance of independent frames times
1 + 2 * sum over j of (1 - j/n) * c(j)**2
where n is the frame count and c(j) is the window's correlation
between frames j hops apart (frame_correlation), and the
equivalent count is 2n over that factor. Only the lags at which two
frames still share samples contribute. Records are separate pieces
of signal and share nothing, so each adds its frames' count in full.
The count is that of a line between zero and the Nyquist frequency. The line at zero (and at Nyquist, for an even frame) is real, so it carries one degree of freedom per frame rather than two. Taking out each frame's mean or trend costs the lowest lines a little more. Neither is folded in here: the count is the one the spectrum's interior lines have, which is where the band is read.
Source code in src/visualdynamics/core/averaging.py
line_correlation
¶
How much the averaged power at two lines varies together, at each separation in lines: 1 at a separation of 0, and the correlation coefficient between the two estimates after it.
A window is a product in time, so it is a convolution in frequency:
each line of a windowed frame holds some of its neighbors' power,
and the estimates at nearby lines are not independent. For a
stationary Gaussian signal whose spectrum is smooth across a few
lines, the covariance of the squared magnitudes at lines k and k + d
of frames a and b is the squared magnitude of their cross term, so
for an average of frames frames, each hop samples after the last,
the correlation at separation d is
sum over j of v(j) |W_j(d)|**2 / sum over j of v(j) |W_j(0)|**2
where W_j(d) is the transform, at the frame's d-th line, of the
window times itself shifted j hops, and v(j) is 1 at j = 0 and
2 (1 - j/frames) after it. The denominator is Welch's overlap sum
(statistical_dof), so the two agree. A rectangle without overlap
gives no correlation at all; a Hann window gives 4/9 at one line
apart and 1/36 at two for a single frame, which is the square of the
window's own adjacent-line correlation, 2/3 (Harris).
The correlation is what a sum of adjacent lines is worth: the sum of N lines of n degrees of freedom each, with these correlations, has the degrees of freedom of N n / (1 + 2 sum over d of (1 - d/N) r(d)), about 1.8 times fewer than N n for eight lines of a Hann window.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
shape
|
array_like
|
The window, one value per sample of a frame. |
required |
hop
|
int
|
Samples from one frame's start to the next ( |
required |
frames
|
int
|
How many frames are averaged. |
1
|
most
|
int
|
The farthest separation, in lines, worked out. |
64
|
Returns:
| Type | Description |
|---|---|
ndarray
|
The correlation at separations 0 to the last one at or above
|
Source code in src/visualdynamics/core/averaging.py
chi_square_factors
¶
(lower, upper): the factors the true spectrum lies between, as
multiples of an estimate with dof statistical degrees of freedom,
at confidence level.
An averaged autospectrum is the true one times a chi-square variable
over its degrees of freedom, so the true value lies between
dof / chi2(1 - a/2) and dof / chi2(a/2) times the estimate, where a
is one minus the level and chi2(p) is the chi-square quantile at p.
The band is not symmetric: the factor above is the larger, in
decibels as well as in ratio, because an average of few frames
reads low more often than it reads high. dof need not be a whole
number, since an equivalent count seldom is.
Examples: at two degrees of freedom (one frame) the 90 percent band runs from 0.334 to 19.5 times the estimate; at 120 it runs from 0.819 to 1.25.
Source code in src/visualdynamics/core/averaging.py
db_text
¶
One edge of a band in decibels, signed, to a tenth of a decibel: '-0.8 dB'. A band's edge is known to far better, but a hundredth of a decibel is not a difference anyone reads off a spectrum, and the Averaging view, the confidence reading and the status line all show this one figure.
Source code in src/visualdynamics/core/averaging.py
dof_text
¶
A count of statistical degrees of freedom as it is shown: four significant figures, so an equivalent count reads 225.9 and a whole one 120, and a whole number from the point four figures would round to 10,000 up, 12500 rather than 1.25e+04; the same figure in the Averaging view, the confidence reading and a plot's legend.
Source code in src/visualdynamics/core/averaging.py
confidence_bands
¶
confidence_bands(estimate: ArrayLike, dof: float, levels: Sequence[float] = CONFIDENCE_LEVELS) -> list[ConfidenceBand]
The chi-square confidence band on an averaged autospectrum, one
per level in levels, 90 and 95 percent by default.
estimate is the averaged spectrum's values, any shape, and is not
converted: the band comes back in whatever units it was given,
because each edge is the estimate times a pure number. dof is the
statistical degrees of freedom the estimate was averaged to,
normally the equivalent count from statistical_dof. The band
belongs to an autospectrum (a PSD, or the squared magnitude of a
spectrum), not to a cross-spectrum or a transfer function, whose
errors follow other laws.