ezmsg.baseproc.util.segment#
Detect breaks in a stream’s time base and number its segments.
A source stamps every message with
segment, and changes the value on
the first message that does not continue the previous one. Stateful processors
reset when they see it change (BaseStatefulProcessor._reset_segment), so
gap detection happens once, at the source, instead of in every processor.
Functions
- end_of_stream(message)[source]#
The message a source emits after
message, its last, to say its stream has ended.For a source that knows: a file reader at the end of the file, or at the end of a section before a gap it knows about. Downstream units see the new
segmentand flush what they hold now, instead of when (or if) more data arrives.The result has
message’s layout,config_id, key and attrs, but no samples along its stream dimension. ALinearAxisthere is moved to just aftermessage’s last sample; aCoordinateAxisthere is emptied. Itssegmentismessage.segment + 1, and nothing else may share it: a source keeping its own count continues it from the result, so that its next message, even data that continues in time, takesresult.segment + 1or more. A source usingSegmentTrackercallsSegmentTracker.end_of_stream()instead.
- find_gaps(timestamps, period, tolerance)[source]#
Indices of the samples that do not follow the sample before them.
Sample
ifollows samplei - 1whentimestamps[i] - timestamps[i - 1]is withintoleranceofperiod. A step backward breaks the run just as a jump forward does. Use the result to split a chunk before it is given aLinearAxis, which cannot describe a gap:np.split(data, find_gaps(...)).- Parameters:
timestamps (
ArrayLike) – Per-sample times, in seconds.period (
float) – Expected time between samples, in seconds (1 / fs).tolerance (
float) – Largest deviation fromperiod, in seconds, that is still continuous. Size it to the jitter of the timestamps, not to the sample period: a slip of a sample or two costs a filter far less than the reset a reported gap causes.
- Return type:
- Returns:
Sorted indices in
[1, len(timestamps)); empty when the chunk is continuous.
Classes
- class SegmentTracker(tolerance)[source]#
Bases:
objectNumbers the segments of one stream from the offsets of its chunks.
Counting starts at 0 and goes up by one per break.
Call
update()once per chunk, in order, and put the result in the message’ssegment. Each chunk is compared with the one before it only, so neither clock drift nor one late chunk accumulates into a false gap. Callbump()for a break the offsets cannot show, such as a device reopened with known data loss. A changed configuration alone is not a break: it changesconfig_id, not the segment. End a segment withend_of_stream(), not the module function, so that the next chunk is numbered after the marker.- Parameters:
tolerance (
float) – Largest difference, in seconds, between a chunk’s offset and where the previous chunk ended that is still continuous. Two clock-stamped chunks can each be off by the source’s jitter, so allow twice that.
- update(offset, n_samples, gain)[source]#
Account for a chunk of
n_samplesstarting atoffset; return its segment.