ezmsg.xdf.iter

Functions

labels_from_strm(strm)

Parameters:

strm (dict)

Return type:

list[str]

Classes

class XDFAxisArrayIterator(*args, select, **kwargs)

Bases: XDFIterator

Parameters:

select (str)

__init__(*args, select, **kwargs)

This Iterator loads only a single stream and yields a single AxisArray object per chunk.

Parameters:

• *args

• select (str) – Unlike XDFIterator, this must be a single string, the name of the stream to select.

• **kwargs

class XDFIterator(filepath, select=None, chunk_dur=1.0, start_time=None, stop_time=None, rezero=True)

Bases: object

Parameters:

• filepath (Path | str)

• select (set[str] | None)

• chunk_dur (float)

• start_time (float | None)

• stop_time (float | None)

• rezero (bool)

__init__(filepath, select=None, chunk_dur=1.0, start_time=None, stop_time=None, rezero=True)

An Iterator that yields chunks from an XDF. A typical offline analysis might load the entire file into memory, then perform a processing step on the entire recording duration, and the next step on the entire result of the first step, and so on. This might require a tremendous amount of memory and, if one is not careful about memory layout, can be incredibly slow. An alternative procedure is to load the file into memory a chunk at a time (see Note1), then pass that chunk through the entire processing pipeline, then proceed onto the next chunk (See Note2). We create an Iterator to provide our chunks. > Note1: I have not written a true lazy-loader for XDF because it has not yet been necessary as the files are

all small. Thus, I use pyxdf.load_xdf which loads the entire raw data into memory. The processing is still done chunk-by-chunk.

> Note2: It should be possible to start on chunk[ix+1] while chunk[ix] is still going through the pipeline.

Indeed, this is (optionally) how it works online. However, the overhead of setting this up for offline analysis is not worth the gain, at least not at this stage.

Parameters:

• filepath (Path | str) – The path to the file to load and iterate over.

• select (set[str] | None) – (Optional) A set of stream names to select. If None, then all streams are selected.

• chunk_dur (float) – The duration of each chunk in seconds.

• start_time (float | None) – Start playback at this time. If rezero is True then this is relative to the file start time. If rezero is False then this is relative to the original timestamps.

• stop_time (float | None) – Truncate the playback to stop at this time. If rezero is True then this is relative to the file start time. If rezero is False then this is relative to the original timestamps.

• rezero (bool) – The absolute value of timestamps in an XDF file are useful for synchronization WITHIN file, but they are absolutely meaningless outside the exact XDF file like in an ezmsg application. Thus, by default we rezero the timestamps to start at t=0.0 for simplicity. However, there may be rare circumstances where one wants to compare the timestamps produced by ezmsg to timestamps produced by another XDF analysis tool that does not rezero. In that case, set rezero=False.

property stream_meta: list[dict] | dict

property n_chunks: int

class XDFMultiAxArrIterator(*args, force_single_sample={}, **kwargs)

Bases: XDFIterator

Parameters:

force_single_sample (set)

__init__(*args, force_single_sample={}, **kwargs)

This Iterator loads multiple streams and yields a AxisArray object per iteration, but the stream source might different between chunks.

Parameters:

• *args

• force_single_sample (set) – Use this to identify irregular-rate streams that might conceivably have more than one event within the defined chunk_dur, for which AxisArray cannot represent timestamps properly.

• **kwargs

class XDFIterator(filepath, select=None, chunk_dur=1.0, start_time=None, stop_time=None, rezero=True)

Bases: object

Parameters:

• filepath (Path | str)

• select (set[str] | None)

• chunk_dur (float)

• start_time (float | None)

• stop_time (float | None)

• rezero (bool)

__init__(filepath, select=None, chunk_dur=1.0, start_time=None, stop_time=None, rezero=True)

An Iterator that yields chunks from an XDF. A typical offline analysis might load the entire file into memory, then perform a processing step on the entire recording duration, and the next step on the entire result of the first step, and so on. This might require a tremendous amount of memory and, if one is not careful about memory layout, can be incredibly slow. An alternative procedure is to load the file into memory a chunk at a time (see Note1), then pass that chunk through the entire processing pipeline, then proceed onto the next chunk (See Note2). We create an Iterator to provide our chunks. > Note1: I have not written a true lazy-loader for XDF because it has not yet been necessary as the files are

all small. Thus, I use pyxdf.load_xdf which loads the entire raw data into memory. The processing is still done chunk-by-chunk.

> Note2: It should be possible to start on chunk[ix+1] while chunk[ix] is still going through the pipeline.

Indeed, this is (optionally) how it works online. However, the overhead of setting this up for offline analysis is not worth the gain, at least not at this stage.

Parameters:

• filepath (Path | str) – The path to the file to load and iterate over.

• select (set[str] | None) – (Optional) A set of stream names to select. If None, then all streams are selected.

• chunk_dur (float) – The duration of each chunk in seconds.

• start_time (float | None) – Start playback at this time. If rezero is True then this is relative to the file start time. If rezero is False then this is relative to the original timestamps.

• stop_time (float | None) – Truncate the playback to stop at this time. If rezero is True then this is relative to the file start time. If rezero is False then this is relative to the original timestamps.

• rezero (bool) – The absolute value of timestamps in an XDF file are useful for synchronization WITHIN file, but they are absolutely meaningless outside the exact XDF file like in an ezmsg application. Thus, by default we rezero the timestamps to start at t=0.0 for simplicity. However, there may be rare circumstances where one wants to compare the timestamps produced by ezmsg to timestamps produced by another XDF analysis tool that does not rezero. In that case, set rezero=False.

property stream_meta: list[dict] | dict

property n_chunks: int

labels_from_strm(strm)

Parameters:

strm (dict)

Return type:

list[str]

class XDFAxisArrayIterator(*args, select, **kwargs)

Bases: XDFIterator

Parameters:

select (str)

__init__(*args, select, **kwargs)

This Iterator loads only a single stream and yields a single AxisArray object per chunk.

Parameters:

• *args

• select (str) – Unlike XDFIterator, this must be a single string, the name of the stream to select.

• **kwargs

class XDFMultiAxArrIterator(*args, force_single_sample={}, **kwargs)

Bases: XDFIterator

Parameters:

force_single_sample (set)

__init__(*args, force_single_sample={}, **kwargs)

This Iterator loads multiple streams and yields a AxisArray object per iteration, but the stream source might different between chunks.

Parameters:

• *args

• force_single_sample (set) – Use this to identify irregular-rate streams that might conceivably have more than one event within the defined chunk_dur, for which AxisArray cannot represent timestamps properly.

• **kwargs