ezmsg.tools.shmem.aux_meta#
Codec for the static half of an AxisArray, carried alongside the shmem ring.
The circular buffer in shmem transports raw samples plus the little that
fits in a fixed ctypes header: dtype, shape, sample rate, and key. Everything
else an AxisArray knows – the ch
coordinate axis holding per-channel bank/elec/label, the units, the
message attrs – is dropped at the boundary. A consumer on the far side can
therefore plot the signal but cannot say what any of it is.
This module encodes that dropped metadata into a self-describing byte blob which
ShMemCircBuff publishes into its own shared-memory segment, and
which EZShmMirror decodes on the far side.
Wire format#
A pickled dict of plain Python and numpy types only – never ezmsg
classes. The two halves of a shmem link are separate processes and may be
separate environments with different ezmsg versions installed; pinning the wire
format to ezmsg’s dataclass layout would make an upgrade on one side a silent
decode failure on the other. Plain dicts cost one down-conversion and buy
version independence.
AUX_FORMAT_VERSION guards incompatible changes to the dict. Adding a key
is not one: a reader that predates it ignores what it does not know, and a
reader that expects it reads a default when an older writer omits it, so both
directions keep working. Bumping the version for an additive change would break
exactly the mixed-version pairing this format exists to support.
Axes decode to:
{"kind": "linear", "unit": str, "gain": float, "offset": float}
{"kind": "coord", "unit": str, "dims": list[str], "data": np.ndarray}
stream_dim carries the source message’s declaration of which dimension it
accumulates along, or None from a producer that declares nothing. It is
what the sink used to choose buffered_axis, recorded so a consumer can tell
the two apart – an operator may have overridden the buffered axis.
The buffered axis (normally time) is a deliberate special case: its
offset advances with every message and a coordinate time axis’s data is
wholly new each message, so including either would make the metadata change
continuously and defeat the point of a low-rate side channel. Only its static
descriptors are kept – see encode_aux().
Functions
- attrs_equal(a, b)[source]#
Cheap “have the attrs changed?” test.
Identity only.
attrsvalues are arbitrary – a numpy array’s==returns an array, and a user class’s__eq__could be arbitrarily expensive – so value equality is deliberately not attempted here. A producer that rebuilds equal attrs every message trips this check; the caller absorbs that by comparing the encoded blob before it republishes anything.
- axes_equal(a, b)[source]#
Cheap “have the axes changed?” test, for the per-message hot path.
Identity is checked before value at every level, which is what makes this affordable at kHz rates: an ezmsg processor that leaves an axis alone passes the same object through, so the common case costs one pointer comparison per axis. An element-wise comparison happens only when a producer rebuilt an axis – rare, and precisely the case we must not get wrong.
- axis_to_plain(axis, *, static_only=False)[source]#
Down-convert one ezmsg axis to plain types.
static_onlykeeps just the descriptors that do not change from message to message – used for the buffered axis, whose position along the stream is carried by the ring’s write index rather than by this blob.
- decode_aux(blob)[source]#
Inverse of
encode_aux().- Raises:
ValueError – if the blob is unreadable or was written by a format version this build does not understand.
- Parameters:
blob (bytes)
- Return type:
- axis_to_plain(axis, *, static_only=False)[source]#
Down-convert one ezmsg axis to plain types.
static_onlykeeps just the descriptors that do not change from message to message – used for the buffered axis, whose position along the stream is carried by the ring’s write index rather than by this blob.
- encode_aux(dims, axes, attrs, key, buffered_axis, stream_dim=None)[source]#
Serialize an AxisArray’s static metadata.
Returns the blob and the list of
attrskeys that were dropped for not being plain types, so the caller can log them once rather than per message.
- decode_aux(blob)[source]#
Inverse of
encode_aux().- Raises:
ValueError – if the blob is unreadable or was written by a format version this build does not understand.
- Parameters:
blob (bytes)
- Return type:
- axes_equal(a, b)[source]#
Cheap “have the axes changed?” test, for the per-message hot path.
Identity is checked before value at every level, which is what makes this affordable at kHz rates: an ezmsg processor that leaves an axis alone passes the same object through, so the common case costs one pointer comparison per axis. An element-wise comparison happens only when a producer rebuilt an axis – rare, and precisely the case we must not get wrong.
- attrs_equal(a, b)[source]#
Cheap “have the attrs changed?” test.
Identity only.
attrsvalues are arbitrary – a numpy array’s==returns an array, and a user class’s__eq__could be arbitrarily expensive – so value equality is deliberately not attempted here. A producer that rebuilds equal attrs every message trips this check; the caller absorbs that by comparing the encoded blob before it republishes anything.