Telemetry payloads¶

BTP telemetry is designed for channels where measurements may be transmitted continuously, sometimes at medium or high rates, and where bandwidth cannot always be treated as abundant.

A telemetry producer may expose many measurements:

temperature
pressure
speed
current
voltage
vibration
...

A straightforward approach would be to serialize every sample as a self-describing text object:

{
  "temperature": 32.4,
  "pressure": 101.2,
  "speed": 1450
}

This is convenient, but it also means transmitting field names, separators and textual representations again and again even when the structure of the data does not change.

For a low-rate application over Ethernet, that overhead may be irrelevant.

For an embedded device transmitting continuously over a constrained RF link, it may not be.

BTP therefore separates two things:

what the data means
        |
        v
      schema


current measured values
        |
        v
    TELEMETRY

The schema describes the structure once.

Telemetry messages then carry primarily the values.

This chapter specifies the logical payload of messages whose envelope type is TELEMETRY. The envelope, CRC, fragmentation and transport limits are unchanged and are covered in The datagram.

Everything below describes the payload after reassembly. A consumer never decodes an isolated fragment.

The reference library covers this chapter in two halves. btp::messages decodes the manifest FieldRecords that describe a schema (type, unit, scale, offset, enum labels). btp::telemetry decodes and encodes a TELEMETRY sample against a schema the caller holds: the PACKED_LE / TLV_LE body, the nullable presence bitmap and the engineering conversion (raw * scale + offset). What stays outside the library is the JSON_UTF8 / CSV_UTF8 text parse and the schema storage -- the caller keeps the FieldSpec table, the same way it keeps any other discovered state (Using the library §12.4).

1. Why telemetry needs a schema¶

Compact binary telemetry creates a fundamental problem.

If a producer transmits only:

42 10 00 00 80 3f 2c 01

the receiver cannot determine by looking at those bytes whether they mean:

uint16
float32
int16

or:

four uint16 values

or something completely different.

The interpretation must come from somewhere.

One possible solution would be to define the same structure manually in both applications:

struct Telemetry {
    float temperature;
    float pressure;
    uint16_t speed;
};

The producer and consumer could then agree beforehand that every message follows that structure.

BTP deliberately does not use this model.

A native structure introduces several problems:

both applications must be compiled with prior knowledge of the layout;
changing the layout requires coordinated changes;
padding and alignment may differ;
compiler and ABI behavior may differ;
integer representation and endianness must still be defined;
directly transmitting structure memory creates platform-dependent wire data.

A wire protocol should not depend on a compiler's in-memory representation.

BTP instead explicitly defines the serialized representation and gives that representation a versioned schema.

The producer may therefore describe:

topic: motor_state
schema_version: 1

field 1:
    name: speed
    type: float32
    unit: rad/s

field 2:
    name: current
    type: int16
    unit: A
    scale: 0.01

and subsequently transmit only something conceptually equivalent to:

schema_version
speed
current

The receiver does not need the producer's C or C++ structure.

It needs the BTP schema.

This gives BTP both properties:

             self-description
                   +
             compact samples

without requiring the description to be repeated in every telemetry message.

2. Description once, values many times¶

Telemetry schemas normally change much less frequently than telemetry values.

Consider a sensor transmitting at 100 samples per second.

The following information may remain constant for hours, days or the entire firmware version:

field name
field type
engineering unit
scale
offset
field order
array size

There is little value in transmitting that information 100 times every second.

BTP therefore keeps schema metadata outside the normal telemetry stream.

Conceptually:

                     once / when changed
                            |
                            v
                    +----------------+
                    |    MANIFEST    |
                    |                |
                    | temperature    |
                    | float32        |
                    | degC           |
                    |                |
                    | pressure       |
                    | float32        |
                    | kPa            |
                    +----------------+
                            |
                            |
              +-------------+-------------+
              |             |             |
              v             v             v

         TELEMETRY      TELEMETRY      TELEMETRY
            32.4           32.5           32.6
           101.2          101.1          101.3

The description is transmitted when required.

The values can then be transmitted repeatedly in a compact representation.

This distinction becomes increasingly useful as:

the number of fields grows;
the publication rate increases;
the transport payload becomes smaller;
several devices share the same communication channel.

3. Discovering the schema¶

The consumer does not need to have every telemetry structure compiled into the application beforehand.

BTP provides runtime discovery through the manifest mechanism defined in Commands and discovery.

A consumer may request the description of a source using MANIFEST_REQUEST.

The source responds with MANIFEST_DATA, which describes its topics and their fields.

A typical exchange is:

Consumer                                  Device
   |                                         |
   |------------ MANIFEST_REQUEST ---------->|
   |                                         |
   |<------------- MANIFEST_DATA ------------|
   |                                         |
   |          schema is now known            |
   |                                         |
   |<--------------- TELEMETRY --------------|
   |<--------------- TELEMETRY --------------|
   |<--------------- TELEMETRY --------------|
   |<--------------- TELEMETRY --------------|

The manifest may describe, for example:

source_id:      0x11223344
topic_id:       0x0101
schema_version: 1
name:           motor_state
encoding:       PACKED_LE

field 1 -> left_speed
field 2 -> right_speed
field 3 -> left_current
field 4 -> right_current

Once this information is known, a telemetry sample does not need to repeat:

left_speed
right_speed
left_current
right_current
rad/s
A
float32
int16

It only needs to identify the schema version and carry the encoded values.

3.1 Runtime discovery does not require dynamic memory¶

Discovering a schema while the program is running does not imply that an implementation must dynamically allocate memory.

The protocol specifies what information may be discovered.

It does not prescribe how the implementation stores that information internally.

A desktop implementation may choose dynamic containers.

An embedded implementation may instead reserve fixed-capacity tables:

+------------------------------------+
| topic descriptors                  |
| maximum N topics                   |
+------------------------------------+

+------------------------------------+
| field descriptors                  |
| maximum M fields                   |
+------------------------------------+

The capacities can be known at compile time or configured for the application.

Therefore:

runtime discovery
        !=
dynamic allocation

A device can discover the topics offered by another node while still keeping deterministic memory usage.

The protocol and session limits define what may be represented on the wire. An implementation intended for a constrained target may operate with smaller configured capacities and reject configurations that exceed them rather than allocating memory without bound.

This separation is intentional:

BTP
 |
 +-- defines what is transmitted
 |
 +-- defines how it is interpreted
 |
 `-- does not dictate the application's memory allocator

4. Topic and schema identity¶

In a TELEMETRY message, the envelope's object_id is the topic id, and it must be non-zero.

A topic is identified by:

(source_id, topic_id)

topic_id belongs to the source's namespace.

Two different sources may therefore use the same topic_id for completely unrelated data:

source 0x1001
topic  0x0101 -> motor_state


source 0x2002
topic  0x0101 -> ambient_conditions

There is no collision because topic identity includes source_id.

The pair:

(source_id, topic_id)

remains stable across boots as long as it continues to represent the same logical topic.

4.1 Schema identity¶

The schema used to decode a particular telemetry sample is identified by:

(source_id, topic_id, schema_version)

schema_version is a uint16_le between:

1 .. 65535

Zero is invalid.

Versions are monotonic within one topic.

A version is never reused for a different layout or interpretation.

For example:

motor_state
    |
    +-- schema 1
    |      speed
    |      current
    |
    +-- schema 2
           speed
           current
           temperature

If schema version 1 meant one layout in the past, it must never later be reassigned to another layout.

After version 65535, a source that requires another schema allocates a new topic_id.

The version does not wrap to zero or back to one.

5. Changing a schema¶

A new schema_version is required whenever the interpretation of the payload changes.

Examples include changing:

field type;
field order;
element count;
engineering unit;
scale;
offset;
nullability;
encoding;
semantic meaning.

For example:

schema 1

field 1 -> speed : uint16 : rpm

cannot silently become:

schema 1

field 1 -> speed : float32 : rad/s

That would make an old consumer interpret valid bytes incorrectly.

Instead:

schema 2

field 1 -> speed : float32 : rad/s

must be published.

A presentation-only change that does not modify identity or interpretation does not necessarily require a new schema version.

6. Payload shape¶

Every TELEMETRY logical payload starts with the schema version.

Offset	Size	Field	Wire type
0	2	`schema_version`	`uint16_le`
2	variable	`encoded_body`	according to the schema encoding

Library. btp::SampleReader decodes this payload against a btp::FieldSpec array (SampleReader::schema_version() peeks the prefix; body() returns the encoded_body for a text encoding). btp::SampleWriter encodes it for PACKED_LE. See Using the library §12.4.

Conceptually:

+----------------------+-------------------------------+
| schema_version       | encoded_body                  |
| uint16_le            | variable                      |
+----------------------+-------------------------------+
0                      2

The schema definition itself is not repeated.

There are no field names, engineering units or descriptions unless the selected body encoding explicitly requires textual data.

For a fragmented message, this is the structure of the logical payload.

The schema version appears once:

fragment 0
fragment 1
fragment 2
      |
      v
reassembly
      |
      v
+------------------------------------+
| schema_version | encoded_body      |
+------------------------------------+

It does not appear at the start of every fragment.

7. The schema model¶

A schema declares at minimum:

source_id
topic_id
schema_version
name
encoding

For structured encodings it also defines an ordered list of fields.

Each field declares:

Property	Rule
`field_id`	Non-zero `uint16`, unique, stable, never reused within the topic
`name`	Human-readable; not an identity on the wire
`order`	Unique index, contiguous, starting at zero
`type`	One of the wire types defined later in this chapter
`unit`	Stable engineering unit symbol; `1` means dimensionless
`scale`	Finite multiplier; default `1`
`offset`	Finite offset; default `0`
`element_count`	`1`, a fixed count from 1 to 65535, or `variable`
`max_element_count`	Required when the count is variable
`nullable`	Boolean; default `false`

field_id provides stable identity.

order defines serialization order.

They are intentionally different concepts.

For example:

field_id = 15
order    = 0

field_id = 3
order    = 1

is valid.

A consumer must not assume:

field_id == order

7.1 Engineering conversion¶

A raw wire value is converted to its engineering representation using:

engineering_value = raw * scale + offset

For example:

raw    = 300
scale  = 0.01
offset = 0

engineering_value = 3.00

scale and offset belong to the schema.

They are not transmitted with every sample.

They are not applied to bool.

For enums, they do not change which label is selected. Enum labels always use the raw integer.

7.2 Arrays¶

A field may contain:

one scalar;
a fixed-size array;
a variable-size array.

For a fixed array:

element_count = N

and every sample contains exactly N elements when the field is present.

For a variable array, the schema provides:

element_count     = variable
max_element_count = N

and each sample carries the actual count.

If an array field is nullable, nullability applies to the whole field.

Individual elements cannot independently be null.

8. Encodings¶

BTP defines the following telemetry body encodings:

Value	Name	Use
`0x00`	`INVALID`	Reserved; never announced
`0x01`	`OPAQUE_BYTES`	Bytes whose meaning is external to BTP
`0x02`	`UTF8`	UTF-8 text with an explicit size
`0x03`	`JSON_UTF8`	One JSON value in UTF-8
`0x04`	`CSV_UTF8`	One CSV record in UTF-8
`0x05`	`PACKED_LE`	Compact fields in schema order
`0x06`	`TLV_LE`	Tag-length-value fields
`0x07`-`0xFF`	—	Reserved

PACKED_LE is the production encoding.

CSV_UTF8 is intended for tests and diagnostics and must never be used as an automatic fallback when binary encoding fails.

A producer does not change encoding without publishing a new schema version.

9. Why not just use JSON?¶

JSON remains useful because it is:

human-readable;
easy to inspect;
widely supported;
convenient during development.

But text representation has a cost.

A traditional self-describing JSON message might look like:

{
  "temperature": 32.4,
  "pressure": 101.2,
  "speed": 1450
}

A large portion of the transmitted information is not the measurement itself.

It is structure:

"temperature"
"pressure"
"speed"
:
,
"
{
}

If that same set of measurements is transmitted continuously, this structural information is repeatedly sent across the channel.

BTP avoids repeating the schema because the receiver can obtain it through the manifest.

Even BTP's JSON_UTF8 representation, which uses schema order rather than repeating field names, still represents numbers as text and carries JSON punctuation.

For example:

[32.4,101.2,1450]

is larger and more expensive to parse than the corresponding fixed-width binary representation.

With PACKED_LE, a schema containing:

temperature : float32
pressure    : float32
speed       : uint16

needs only:

4 + 4 + 2 = 10 octets

for the actual values.

Including the two-octet schema_version, the complete logical telemetry payload is:

12 octets

before the BTP envelope.

The receiver already knows what those 10 data octets mean.

The metadata was described separately.

This is the core trade:

SELF-DESCRIBING EVERY SAMPLE

name + type + value
name + type + value
name + type + value
name + type + value
        |
        v
simple, but repetitive


BTP

describe schema once
        |
        v
value + value + value
value + value + value
value + value + value
        |
        v
compact repeated telemetry

10. `PACKED_LE`¶

PACKED_LE is the most compact structured telemetry representation in BTP.

Library. btp::SampleReader (encoding btp::kEncodingPackedLe) and btp::SampleWriter implement this section, including the presence bitmap and its zero-padding rule.

Fields are serialized directly in schema order.

There is:

no padding;
no alignment;
no field name;
no field identifier;
no separator;
no native structure representation.

The receiver already knows the order and type from the schema.

10.1 Nullable fields¶

The body starts with a presence bitmap only when the schema contains nullable fields.

With K nullable fields:

bitmap_size = ceil(K / 8)

octets are used.

Bit zero of the first bitmap octet represents the first nullable field in schema order.

Bits advance from least significant to most significant and then continue in the next octet.

1 = field present
0 = field null

Unused bits in the final bitmap octet must be zero.

For example, if a schema has three nullable fields:

bit 0 -> nullable field A
bit 1 -> nullable field B
bit 2 -> nullable field C

then:

00000101

means:

A -> present
B -> null
C -> present

10.2 Field layout¶

After the presence bitmap, all present fields are encoded once in increasing order.

Non-nullable fields are always present.

A null field consumes no bytes beyond its presence bit.

A scalar occupies exactly the width of its wire type.

A fixed array contains exactly:

element_count

consecutive elements.

No count is transmitted because the receiver already knows it from the schema.

A variable array starts with:

element_count:uint16_le

followed by that many consecutive elements.

The count must not exceed:

max_element_count

The decoder knows every expected bound from the schema and must consume the body exactly.

Any of the following invalidate the sample:

truncation;
excessive array count;
missing required bytes;
unexpected trailing bytes.

11. `TLV_LE`¶

TLV_LE makes each field explicit.

Library. btp::SampleReader (encoding btp::kEncodingTlvLe) decodes this section, presenting fields in schema order and reporting an omitted nullable field as null. btp::SampleWriter does not emit TLV_LE.

The body consists of zero or more entries:

+----------------------+----------------------+----------------------+
| field_id:uint16_le   | value_size:uint16_le | value[value_size]    |
+----------------------+----------------------+----------------------+
0                      2                      4

Entries appear in increasing field_id.

A field_id never repeats in the same sample.

A nullable field is represented as null by omitting its entry.

Every non-nullable field must be present.

Variable arrays retain their:

element_count:uint16_le

prefix inside value.

An unknown field_id must still have its value_size validated so that the parser can skip the bytes safely.

However, an unknown field makes the sample incompatible.

The sample is rejected and counted rather than partially consumed.

TLV_LE allows sparse representation and explicit field identification, but costs four additional octets per field:

2 octets field_id
+
2 octets value_size

It does not eliminate schema versioning.

Changing the interpretation of an existing field still requires a new schema version.

12. Text and opaque encodings¶

12.1 `OPAQUE_BYTES`¶

OPAQUE_BYTES is an arbitrary sequence of octets.

Its meaning is external to BTP.

It may contain any value, including:

0x00
CR
LF

There is no implicit terminator.

Nullability and scaling do not apply.

12.2 `UTF8`¶

UTF8 contains well-formed UTF-8 text.

There is:

no BOM;
no implicit null terminator;
no special treatment for 0x00.

0x00 represents Unicode U+0000 and does not terminate the value.

Invalid UTF-8 invalidates the sample.

12.3 `JSON_UTF8`¶

JSON_UTF8 contains exactly one top-level JSON array.

There is one entry per schema field in schema order.

For example:

[32.4, 101.2, 1450]

An array-valued field is represented by a nested JSON array.

A null field is represented as:

null

The schema remains authoritative for field identity, type and interpretation.

JSON member names are therefore not used as the wire identity of telemetry fields.

12.4 `CSV_UTF8`¶

CSV_UTF8 contains exactly one CSV record.

There is:

no header;
no line terminator.

Fields are comma-separated and may use double quotes where required.

Only scalar fields are supported.

A bare unquoted:

null

represents a null field.

An empty field is invalid.

CSV_UTF8 is a diagnostic representation.

It is not a fallback for a failed binary encoder.

13. Wire types¶

Structured telemetry uses explicit BTP wire types. A field record (Field records) carries the type octet below.

`type`	Type	Size	Representation
`0x01`	`uint8`	1	Unsigned, little-endian
`0x02`	`uint16`	2	Unsigned, little-endian
`0x03`	`uint32`	4	Unsigned, little-endian
`0x04`	`uint64`	8	Unsigned, little-endian
`0x05`	`int8`	1	Two's complement, little-endian
`0x06`	`int16`	2	Two's complement, little-endian
`0x07`	`int32`	4	Two's complement, little-endian
`0x08`	`int64`	8	Two's complement, little-endian
`0x09`	`float32`	4	IEEE-754 bit pattern, little-endian
`0x0A`	`float64`	8	IEEE-754 bit pattern, little-endian
`0x0B`	`bool`	1	`0x00` false, `0x01` true; any other octet invalid
`0x0C`	`enum8`	1	Unsigned value; labels come from the schema
`0x0D`	`enum16`	2	Unsigned value; labels come from the schema

0x00 and 0x0E-0xFF are reserved and invalidate the field.

Library. btp::WireType names these values; btp::wire_type_width(type) returns the octet width (0 for a reserved value). See Using the library §12.4.

Values are serialized field by field.

They are not created by casting a C or C++ structure to a byte pointer.

For example, a float32 is transmitted as its IEEE-754 bit representation in little-endian order.

It is not transmitted as:

"1.500000"

and the protocol does not depend on the compiler's structure layout.

14. Value policy¶

14.1 Non-finite floating-point values¶

Non-finite values are invalid.

A producer must not emit:

NaN
+Infinity
-Infinity

A consumer rejects a sample containing one.

A missing measurement must use a nullable field rather than NaN.

These two situations intentionally remain distinct:

measurement unavailable
        |
        v
       null


invalid arithmetic result
        |
        v
       NaN
        |
        v
     rejected

Using NaN as "measurement absent" would make those conditions indistinguishable.

14.2 Boolean values¶

A boolean accepts only:

0x00 -> false
0x01 -> true

Any other byte is invalid.

14.3 Unknown enum values¶

An unknown enum value is recoverable.

Because the integer width is already known, an unknown label does not shift the remaining payload.

The receiver:

keeps the raw integer;
marks the value as an unknown enum;
does not invent a label;
reports the condition;
continues processing the rest of the sample.

This is the only recoverable telemetry payload error.

Library. btp::SampleValue hands back the raw integer (i64() / u64()). Whether that integer names a known label is a lookup against the field's enum entries, which the caller holds (from btp::messages); btp::telemetry never sees them, so "unknown enum" is the caller's determination and its counter, the same line drawn for other cross-record semantics.

14.4 Structural errors¶

Structural errors invalidate the entire logical sample.

Examples include:

payload shorter than two octets;
truncated field;
invalid nullable bitmap;
oversized variable array;
TLV size mismatch;
missing required field;
trailing unconsumed bytes.

There is:

no padding;
no implicit terminator;
no default value for missing bytes;
no partial structural decode.

15. Unknown schemas¶

A valid BTP frame may carry a telemetry schema that the consumer does not currently know.

This is not a frame error.

For example:

valid envelope
valid CRC
source_id       = 0x11223344
topic_id        = 0x0101
schema_version  = 7

but the consumer may only know:

schema_version = 6

The consumer must not guess that version 7 is "probably similar".

It must not:

decode using version 6;
infer field positions;
search fields by name;
assume appended fields;
partially publish the sample.

Instead:

TELEMETRY schema 7
        |
        v
schema unknown
        |
        +----> reject sample
        |
        `----> refresh manifest

The consumer should request a catalog refresh and obtain the current schema through MANIFEST_REQUEST.

At minimum, the rejection should record:

source_id
topic_id
received schema_version
reason

and increment the appropriate rejection counter.

The BTP frame itself remains structurally valid.

The telemetry sample could simply not be interpreted.

16. Schema caching¶

A consumer does not need to request a manifest before every sample.

Once a schema is known, it may be cached.

A typical lifecycle is:

1. connect

2. obtain manifest

3. cache:
       source
       topics
       schema versions
       field descriptions

4. receive telemetry

5. decode using cached schema

6. configuration changes

7. unknown/new schema observed

8. refresh manifest

The manifest includes a config_revision so that consumers can detect whether cached descriptors remain current.

This keeps normal telemetry traffic compact while still allowing devices to evolve their exposed configuration.

17. Publishing and fragmentation¶

A frequently transmitted sample should ideally fit inside one frame of the most restrictive transport on its path.

For ESP-NOW, the BTP payload available to a frame is:

210 octets

The first two telemetry octets contain:

schema_version

leaving up to:

208 octets

for the encoded body of a non-fragmented sample.

This is one reason compact telemetry representation matters.

For example, if the same RF channel is carrying:

telemetry
commands
command responses
terminal input
terminal output
control traffic

unnecessary telemetry overhead consumes capacity that could otherwise be used by the other message classes.

Larger telemetry samples use the normal fragmentation mechanism described in Getting it across the link.

A consumer must not decode or publish an isolated fragment.

Processing is:

fragment 0
fragment 1
fragment 2
    |
    v
complete reassembly
    |
    v
telemetry validation
    |
    v
schema decode
    |
    v
publish sample

An incomplete, expired or inconsistent fragmented message is discarded as a whole.

18. Client binding¶

After telemetry has been decoded, applications need a stable way to refer to individual measurements.

A scalar datum is identified by:

(source_id, topic_id, field_id)

An array element additionally includes its index:

(source_id, topic_id, field_id, element_index)

The field name is not the identity.

For example:

field_id = 7
name     = "temperature"

may later receive a presentation-label correction without changing the identity of field 7.

Client applications should therefore bind to identifiers rather than human-readable labels.

18.1 Variable arrays¶

If a variable array does not contain a previously bound element index in the current sample, the binding temporarily has no value.

The client must not:

reuse the previous sample;
fabricate zero;
extend the array artificially.

For example:

previous sample:

[10, 20, 30, 40]


current sample:

[11, 21]

a client bound to element 3 does not receive:

again.

It receives no current value.

18.2 Schema changes¶

schema_version determines how a sample is decoded.

It does not replace the stable identity represented by field_id.

If a new schema version removes a field or changes it incompatibly, the client binding becomes unavailable until reconfigured.

It never migrates automatically by:

field name;
field position;
similarity.

18.3 Presentation is not telemetry metadata¶

A client binding contains no:

chart id;
panel id;
color;
axis;
screen position;
visualization type.

Several visualizations may consume the same field.

Things such as:

label
color
axis limits
layout
aggregation
chart type

belong to client configuration.

They are not part of the telemetry wire format.

19. Worked example¶

Consider the following source and topic:

source_id:      0x11223344
topic_id:       0x0101
schema_version: 1
name:           motor_state
encoding:       PACKED_LE

The manifest describes:

order	field_id	name	type	unit	scale	element_count	nullable
0	1	`left_speed`	`float32`	`rad/s`	1	1	false
1	2	`right_speed`	`float32`	`rad/s`	1	1	false
2	3	`left_current`	`int16`	`A`	0.01	1	false
3	4	`right_current`	`int16`	`A`	0.01	1	false

Assume the current measurements are:

left_speed   =  1.5 rad/s
right_speed  = -2.25 rad/s

left_current raw  = 300
right_current raw = -40

The engineering currents are therefore:

300 * 0.01  =  3.00 A

-40 * 0.01  = -0.40 A

The complete telemetry payload is:

01 00                      schema_version = 1

00 00 c0 3f                left_speed  = 1.5f
00 00 10 c0                right_speed = -2.25f

2c 01                      left_current
                            raw 300 -> 3.00 A

d8 ff                      right_current
                            raw -40 -> -0.40 A

Continuous form:

01 00 00 00 c0 3f 00 00 10 c0 2c 01 d8 ff

There are no nullable fields, so no presence bitmap is required.

The body contains:

4 octets left_speed
4 octets right_speed
2 octets left_current
2 octets right_current
--------------------------
12 octets

plus:

2 octets schema_version

giving a complete logical telemetry payload of:

14 octets

The sample contains:

no field names
no unit strings
no separators
no terminator
no padding

Yet the receiver can interpret every byte because the manifest already provided the schema.

That is the intended telemetry model.

20. Nullable example¶

Consider a second topic:

schema_version: 3
encoding:       PACKED_LE

with:

order 0 -> speed        float32   non-nullable
order 1 -> temperature  float32   non-nullable
order 2 -> pressure     float32   nullable

Only one field is nullable.

Therefore the body begins with one bitmap octet.

Bit zero represents pressure.

If pressure is available:

00000001

followed by:

speed
temperature
pressure

If pressure is unavailable:

00000000

followed only by:

speed
temperature

No placeholder bytes for pressure are transmitted.

The schema tells the decoder where the nullable field would have appeared and what type it would have contained.

21. End-to-end example¶

The complete idea can be summarized through an unknown device joining a system.

Initially, the consumer knows only that a BTP source exists:

Consumer                               Device
   |                                      |
   |<--------------- HELLO ---------------|
   |                                      |

It does not have to contain a compiled C structure for every topic that device may expose.

The consumer requests the manifest:

   |                                      |
   |---------- MANIFEST_REQUEST --------->|
   |                                      |
   |<----------- MANIFEST_DATA -----------|
   |                                      |

The manifest describes:

topic 0x0101
name: motor_state
schema: 1
encoding: PACKED_LE

field 1: left_speed
field 2: right_speed
field 3: left_current
field 4: right_current

The consumer stores that descriptor according to its own implementation strategy.

That may be:

dynamic objects

on a desktop application, or:

fixed-capacity static tables

on an embedded target.

Telemetry can then flow efficiently:

   |                                      |
   |<----------- TELEMETRY -------------- |
   |<----------- TELEMETRY -------------- |
   |<----------- TELEMETRY -------------- |
   |<----------- TELEMETRY -------------- |
   |<----------- TELEMETRY -------------- |

Each sample carries:

schema_version + values

rather than:

name + type + unit + value
name + type + unit + value
name + type + unit + value
...

If the producer changes the layout, it publishes a new schema version.

When the consumer observes an unknown version:

   |<------ TELEMETRY schema = 2 ---------|
   |                                      |
   |          unknown schema              |
   |                                      |
   |---------- MANIFEST_REQUEST --------->|
   |                                      |
   |<----------- MANIFEST_DATA -----------|
   |                                      |
   |          schema 2 known              |

The consumer never guesses the new layout.

22. Validation order¶

After validating the BTP envelope and completing any required reassembly, a telemetry consumer processes the logical sample in the following order:

require at least two payload octets;
read schema_version;
reject version zero;
resolve exactly:

(source_id, topic_id, schema_version)

confirm that the schema encoding is supported;
validate lengths before reading any variable content;
validate presence information;
validate array counts;
validate field widths and types;
ensure parsing never reads beyond the logical payload;
apply boolean, floating-point and enum policies;
require the logical body to be consumed exactly;
apply scale and offset;
associate engineering units;
only then publish the decoded sample to the application.

A failure at any structural step discards the whole logical sample.

The only recoverable payload condition defined in this chapter is an unknown enum label whose raw numeric value remains structurally valid.

23. Summary¶

BTP telemetry separates description from data.

A producer does not need to transmit a self-describing JSON object with every measurement, and a consumer does not need to share the producer's native C or C++ structure.

Instead:

MANIFEST
    |
    | describes
    v
SCHEMA
    |
    | interprets
    v
TELEMETRY

The manifest communicates:

topic identity;
schema version;
encoding;
fields;
field types;
units;
scale;
offset;
array information;
nullability.

The telemetry stream then carries primarily:

schema_version + current values

This keeps repeated telemetry compact while still allowing devices and consumers to discover each other's data models at runtime.

The design also does not require dynamic memory.

Runtime discovery describes when information becomes known, not where it must be stored.

An implementation may use dynamic data structures or fixed-capacity static storage depending on the target platform.

For high-rate or bandwidth-constrained applications, PACKED_LE provides the most compact structured representation by encoding fields directly in schema order without names, padding, separators or alignment.

The result is a telemetry model intended to provide both:

flexibility
+
deterministic binary representation
+
compact transmission
+
runtime discoverability

without coupling the BTP wire format to any particular programming language, compiler, processor or in-memory structure.

Telemetry payloads¶

1. Why telemetry needs a schema¶

2. Description once, values many times¶

3. Discovering the schema¶

3.1 Runtime discovery does not require dynamic memory¶

4. Topic and schema identity¶

4.1 Schema identity¶

5. Changing a schema¶

6. Payload shape¶

7. The schema model¶

7.1 Engineering conversion¶

7.2 Arrays¶

8. Encodings¶

9. Why not just use JSON?¶

10. PACKED_LE¶

10.1 Nullable fields¶

10.2 Field layout¶

11. TLV_LE¶

12. Text and opaque encodings¶

12.1 OPAQUE_BYTES¶

12.2 UTF8¶

12.3 JSON_UTF8¶

12.4 CSV_UTF8¶

13. Wire types¶

14. Value policy¶

14.1 Non-finite floating-point values¶

14.2 Boolean values¶

14.3 Unknown enum values¶

14.4 Structural errors¶

15. Unknown schemas¶

16. Schema caching¶

17. Publishing and fragmentation¶

18. Client binding¶

18.1 Variable arrays¶

18.2 Schema changes¶

18.3 Presentation is not telemetry metadata¶

19. Worked example¶

20. Nullable example¶

21. End-to-end example¶

22. Validation order¶

23. Summary¶

10. `PACKED_LE`¶

11. `TLV_LE`¶

12.1 `OPAQUE_BYTES`¶

12.2 `UTF8`¶

12.3 `JSON_UTF8`¶

12.4 `CSV_UTF8`¶