You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
70 lines
3.2 KiB
70 lines
3.2 KiB
|
|
See also: [device](device.md)
|
|
|
|
|
|
# Metadata
|
|
|
|
Streams are divided into Video and Motion.
|
|
|
|
**Motion streams do not have metadata.** They stand alone, are output at a high frequency, and need to be as efficient as possible. Their frequency makes it very hard to synchronize with other streams, and any delay generated by keeping them around until they can be synchronized would be detrimental.
|
|
|
|
Video streams use metadata to describe the image. Common metadata include per-frame exposure & brightness value, and may contain any information that's relevant to the generation or handling if the image.
|
|
However, image data and metadata cannot be packaged together: the [ROS2 Image format](https://github.com/ros2/common_interfaces/blob/rolling/sensor_msgs/msg/Image.msg) cannot carry additional information besides a timestamp. The metadata must be sent separately and somehow synchronized with the image data.
|
|
|
|
|
|
## Metadata Topic
|
|
|
|
A single topic is used to communicate metadata to a client:
|
|
> `<device-topic-root>/metadata`
|
|
|
|
This topic conveys metadata for all video streams.
|
|
|
|
|
|
#### Quality of Service
|
|
|
|
- Reliability: `BEST_EFFORT`
|
|
- Durability: `VOLATILE`
|
|
|
|
|
|
#### Format
|
|
|
|
Metadata uses [flexible](../include/realdds/topics/flexible/) messages.
|
|
|
|
As such, metadata content is itself flexible and easily changed without predefined structures:
|
|
|
|
```JSON
|
|
{
|
|
"stream-name":"Color",
|
|
"header":{"frame-number":1234, "timestamp":123456789, "timestamp-domain":0},
|
|
"metadata":{"Exposure":123, "Gain":456}
|
|
}
|
|
```
|
|
|
|
* The `stream-name` is the stream with which this metadata should be synchronized
|
|
* The `header` contains important "non-metadata" metadata pertinent for proper client handling:
|
|
* `frame-number` supplied the sequential number of the frame - optional
|
|
* `timestamp` is **critical** and is the frame timestamp, same as was sent with the image - this is used to synchronize to the image
|
|
* `timestamp-domain` tells the client how to interpret the timestamp - optional (if missing, assuming it's a hardware timestamp)
|
|
* The `metadata` contains the actual customizable values for metadata, as `name`-`value` mappings: the `name` is the string name of the metadata field, and the `value` its value
|
|
|
|
No requirements are set for the metadata content.
|
|
However, some agreement is needed between the server and client if the data is to be made use of! For purposes of librealsense, the names must match those returned by `rs2_frame_metadata_to_string` and the values must be integral `long long` values.
|
|
|
|
Metadata that's missing will be marked not-there. Metadata names that're unrecognized will be ignored.
|
|
|
|
|
|
### Send Order
|
|
|
|
It is recommended that images be sent first, then metadata: because the metadata is much smaller (encompassing even a single packet), it will likely arrive before the image transfer is complete.
|
|
|
|
|
|
### Lost Messages
|
|
|
|
Since the topic is best-effort, it is possible that messages will get lost.
|
|
|
|
Or metadata may simply not get sent.
|
|
|
|
Images that cannot be synchronized to metadata (either because of non-matching timestamps or because metadata was lost) will simply have no metadata.
|
|
|
|
Without a frame-number, the client must assume frame-numbers. The timestamp domain may get lost, but video streaming will keep on working.
|