You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
77 lines
3.6 KiB
77 lines
3.6 KiB
# rs-latency-tool Sample
|
|
|
|
## Overview
|
|
|
|
The goal of this sample to show how we could estimate visual latency with computer vision.
|
|
|
|
> This method has a lot of unknowns and should not serve substitute to proper latency testing with dedicated equipment, but can offer some insights into camera performance and provide framework for comparison between devices / configurations.
|
|
|
|
## Why is Latency Important?
|
|
Visual latency (for our use-case) is defined as the time between an event and when it is observed as a frame in the application. Different types of events have different ranges of acceptable latency.
|
|
|
|
|
|
USB webcams usually provide latencies in the order of tens-hundreds of milliseconds, since the video data is expensive to transmit, decode and process.
|
|
|
|
## How this sample works?
|
|
|
|
The demo relies on the idea that if you look at a clock and a video-stream of the same clock side-by-side, the clock in the video will "lag" behind the real one by exactly the visual latency:
|
|
<p align="center"><img src="res/1.png" /></p>
|
|
|
|
The demo will encode current clock value into binary form and render the bits to screen (circle = bit is on):
|
|
<p align="center"><img src="res/2.PNG" /></p>
|
|
|
|
The user is asked to point the camera back at the screen to capture the pattern.
|
|
|
|
|
|
Next, we will use [Hough Transform](https://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/hough_circle/hough_circle.html) to identify sent bits in the `rs2::frame` we get back from the camera (marked as black squares)
|
|
<p align="center"><img src="res/3.png" /></p>
|
|
|
|
The demo filters-out bad detections using basic 2-bit [Checksum](https://en.wikipedia.org/wiki/Checksum).
|
|
|
|
Once it detects bits and decodes the clock value embedded in the image, the sample compares it to the clock value when `rs2::frame` was provided by the SDK.
|
|
|
|
## Implementation Details
|
|
|
|
To make sure expensive detection logic is not preventing us from getting the frames in time, detection is being done on a seperate thread. Frames are being passed to this thread, alongside their respective clock measurements, using a concurrent queue.
|
|
We ensure that the queue will not spill, by emptying it after each successful or unsuccessful detection attempt.
|
|
Please refer to [Frame Buffer Management](https://github.com/IntelRealSense/librealsense/wiki/Frame-Buffering-Management-in-RealSense-SDK-2.0) for further information.
|
|
|
|
## Controlling the Demo
|
|
|
|
Uncomment one of the following lines to select a configuration:
|
|
```cpp
|
|
//cfg.enable_stream(RS2_STREAM_COLOR, RS2_FORMAT_BGR8);
|
|
//cfg.enable_stream(RS2_STREAM_COLOR, 640, 480, RS2_FORMAT_BGR8, 30);
|
|
//cfg.enable_stream(RS2_STREAM_COLOR, 1920, 1080, RS2_FORMAT_BGR8, 30);
|
|
//cfg.enable_stream(RS2_STREAM_INFRARED, 640, 480, RS2_FORMAT_Y8);
|
|
//cfg.enable_stream(RS2_STREAM_INFRARED, 1280, 720, RS2_FORMAT_Y8);
|
|
```
|
|
You can add more than one stream to check how multiple streams are affecting each other. The demo will however just use the first frame out of every frame-set.
|
|
|
|
|
|
This method will not work for the depth stream.
|
|
**D400** will produce readable results for both `RS2_STREAM_COLOR` and `RS2_STREAM_INFRARED`.
|
|
|
|
|
|
In addition, if you want to run this demo with a regular web-cam, use the following code instead of using the `pipeline` object:
|
|
```cpp
|
|
context ctx;
|
|
auto dev = ctx.query_devices().front();
|
|
auto sensor = dev.query_sensors().front();
|
|
auto sps = sensor.get_stream_profiles();
|
|
stream_profile selected;
|
|
for (auto& sp : sps)
|
|
{
|
|
if (auto vid = sp.as<video_stream_profile>())
|
|
{
|
|
if (vid.format() == RS2_FORMAT_BGR8 &&
|
|
vid.width() == 640 && vid.height() == 480 &&
|
|
vid.fps() == 30)
|
|
selected = vid;
|
|
}
|
|
}
|
|
sensor.open(selected);
|
|
syncer pipe;
|
|
sensor.start(pipe);
|
|
```
|